Re: [RFC perf,x86] P4 PMU early draft

[Stephane Eranian]

Cyrill,

On Mon, Feb 8, 2010 at 7:45 PM, Cyrill Gorcunov <gorcunov@xxxxxxxxx> wrote:
> Hi all,
>
> first of all the patches are NOT for any kind of inclusion. It's not
> ready yet. More likely I'm asking for glance review, ideas, criticism.
>
> The main problem in implementing P4 PMU is that it has much more
> restrictions for event to MSR mapping. So to fit into current
> perf_events model I made the following:
>
> 1) Event representation. P4 uses a tuple of ESCR+CCCR+COUNTER
> Â as an "event". Since every CCCR register mapped directly to
> Â counter itself and ESCR and CCCR uses only 32bits of their
> Â appropriate MSRs, I decided to use "packed" config in
> Â in hw_perf_event::config. So that upper 31 bits are ESCR
> Â and lower 32 bits are CCCR values. The bit 64 is for HT flag.
>
> Â So the base idea here is to pack into 64bit hw_perf_event::config
> Â as much info as possible.
>
> Â Due to difference in bitfields I needed to implement
> Â hw_perf_event::config helper which unbind hw_perf_event::config field
> Â from processor specifics and allow to use it in P4 PMU.
>
> 2) ESCR mapping. Since ESCR MSRs are not sequential in address space
> Â I introduced a kind of "event template" structure which contains
> Â MSR for particular event. This MSRs are tupled in array for 2 values
> Â and bind to CCCR index so that HT machine with thread 0 should pick
> Â first entry and thread 1 -- second entry.
>
> Â If HT is disabled or there is an absence of HT support -- we just use
> Â only first counter all the time.
>
> Â Ideally would be to modify x86_pmu field values that way that in case
> Â of absence of HT we may use all 18 counters at once. Actually there is
> Â erratum pointing out that CCCR1 may not have "enable" bit properly working
> Â so we should just not use it at all. Ie we may use 17 counters at once.
>
> 3) ÂDependant events. Some events (such as counting retired instructions)
> Â Ârequires additional "upstream" counter to be armed (for retired instructions
> Â Âthis is tagging FSB). Not sure how to handle it properly. Perhaps at moment
> Â Âof scheduling events in we may check out if there is dependant event and
> Â Âthen we try to reserve the counter.
>
> 4) ÂCascading counters. Well, I suppose we may just not support it for some
> Â Âtime.
>
> 5) ÂSearching the events. Event templates contain (among other fields)
> Â Â"opcode" which is just "ESCR event + CCCR selector" packed so every
> Â Âtime I need to find out restriction for particular event I need to
> Â Âwalk over templates array and check for same opcode. I guess it will
> Â Âbe too slow (well, at moment there a few events, but as only array grow
> Â Âit may become a problem, also iirc some opcodes may intersect).
>
> So the scheme I had in mind was like that:
>
> 1) As only caller asking for some generalized event we pack ESCR+CCCR
> Â into "config", check for HT support and put HT bit into config as well.
>
> 2) As events go to schedule for execution need to find out proper
> Â hw_perf_event::config into proper template and set proper
> Â hw_perf_event::idx (which is just CCCR MSR number\offset) from
> Â template. Mark this template as borrowed in global p4_pmu_config
> Â space. This allow to find out quickly which ESCR MSR is to be used
> Â in event enabling routing.
>
> Â Schematically this mapping is
>
> Â Â Â Âhw_perf_event::config -> template -> hw_perf_event::idx
>
> Â And back at moment of event enabling
>
> Â Â Â Âhw_perf_event::idx -> p4_pmu_config[idx] -> template -> ESCR MSR
>
> 3) I've started unbinding x86_schedule_events into per x86_pmu::schedule_events
> Â and there I hit hardness in binding HT bit. Have to think...
>
>
> All in one -- If you have some spare minutes, please take a glance. I can't
> say I like this code -- it's overcomplicated and I fear hard to understand.
> and still a bit buggy. Complains are welcome!
>
> So any idea would be great. I'm fine in just DROPPING this approach if we may
> make code simplier. Perhaps I used wrong way from the very beginning.
>
> Stephane, I know you've been (and still is, right?) working on P4 driver
> for libpmu (not sure if I remember correctly this name). Perhaps we should
> take some ideas from there?
>
The library is called libpfm (perfmon) and libpfm4 (perf_events). It has
support for Netburst (pentium4). It does the assignment of events to
counters in user mode. I did not write the P4 support and it was a very long
time ago. I don't recall all the details, except that this was really difficult.
You may want to take a look at pfmlib_pentium4.c.

I am still actively developing libpfm4 for perf_events. The P4 support is
not available because I don't know how perf_events is going to support
it. I'll update the library once that is settled.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/