Re: Fix powerTOP regression with 2.6.39-rc5

From: Michael Rubin
Date: Tue May 17 2011 - 03:15:53 EST

This thread is unsettling for at least one customer of kernel tracing.

Google spent a lot of time writing their own kernel tracing
infrastructure ktrace. It was working just fine for us but we
abandoned it in order to work with the community. We evaluated perf,
ftrace and LTTNG and opted for ftrace. We saw it as a efficient kernel
system that had been around long enough we could depend on it to
continue to be around. Also we could share our work this way. We
started sending patches and tried to be good Open Source citizens.

Switching from ktrace to ftrace was very painful for us. In order to
monitor the tens of thousands of computers Google maintains we wrote a
lot of tools on top of ftrace that are very specific to Google's user
space technology. What was not fun was to ask engineers to make
changes to their working systems to accommodate the switch from ktrace
to ftrace. We are not going to do this again in the near future.

On Wed, 2011-05-11 at 23:51 +0200, Ingo Molnar wrote:
> This is sadly how 'splitting a small pond into two' tends to
> work out in practice: both halves stink a little bit more than they would if
> they were kept together ;-)

I heavily agree with this statement. Having duplicate solutions
doesn't help anything.

On Wed, 2011-05-11 at 23:51 +0200, Ingo Molnar wrote:
> - If the perf UI/API/ABI design is better then ftrace can be migrated to it
> and we can use the perf APIs to do more tooling goodness.
> Everyone will be happy.

But I don't agree here. Everyone will _not_ be happy. Existing
customers will have to migrate to a new system, API or even worse new

On Wed, 2011-05-11 at 23:51 +0200, Ingo Molnar wrote:
> So i really prefer the 'apps are using us' situation we are in today, and not
> breaking them is a *small* price to pay and it is a very small loss of the near
> infinite degrees of development freedom we still enjoy in the kernel.

I really prefer the 'apps are using us' situation too. Both as someone
who is working with ftrace development and also working with kernel
tracing customers.

What is the plan for customers going forward? Is it going to involve
removing ftrace in favor for perf? Removing perf in favor for ftrace?
We love perf and don't want to see it go away either. We tend to use
the two systems differently. Do customers basically have to wait a few
years to see not only which system wins but which ones stays on top?

I apologize if this is obvious to others but I am confused.

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at