Re: [ltt-dev] Re: [PATCH] LTT for 2.5.38 1/9: Core infrastructure

From: bob (
Date: Mon Sep 23 2002 - 08:59:05 EST

Ingo Molnar writes:
> On Sun, 22 Sep 2002, bob wrote:
> > Yes this is simple code - similar to the model we use in K42. Still,
> > couple of things about the below.
> >
> > 1) the !event_wanted can be done outside the function, in a macro so
> > that the only cost if tracing is disabled is a hot cache hit on a mask
> > (not function call) - that helps with your comment:
> > > The event_wanted() filter function should be made as fast as possible.
> yes. It's a cost to be considered, but the main issue these days is the
> icache cost of inlining. So generally we are leaning towards the
> least-impact inlining cost.

mmm - that seems a reasonable trade-off.

> > 2) If you use the lockless scheme you do not need to disable interrupts.
> > In K42 we manage to do the entire log operation in 21 instructions and
> > about as many cycles (couple more for getting time). We do this from
> > user space as well, disabling interrupts precludes this model (may of
> > may not be a problem). I was really leaning hard away from even the
> > cost of making a system call and disabling interrupts. Do people on the
> > kernel dev team feel this is an acceptable cost? Is migration prevented
> > when interrupts are disabled? This is something for us to consider.
> the trace() functions runs purely in kernel-space, so doing a cli/sti is
> not a performance problem - if it can be avoided it saves a few cycles,
> but it does not have any global costs. But i dont think reliable tracing
> can be done without disabling interrupts - how do you guarantee that there
> will be no trace 'holes' due to interruption at the wrong instruction?

We do have a way of guaranteeing no 'holes' get created unless the process
is interrupted for a *very* long time or killed (which could happen) during
the logging of an event. The code is a little more complicated and does
require an atomic operation that may be more or less equivalent to the cli
cost. In K42, and other OSes I worked on, we wanted very efficient logging
from user space as well. I think there might be a place for understanding
libc, database, jvm, performance, for examples, but if we really only do
log events in kernel space then the cli/sti approach is simpler and roughly
equivalent performance.

> > 3) All trace events should not have to have the same number of data
> > words logged - though I think that's just a packaging/interface issue
> > the code below would just be placed behind macros which correctly
> > package up the right number of arguments.
> yes, agreed, this can be solved by having some sort of RLA, tightly packed
> trace buffer. Trace buffer usage is definitely one of the more important
> points.

Yes! and we also have a scheme to allowed such a packed buffer stream to be
randomaly accessed on disk (useful if you have 100Ms or Gs of data).


Robert Wisniewski
The K42 MP OS Project
Advanced Operating Systems
Scalable Parallel Systems
IBM T.J. Watson Research Center
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

This archive was generated by hypermail 2b29 : Mon Sep 23 2002 - 22:00:38 EST