Re: 2.6.11-rc1-mm1

[Tom Zanussi]

Karim Yaghmour writes:
 > 
 > Aaron Cohen wrote:
 > >   I've got a quick question and I just want to be clear that it
 > > doesn't have a political agenda behind it.
 > 
 > :)
 > 
 > > Here goes, why can't LTT and/or relayfs, work similar to the way
 > > syslog does and just fill a buffer (aka ring-buffer or whatever is
 > > appropriate), while a userspace daemon of some kind periodically reads
 > > that buffer and massages it.  I'm probably being naive but if the
 > > difficulty is with huge several hundred-gig files, the daemon if it
 > > monitors the buffer often enough could stuff it into a database or
 > > whatever high-performance format you need.
 > 
 > Because of the bandwidth it is not possible to do any sort of live
 > processing of any kind. The only thing the daemon can possibly do
 > is write large blocks of tracing info to disk as rapidly as possible.

I have to disagree.  Awhile back, if you remember, I posted a patch to
the LTT daemon that would monitor the trace stream in real time, and
process it using an embedded Perl interpreter, no less:

http://marc.theaimsgroup.com/?l=linux-kernel&m=109405724500237&w=2

It didn't seem to have any problems keeping up with the trace stream
even though it was monitoring all LTT event types (and a couple of
others - custom events injected using kprobes) and not doing any
filtering in the kernel, through kernel compiles, normal X traffic,
etc.  I don't know what volume of event traffic would cause this model
to break down, but I think it shows that at least some level of
non-trivial live processing is possible...

Tom

 > 
 > >  It also seems to me that Linus' nascent "splice and tee" work would
 > > be really useful for something like this to avoid a lot of unnecessary
 > > copying by the userspace daemon.
 > 
 > There is no copying by the userspace daemon. All it does is open(),
 > then mmap(), and then it sleeps until it is woken up by the ltt
 > kernel subsystem. When that happens, it only does a write() on the
 > mmaped area, tells the ltt subsystem that it commited X number of
 > sub-buffers and goes back asleep. This is all zero-copy.
 > 
 > Karim
 > -- 
 > Author, Speaker, Developer, Consultant
 > Pushing Embedded and Real-Time Linux Systems Beyond the Limits
 > http://www.opersys.com || karim@xxxxxxxxxxx || 1-866-677-4546

-- 
Regards,

Tom Zanussi <zanussi@xxxxxxxxxx>
IBM Linux Technology Center/RAS

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/