Re: Better pagecache statistics ?

[Martin J. Bligh]

>> > > Out of "Cached" value - to get details like
>> > > 
>> > > 	<mmap> - xxx KB
>> > > 	<shared mem> - xxx KB
>> > > 	<text, data, bss, malloc, heap, stacks> - xxx KB
>> > > 	<filecache pages total> -- xxx KB
>> > > 		(filename1 or <dev>, <ino>) -- #of pages
>> > > 		(filename2 or <dev>, <ino>) -- #of pages
>> > > 		
>> > > This would be really powerful on understanding system better.
>> > 
>> > to some extend it might be useful.
>> > I have a few concerns though
>> > 1) If we make these stats into an ABI then it becomes harder to change
>> > the architecture of the VM radically since such concepts may not even
>> > exist in the new architecture. As long as this is some sort of advisory,
>> > humans-only file I think this isn't too much of a big deal though. 
>> > 
>> > 2) not all the concepts you mention really exist as far as the kernel is
>> > concerned. I mean.. a mmap file is file cache is .. etc.
>> > malloc/heap/stacks are also not differentiated too much and are mostly
>> > userspace policy (especially thread stacks). 
>> > 
>> > A split in
>> > * non-file backed
>> >   - mapped once
>> >   - mapped more than once
>> > * file backed
>> >   - mapped at least once
>> >   - not mapped
>> > I can see as being meaningful. Assigning meaning to it beyond this is
>> > dangerous; that is more an interpretation of the policy userspace
>> > happens to use for things and I think coding that into the kernel is a
>> > mistake.
>> > 
>> > Knowing which files are in memory how much is, as debug feature,
>> > potentially quite useful for VM hackers to see how well the various VM
>> > algorithms work. I'm concerned about the performance impact (eg you can
>> > do it only once a day or so, not every 10 seconds) and about how to get
>> > this data out in a consistent way (after all, spewing this amount of
>> > debug info will in itself impact the vm balances)
>> 
>> Most of the issues you mention are null if you move the stats
>> maintenance burden to userspace. 
>> 
>> The performance impact is also minimized since the hooks 
>> (read: overhead) can be loaded on-demand as needed.
>> 
> 
> The overhead is - going through each mapping/inode in the system
> and dumping out "nrpages" - to get per-file statistics. This is
> going to be expensive, need locking and there is no single list 
> we can traverse to get it. I am not sure how to do this.

I made something idiotic to just walk the mem_map array and gather
stats on every page in the system. Not exactly pretty ... but useful.
Can't lay my hands on it at the moment, but Badari can ask Janet
for it, I think ;-)

M.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/