Re: [Patch 0/8] per-task delay accounting

[Shailabh Nagar]

Andrew Morton wrote:

> Balbir Singh <balbir@xxxxxxxxxx> wrote:
>  
>
> > On Wed, Mar 29, 2006 at 09:03:14PM -0800, Andrew Morton wrote:
> >    
> >
> > > Shailabh Nagar <nagar@xxxxxxxxxxxxxx> wrote:
> > >      
> > >
> > > > Could you please include the following delay accounting patches
> > > > in -mm ?
> > > >        
> > > I'm at a loss to evaluate the suitability of this work, really.  I always
> > > am when accounting patches come along.
> > >
> > > There are various people and various groups working on various different
> > > things and there appears to be no coordination and little commonality of
> > > aims.  I worry that picking one submission basically at random will provide
> > > nothing which the other groups can work on to build up their feature.
> > >
> > > On the other hand, we don't want to do nothing until some uber-grand
> > > all-singing, all-dancing statistics-gathering infrastructure comes along.
> > >
> > > So I'm a bit stuck.  What I would like to see happen is that there be some
> > > coordination between the various stakeholders, and some vague plan which
> > > they're all happy with as a basis for the eventual grand solution.
> > >
> > > We already have various bits and pieces of statistics gathering in the
> > > kernel and it's already a bit ad-hoc.  Adding more one-requirement-specific
> > > accounting code won't improve that situation.
> > >
> > > But then, I said all this a year or two ago and nothing much has happened
> > > since then.  It's not your fault, but it's a problem.
> > >      
Yes, I agree it is a problem. We found it ourselves while developing 
this patchset. BSD accounting had
some properties we liked (like availability of stats for a process after 
it died) but the way to extend it or
get access to those stats while a process was alive wasn't all that 
good. Similarly CSA had needs like ours
but not quite the same.

Our compromise solution, prompted by your comments on getting a 
consensus for the use of a "statistics
connector" for all accounting stakeholders, was the taskstats interface, 
as described by Balbir below.

But it is not the complete solution or an attempt to get some common 
accounting infrastructure, true :-(


> > > Perhaps a good starting point would be a one-page bullet-point-form
> > > wishlist of all the accounting which people want to get out of the kernel,
> > > and a description of what the kernel<->user interface should look like. 
> > > Right now, I don't think we even have a picture of that.
> > >
> > > We need a statistics maintainer, too, to pull together the plan,
> > > coordinate, push things forwards.  The first step would be to identify the
> > > stakeholders, come up with that page of bullet-points.
> > >      
> > >
> > > Then again, maybe the right thing to do is to keep adding low-impact
> > > requirement-specific statistics patches as they come along.  
Personally, this is the approach I favor with unification happening 
piecewise, atleast as far as the
collection of statistics is concerned.

The interface for making stats available outside would seem to be more 
in need of a unified
approach since we already have a profusion of export methods, some 
legacy and some being
introduced by folks like us.


> > > But if we're
> > > going to do it that way, we need an up-front reason for doing so, and I
> > > don't know what that would be.
> > >
> > > See my problem?
> > >      

> > One of the issues we have tried to address is the ability to provide some
> > form of a common ground for all the statistics to co-exist. Various methods
> > were discussed for exchanging data between kernel and user space, genetlink
> > was suggested often and the clear winner.
> >
> > To that end, we have created a taskstats.c file. Any subsystem wanting
> > to add their statistics and sending it to user space can add their own
> > types by extending taskstats.c (changing the version number) and creating
> > their own types using genetlink. They will have to do the following
> >
> > 1. Add statistics gathering in their own subsystem
> > 2. Add a type to taskstats.c, extend it and use data from (1) and send
> >   it to user space.
> >
> > The data from various subsystems can co-exist. I feel that this could serve as
> > the basic common infrastructure to begin with and refined later (depending on
> > the needs of other people).
> >
> >    
>
> Sounds fine to me, but I'm not a stakeholder.
>
> Trolling back through lse-tech gives us:
>
> pnotify:
>  Erik Jacobson <erikj@xxxxxxx>
>
> CSA accounting/PAGG/JOB:
>  Jay Lan <jlan@xxxxxxxxxxxx>
>  Limin Gu <limin@xxxxxxxxxxxxxxxxxx>
>
> per-process IO statistics:
>  Levent Serinol <lserinol@xxxxxxxxx>
>
> ELSA:
>  Guillaume Thouvenin <guillaume.thouvenin@xxxxxxxx>
>
> per-cpu time statistics:
>  Erich Focht <efocht@xxxxxxxxxx>
>
> Scalable statistics counters with /proc reporting:
>  Ravikiran G Thirumalai <kiran@xxxxxxxxxx>
>  (Kiran feft IBM, but presumably the requirement lives on)
>  
To this list we can also add

   Microstate accounting
      Peter Chubb <peter@xxxxxxxxxxxxxxxxxx>
    I don't know if Peter is still interested in pursuing this or it 
was rejected.

> There was a long thread "A common layer for Accounting packages".  Did it
> come to a conclusion?
>  
Unfortunately, not.

> Anyway, if mostly everyone is mostly happy with what you propose then that
> it good news.
>  
It would seem like a good first step then, for me to contact the folks 
above and see if they are able to
use the interface we're proposing and modify it if needed.

--Shailabh

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/