Re: Process Aggregates: module based support for jobs

From: Sam Watters (watters@sgi.com)
Date: Sat Jun 17 2000 - 14:42:46 EST


Andrey,

In particular, thank you for the link to the latest beancounter patch and the
design/status information. I'll patch a 2.4-test1 kernel and take a look at how
beancounters operate.

I will be very interested to see how the memory limits are handled.

Resource limits are a primary concern for the job container. This was the
driving force for implementing formalized jobs on IRIX - job accounting was a
secondary (though very important) concern. However, resource limits tend to
have more impact on the kernel code. Additionally, there was the issue of what
job limits mean in the context of a cluster job.

LANL approached us concerning the need for better accounting tools on Linux.
What they required (and other sites as well) was best met by providing a job
accounting solution. So, that was our motivation for focusing on the accounting
aspect first - LANL wanted it and it allowed us to work with them on a project
(Comprehensive System Accounting).

So, the goal for jobs is ultimately to provide resource limits. But,
circumstances dictated that job accounting be made high on our priority list.

Andrey Savochkin wrote:
>
> > > 1. How to handle the counting of shared memory pages. This has been an
> > > extremely difficult problem on IRIX to get right - in fact, it is still not done
> > > correctly.
> >
> > All VM pages would be probably left out in the first implementation;
> > it requires lots of infrastructure in the VM that just isn't there yet. They
> > aren't that critical because they can be swapped out anyways. I'm looking
> > primarily for a solution of limiting pinned down buffers (like network buffers)
> > per user/group/job/login in.
>
> It was resource limiting rather than just accounting that was the goal for
> user beancounter patch. And the first and most important part was exactly
> the accounting and limiting the amount unswappable memory allocated by the
> kernel to serve user's needs.
>
> The most important component which is already implemented in user beancounter
> patch is accounting for pages allocated to be used as page directories for
> virtual to physical address mappings. Socket buffer accounting is the first
> thing in my current todo list.
>
> Certainly, the patch touches critical parts of the kernel, and, thus, I tried
> to do it as much straightforward and efficient as possible.
>
> I recently announced to linux-kernel that user beancounter patch is
> resurrected. Here is 2.4.0-test1 port of the old patch:
> http://www.asplinux.com.sg/install/user_beancounter-IV-0002
> (oh dear, how quickly time passes!)
>
> I've also extracted bits of information describing the design and current
> status of the patch and put it as a separate web page:
> http://www.asplinux.com.sg/install/ubpatch.html

-- 
----------------------------------------
Sam Watters
SGI
watters@sgi.com
(651) 683-5647
----------------------------------------

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Jun 23 2000 - 21:00:15 EST