Re: Resource Limits Architecture

Jesse Pollard (pollard@tomcat.admin.navo.hpc.mil)
Thu, 16 Sep 1999 08:52:51 -0500 (CDT)

> From: Jordan Mendelson <jordy@wserv.com>
>
> A few days ago I mentioned that robust resource limits really didn't exist in
> Linux. Alan Cox (who I thought moved to the US, but uses a UK email address)
> mentioned
>
> ftp://ftp.nc.orc.ru/pub/Linux/people/saw/kernel/userbean.5
>
> which appears to be a good start for resource limits, however I still don't
> find it robust enough.
>
> I have searched high and low and can't find a good complete resource limits
> architecture in any Unix implementation. As I see it, there should be a few
> different categories of limits:
>
> Per-System : Limits which are placed on all processes
> Per-Group : Limits which are placed on all processes from a group id
> Per-User : Limits which are placed on all processes from a user id
> Per-Process Group : Limits which are placed on a group of processes (shared)
> Per-Process : Limits which are placed on a process
> Per-Thread Group : Limits which are placed on a group of threads (shared)
> Per-Thread : Limits which are placed on a thread
>
> Now of course, these will overlap since a thread is part of a process which
> has a user id a group id and runs on a system. The accepted semantics of the
> most generalized type being the upper limit of the more specific type should
> work fine, so:
>
> Per-Thread's hard limit is Per-Thread-Group
> Per-Thread-Group's hard limit is Per-Process
> Per-Process's hard limit is Per-Process-Group
> Per-Process-Group's hard limit is Per-User
> Per-User's hard limit is Per-Group (if mult groups, lowest would be hard
> limit)
> Per-Group's hard limit would be Per-System
> Per-System's hard limit would be the kernel's upper limit
>
> The commercial unices I've seen really only do per-user, per-process,
> per-thread and per-system which is probably adequate most of the time.
>
> A lot of resources themselves overlap as well. So the hard limit of network
> buffers would be total memory.
>
> There are of course some resource limits which I'd love to see implemented:
>
> Wall clock time - Time in seconds the process has physically ran, current time
> - start time. As far as I know, the only time related resource limit is CPU
> time which only measures time in the run state.
>
> Number of CPUs - Processes/Threads will be restricted to use a maximum of x
> cpus. This would be a great thing for a per-group limit, you could set users
> in group 'cpuhogs' to a maximum of 2 CPUs and the rest of the system to 4
> CPUs.
>
> Disk Space - Sure would be nice if this was all handled by a single interface.
> Right now we have a separate quota interface, I'm not sure how feasible it is
> to have disk space limits settable by the same interface as other things.
>
> Is this a bit of an overkill?
>
> Are there any unices or other OS's which implement these types of resource
> limits?

yes - look at Cray UNICOS. Besides resource limits there are also security
limits (MLS), fair-share limits (fading away now), project (project level
accounting) limits, a different set for batch and interactive (resource/project
limits). There is also an utility to generate/collate accounting records
into an extended accounting record for job level accounting.

Note: I want project level accounting separate from group access control.
The place I work has several projects that share group access to a given set
of files that is owned (and quota limited) to only one project. Files may
be created by all project members, with group access - but accounting for
each file/process is done to the specific project the user belongs to.

Now that Linux is reaching into the "supercomputer" range with beowuf
clusters, it is becoming more important to track (and justify) the usage
on large systems. If my site (a DoD shared resource center) were ever to
consider a cluser then we will have to be able to track (and report on)
this stuff.

-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/