Re: [discussion] Swap overcommitment recovery

Matt Agler (matagl@sypher.com)
Mon, 17 Aug 1998 16:09:46 -0400 (EDT)

On Mon, 17 Aug 1998, Rik van Riel wrote:
> > - put all processes to sleep and notify a userland identified process of
> > the situation and let that process decide what to do based on what the
> > user has in a configuration file. note, this process -must- have all
> > it's pages previously allocated or it too is simply part of the problem.
>
> This userland process takes up more space than a kernel-based
> solution. Furthermore, it still can't do without kernel support.
>
> I don't really see the advantage above kernel-based code, except
> the point of flexibility, which is something most people won't
> really care about anyway when they have to loose something
> anyway.
>
> > Such a program can also be made a module which would make the kernel
> > interface with it much easier.
>
> This could be nice indeed. But still, I don't think it is needed
> to code up solutions that large for problems that rare. Most
> folks won't configure their systems to handle OOM situations, instead,
> they'll configure them to have enough swap.
>
> > Now granted that an admin should have set up resource limits, but....let's
> > assume a new root exploit has come out as they tend to every other day on
> > bugtraq. So. A new machine killer is born and more admins rip out their
> > hair.
>
> This is why a good-enough solution in kernelspace is preferable.
> Killing something is bound to make someone unhappy, so not
> making it configurable will place the blaim on me instead of
> on some innocent sysadmin... Besides, it will force people to
> carefully make sure they don't run out of swap often, which is
> a far better solution.

Rik,

Here's a summary of my idea. I'd really be interested in any additional
comments you may have - even if it's just that you think it's too much
work for too little result. (Personally however, I think it's just these
type of up-against-the-wall extreems where linux really shines and gives
you a solution while other OSs trip up.)

-Matt

Problem: When linux runs out of swap, kswapd uses all CPU, essentially
hanging the system.

Solution: Detect when swap is about to be exhausted and recover the
machine by reducing CPU and disk usage to noncritical levels and then
running a recover procedure in userspace.

Solution Detail:

1) Add a flag to the kernel to indicate that we are in recovery mode.

2) Modify the swap subsystem to set the flag when we reach a configurable
threshhold. (ex. 90% utilization) Reset the flag when we drop back below
this threshhold. 2.1) When the flag is set, signal init to run the
nomemory action for the current runlevel.

3) Modify the vm subsystem to check this flag. If it is set, any non-root
process that attempts to use more swap by faulting on a nonexistent but
legal page (COW) is put to sleep.

4) Modify the scheduler to not wakeup these specially sleeping processes
until the flag is reset.

5) Modify init to allow for a nomemory action.

6) Add a way to set the utilization threshold at which the flag is set.

Precident:

1) ext2fs reserves space for root, so why not reserve swap for root.

2) init has actions for critical states like powerfail and ctrlaltdel
(critical because root wants us to die), so why not a nomemory action.

Benefits:

1) The kernel can avoid the consequences of optomistic memory allocation
by defering the hard decisions to userspace.

2) The kernel will never swap to death. At worst it will hang in a state
where root can login and do stuff without loosing any inprocess work. At
best the nomemory action of init will cleanup whatever has gone wrong and
the machine can continue unattended.

3) All policy is left to userspace. Processes aren't killed unless
userspace decides to kill them. Everything is fully configurable.

4) Processes that are not allocating memory are not penalized by the
kernel. (although an indescriminate nomemory action script may do so -
but that's another problem)

5) Nice defaults can be put in inittab for people who don't want to think
about it.

Notes:

1) Runaway root processes can still hang the system - but so what?

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.altern.org/andrebalsa/doc/lkml-faq.html