Re: L1/L2 Cache Coloring from Userland ?

Linus Torvalds (torvalds@transmeta.com)
20 Sep 1997 22:19:08 GMT


In article <199709190525.WAA26683@linux.cobaltmicro.com>,
Larry McVoy <lm@cobaltmicro.com> wrote:
>In a previous life I came up with the following hack which approximates
>page coloring (my hack was in the swap system, a different area, but the
>problems were quite similar).
>
>Suppose you know all the page color stuff you need to, i.e., cache size,
>associativity, etc. So that if you could do it, you had all the info
>necessary to stuff pages in buckets and allocate them properly.
>
>It's sort of a bitch to keep track of all that stuff and Linus has shot down
>past attempts to do anything about because he doesn't want to screw up
>page allocation performance.
>
>What if: the pages were "sorted" on the free page list such that if an
>allocation occurs then the pages allocated will be colored properly.
>In other words, you don't think at alloca time, you think at free time.

This is the kind of approach I would like. Something that maybe does
not guarantee colouring, but it would try to make it more probable that
we get the right colour through hinting and nice behaviour.

The pages that would probably improve most from colouring are probably
(a) page cache and (b) anonymous page allocations. If there is some
simple way to sybtly weigh the likelyhood of allocation towards what we
want for these pages I'd be interested.

For example, one approach might be to keep the current power-of-two
buddy system, but selecting the starting point of the search depending
on the page colour hint. When we break up a larger buddy there is no
major reason why we can't select the page that suits us best rather than
selecting the one with the lowest address (or is it highest? I forget)
like we do now.

We do try to keep some pages free, so we obviously have _some_ choice in
which page to select. The trouble is to find a way of selecting the one
we want without making performance suffer too badly.

Note that the current code already _does_ have a very stupid form of
colouring: it has a rather black-and-white view of the world, though
(DMA'able vs not DMA'able). That one-bit choice could become a fuzzier
colouring choice, for example (make the colour a "distance" from what we
want, and make the DMA part of the distance be a maximum distance we
can't go around).

No, I don't know if that would work, but that's the kind of approach I
wouldn't dismiss out-of-hand..

Linus