Re: [PATCH resend^2] mm: increase RECLAIM_DISTANCE to 30

[David Rientjes]

On Tue, 24 May 2011, Andrew Morton wrote:

> How's that digging coming along?
> 
> I'm pretty wobbly about this patch.  Perhaps we should set
> RECLAIM_DISTANCE to pi/2 or something, to force people to correctly set
> the dang thing in initscripts.
> 

I think RECLAIM_DISTANCE as a constant is the wrong approach to begin 
with.

The distance between nodes as specified by the SLIT imply that a node with 
a distance of 30 has a relative distance of 3x than a local memory access.  
That's not the same as implying the latency is 3x greater, though, since 
the SLIT is based on relative distances according to ACPI 3.0.  In other 
words, it's perfectly legitimate for node 0 to have a distance of 20 and 
30 to nodes 1 and 2, respectively, if their memory access latencies are 5x 
and 10x greater, while the SLIT would remain unchanged if the latencies 
were 2x and 3x.

So basing zone reclaim by default off of a relative distance specified in 
the SLIT is wrong to begin with, and that's probably why we notice that 
the old value of 20 doesn't suffice on some machines anymore.

As I suggested earlier, I think it would be far better to actually measure 
the memory access latency to remote nodes at boot to determine whether to 
prefer zone reclaim or not rather than basing it off a false SLIT 
assumption.

Notice also that the machines that this patch was proposed for probably 
also didn't have a custom SLIT to begin with and so remote nodes get a 
default value of REMOTE_DISTANCE, which equaled RECLAIM_DISTANCE.  The 
same effect would have been achieved if you had decreased REMOTE_DISTANCE 
to 15.

We probably shouldn't be using SLIT distances at all within the kernel.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/