Re: sched: CPU #1's llc-sibling CPU #0 is not on the same node!

From: Don Morris
Date: Wed Feb 27 2013 - 07:40:17 EST

On 02/27/2013 12:11 AM, Yinghai Lu wrote:
> On Tue, Feb 26, 2013 at 8:43 PM, Yasuaki Ishimatsu
> <isimatu.yasuaki@xxxxxxxxxxxxxx> wrote:
>> 2013/02/27 13:04, Yinghai Lu wrote:
>>> On Tue, Feb 26, 2013 at 7:38 PM, Yasuaki Ishimatsu
>>> <isimatu.yasuaki@xxxxxxxxxxxxxx> wrote:
>>>> 2013/02/27 11:30, Yinghai Lu wrote:
>>>>> Do you mean you can not boot one socket system with 1G ram ?
>>>>> Assume socket 0 does not support hotplug, other 31 sockets support hot
>>>>> plug.
>>>>> So we could boot system only with socket0, and later one by one hot
>>>>> add other cpus.
>>>> In this case, system can boot. But other cpus with bunch of ram hot
>>>> plug may fails, since system does not have enough memory for cover
>>>> hot added memory. When hot adding memory device, kernel object for the
>>>> memory is allocated from 1G ram since hot added memory has not been
>>>> enabled.
>>> yes, it may fail, if the one node memory need page table and vmemmap
>>> is more than 1g ...
>>> for hot add memory we need to
>>> 1. add another wrapper for init_memory_mapping, just like
>>> init_mem_mapping() for booting path.
>>> 2. we need make memblock more generic, so we can use it with hot add
>>> memory during runtime.
>>> 3. with that we can initialize page table for hot added node with ram.
>>> a. initial page table for 2M near node top is from node0 ( that does
>>> not support hot plug).
>>> b. then will use 2M for memory below node top...
>>> c. with that we will make sure page table stay on local node.
>>> alloc_low_pages need to be updated to support that.
>>> 4. need to make sure vmemmap on local node too.
>> I think so too. By this, memory hot plug becomes more useful.
>>> so hot-remove node will work too later.
>>> In the long run, we should make booting path and hot adding more
>>> similar and share at most code.
>>> That will make code get more test coverage.
> Tang, Yasuaki, Andrew,
> Please check if you are ok with attached reverting patch.
> Tim, Don,
> Can you try if attached reverting patch fix all the problems for you ?

I'm sure from the discussion on how to leave in memory hotplug it
likely won't be just a clean reversion, but as a data point -- yes,
this patch does remove the problem as expected (and I don't see
any new ones at first glance... though I'm not trying hotplug yet

Don Morris

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at