Re: [PATCH v3 5/5] cpusets, suspend: Save and restore cpusets duringsuspend/resume
From: David Rientjes
Date: Tue May 15 2012 - 17:05:26 EST
On Tue, 15 May 2012, Peter Zijlstra wrote:
> > However, if a thread did set_mempolicy(MPOL_BIND, 2-3) where cpuset.mems
> > == node_online_map, cpuset.mems changes to 0-1, then cpuset.mems changes
> > back to node_online_map, then I believe (and implemented in the mempolicy
> > code and added the specification in the man page) that the thread should
> > be bound to nodes 2-3.
> I disagree, but alas that is done :-(
Not sure what you're disagreeing with, it only happens with
MPOL_F_STATIC_NODES or MPOL_F_RELATIVE_NODES and I've clearly defined the
behavior in the man page. I personally never had a use-case for
MPOL_F_RELATIVE_NODES but Paul Jackson asked that it be added for SGI when
we added mempolicy mode flags.
> But what happens if you unplug nodes 2-3?
Mempolicies lack hotplug notifiers entirely so it falls back to no policy
at allocation time, the rebinding of nodes in the context of cpusets only
occurs when cpuset.mems changes, not with hotplug.
> The problem is that if you have some cpusets configuration and then do a
> s/r cycle the entire configuration is wrecked because suspend
> hot-unplugs all but cpu0 and resume re-plugs the cpus.
> This destroys all masks and migrates all tasks in sets not including
> cpu0 to the root set.
> Srivatsa proposed to 'fix' this by remembering state of regular hotplug,
> to which I strongly oppose, hotplug is destructive and should be,
> there's no point in remembering state that might never be used again.
> Worse you temporarily 'break' your cpuset 'promise' to then silently
> restore it.
> The s/r resume case is special in that userspace isn't actually around
> to observe the cpus going away and coming back, also it has the
> guarantee the cpus will be coming back.
> So s/r is special and should not destroy state, hotplug should.
Why can't we leave cpuset.cpus unaltered for all cpusets during suspend?
Having a cpu set in cpuset.cpus does not mean the attached threads are
allowed to run on all of them.
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/