Re: [PATCH] hotplug: Optimize {get,put}_online_cpus()

[Peter Zijlstra]

On Thu, Sep 26, 2013 at 06:58:40PM +0200, Oleg Nesterov wrote:
> Peter,
> 
> Sorry. Unlikely I will be able to read this patch today. So let me
> ask another potentially wrong question without any thinking.
> 
> On 09/26, Peter Zijlstra wrote:
> >
> > +void __get_online_cpus(void)
> > +{
> > +again:
> > +	/* See __srcu_read_lock() */
> > +	__this_cpu_inc(__cpuhp_refcount);
> > +	smp_mb(); /* A matches B, E */
> > +	__this_cpu_inc(cpuhp_seq);
> > +
> > +	if (unlikely(__cpuhp_state == readers_block)) {
> 
> OK. Either we should see state = BLOCK or the writer should notice the
> change in __cpuhp_refcount/seq. (altough I'd like to recheck this
> cpuhp_seq logic ;)
> 
> > +		atomic_inc(&cpuhp_waitcount);
> > +		__put_online_cpus();
> 
> OK, this does wake(cpuhp_writer).
> 
> >  void cpu_hotplug_begin(void)
> >  {
> > ...
> > +	/*
> > +	 * Notify new readers to block; up until now, and thus throughout the
> > +	 * longish synchronize_sched() above, new readers could still come in.
> > +	 */
> > +	__cpuhp_state = readers_block;
> > +
> > +	smp_mb(); /* E matches A */
> > +
> > +	/*
> > +	 * If they don't see our writer of readers_block to __cpuhp_state,
> > +	 * then we are guaranteed to see their __cpuhp_refcount increment, and
> > +	 * therefore will wait for them.
> > +	 */
> > +
> > +	/* Wait for all now active readers to complete. */
> > +	wait_event(cpuhp_writer, cpuhp_readers_active_check());
> 
> But. doesn't this mean that we need __wait_event() here as well?
> 
> Isn't it possible that the reader sees BLOCK but the writer does _not_
> see the change in __cpuhp_refcount/cpuhp_seq? Those mb's guarantee
> "either", not "both".

But if the readers does see BLOCK it will not be an active reader no
more; and thus the writer doesn't need to observe and wait for it.

> Don't we need to ensure that we can't check cpuhp_readers_active_check()
> after wake(cpuhp_writer) was already called by the reader and before we
> take the same lock?

I'm too tired to fully grasp what you're asking here; but given the
previous answer I think not.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/