Re: [BUG] srcu: false-positive WARN in cleanup_srcu_struct() after 78a38cbf6f20

[Zqiang]

> 
> On 8/26/26 22:13, Zqiang wrote:
> 
> > 
> > In this SRCU_SIZE_SMALL mode, when queue delayed work timer, the cblist is always no-empty.
> >  unless invoke start_poll_synchronize_srcu() to begin SRCU garce period,
> >  otherwise invoke call_srcu() or synchronize_srcu*() will insert callback.
> > 
> Right, every grace period which armed the delay_work timer is started by the real call_srcu(&kvm->srcu, &bus->rcu, __free_bus) in kvm_io_bus_register_dev(). I missed one point: the invoke work which invoked the barrier callbacks is not queued by the nearest end of grace period. It was queued by the timer of a previous end of GP, and it started only after the last GP had ended. Meanwhile the srcu_gp_end() from the last GP armed another timer even though the work was already queued. The timeline is as below:
> 
> 1. call_srcu(&kvm->srcu, &bus->rcu, __free_bus)
> 2. One end of GP comes, arms a timer.
> 3. The timer is fired and an invoke work is queued to rcu_gp_wq. The timer is disabled now.
> 4. Another call_srcu(&kvm->srcu, &bus->rcu, __free_bus)
> 5. srcu_barrier() is called and queues barrier callbacks, waits for srcu_invoke_callbacks() to invoke them.

Based on your description below, the 5.srcu_barrier() did not intercept the callback of 4.call_srcu(),
this means that 4.call_srcu() and 5.srcu_barrier() concurrent calls, or calls after 5.srcu_barrier().

The combination of srcu_barrier() and cleanup_srcu_struct() typically occurs on the module exit path.
srcu_barrier() ensures that all previously inserted callbacks complete，so therefore, theoretically,
there shouldn't be any callbacks left to execute after we finish executing srcu_barrier(). 
and of course, there are also shouldn't be any ongoing or newly started SRCU grace period and the
WARN_ON() in cleanup_srcu_struct() is designed to detect it.

If when the cleanup_srcu_struct() detects an incomplete SRCU grace period or any srcu callbacks
that have not yet been executed, this is a risk that needs to be reported, even if the
cleanup_srcu_struct() can intercept it internally.
 
Therefore, we should investigate this issue to ensure that 5.srcu_barrier() can intercept the step 4 callback.

Thanks
Zqiang


> 6. The end of GP from step 4 comes, arms another timer.
> 7. The invoke work queued in step 3 starts, srcu_invoke_callbacks() is called. It starts invoking callbacks without subtracting the cblist len field. It will call rcu_segcblist_add_len(&sdp->srcu_cblist, -len) after the invoking loop is over.
> When I debugged, there were five __free_bus(the real callbacks) and one barrier callback, so the cblist len field was 6.
> 8. Barrier callback is invoked, still the cblist len field is not subtracted as srcu_invoke_callbacks()'s invoking loop is not over.
> 9. srcu_barrier() wakes up by completion and cleanup_srcu_struct() is called before the timer armed in step 6 expires.
> At this point the cblist is physically empty (head == NULL, all seglen are 0) as all six callbacks have already been invoked. Only the cblist len field is stale(>0).
> 
> Thanks
> Sunho Park
>