Re: [PATCH] drm/amdgpu: do not enter fs_reclaim under notifier_lock in lockdep training

[Christian König]

On 6/18/26 07:52, Mikhail Gavrilov wrote:
> amdgpu_lockdep_init() trains lockdep on the driver lock ordering by
> taking a chain of dummy locks in order and calling fs_reclaim_acquire()
> in the middle of it. The fs_reclaim_acquire()/fs_reclaim_release() pair
> is placed while notifier_lock (amdgpu_notifier_lock_key) is held, which
> teaches lockdep that it is legal to enter memory reclaim with the MMU
> notifier lock held:
> 
>   notifier_lock -> vram_lock -> reset_domain->sem -> reset_lock ->
>   fs_reclaim
> 
> notifier_lock is, however, acquired from the MMU notifier invalidate
> callback amdgpu_hmm_invalidate_gfx(), which mm/ runs from inside memory
> reclaim via __mmu_notifier_invalidate_range_start(). That establishes
> the mandatory reverse ordering:
> 
>   fs_reclaim -> mmu_notifier_invalidate_range_start -> notifier_lock
> 
> The two together form a cycle. It stays dormant until reclaim first
> unmaps a page covered by an amdgpu userptr interval notifier, at which
> point kswapd closes the loop and lockdep reports a false circular
> locking dependency:
> 
>   WARNING: possible circular locking dependency detected
>   kswapd0/268 is trying to acquire lock:
>    (&amdgpu_notifier_lock_key){+.+.}, at: amdgpu_hmm_invalidate_gfx
>   but task is already holding lock:
>    (mmu_notifier_invalidate_range_start){+.+.}, at: try_to_unmap_one
> 
> A lock that is taken inside an MMU notifier callback must never be held
> across a reclaiming allocation, so the fs_reclaim annotation does not
> belong inside the notifier_lock region. Drop it. The remaining chain
> still teaches the intended lock nesting.
> 
> Fixes: 1d0f5838b126 ("drm/amdgpu: Add lockdep annotations for lock ordering validation")
> Signed-off-by: Mikhail Gavrilov <mikhail.v.gavrilov@xxxxxxxxx>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_lockdep.c | 8 --------
>  1 file changed, 8 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_lockdep.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_lockdep.c
> index d5d71fd7c70d..eeb3b5007f80 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_lockdep.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_lockdep.c
> @@ -150,12 +150,6 @@ int amdgpu_lockdep_init(void)
>  	/* Level 6: Reset control lock */
>  	mutex_lock(&reset_ctl.reset_lock);
>  
> -	/*
> -	 * Mark potential memory reclaim boundary.
> -	 * GPU operations might trigger memory allocation/reclaim.
> -	 */
> -	fs_reclaim_acquire(GFP_KERNEL);
> -

This shouldn't be removed, but instead moved outside the notifier lock.

The notifier lock and vram_lock are also in incorrect order.

@Vitaly can you take care of fixing that? Thanks in advance.

Amdgpus VM eviction lock needs to be handled here as well, but has another ordering bug with fs_reclaim_acquire().

Patches to fix that are pending on the amdgpu mailing list, but I need to find time to work on them.

Thanks for the report,
Christian.

>  	/* Level 7: SRBM register access */
>  	mutex_lock(&srbm_mutex);
>  
> @@ -175,8 +169,6 @@ int amdgpu_lockdep_init(void)
>  	mutex_unlock(&grbm_idx_mutex);
>  	mutex_unlock(&srbm_mutex);
>  
> -	fs_reclaim_release(GFP_KERNEL);
> -
>  	mutex_unlock(&reset_ctl.reset_lock);
>  	up_read(&reset_domain->sem);
>  	mutex_unlock(&vram_lock);