Re: Possible race condition in oom-killer

[Michal Hocko]

On Sat 29-07-17 13:31:44, Tetsuo Handa wrote:
[...]
> @@ -806,6 +799,20 @@ static bool task_will_free_mem(struct task_struct *task)
>  	}
>  	rcu_read_unlock();
>  
> +	/*
> +	 * It is possible that current thread fails to try allocation from
> +	 * memory reserves if the OOM reaper set MMF_OOM_SKIP on this mm before
> +	 * current thread calls out_of_memory() in order to get TIF_MEMDIE.
> +	 * In that case, allow current thread to try TIF_MEMDIE allocation
> +	 * before start selecting next OOM victims.
> +	 */
> +	if (ret && test_bit(MMF_OOM_SKIP, &mm->flags)) {
> +		if (task == current && !task->oom_kill_free_check_raced)
> +			task->oom_kill_free_check_raced = true;
> +		else
> +			ret = false;
> +	}
> +
>  	return ret;
>  }

I was going to argue that this will not work because we could mark a
former OOM victim again after it passed exit_oom_victim but this seems
impossible because task_will_free_mem checks task->mm and that will be
NULL by that time. This is still an ugly hack and it doesn't provide any
additional guarantee. Once we merge [1] then the oom victim wouldn't
need to get TIF_MEMDIE to access memory reserves.

[1] http://lkml.kernel.org/r/20170727090357.3205-2-mhocko@xxxxxxxxxx
-- 
Michal Hocko
SUSE Labs