Re: [PATCH] tracing/timerlat: Check tlat_var for NULL in timerlat_fd_release

[Steven Rostedt]

On Thu, 5 Sep 2024 12:38:37 +0200
Tomas Glozar <tglozar@xxxxxxxxxx> wrote:

> st 4. 9. 2024 v 16:23 odesílatel Steven Rostedt <rostedt@xxxxxxxxxxx> napsal:
> >
> > When running my tests, the second one would end up deadlocking and
> > triggering lockdep. I found a way to do basically the same thing with a
> > cpumask and no added locking. I'm currently testing it and will be sending
> > out a patch later today (if it passes the tests).
> >
> > -- Steve
> >  
> 
> Oh that's unfortunate. Your cpumask patch does protect from timerlat
> trying to stop a user workload thread via kthread_stop, but I don't
> think it prevents this race in the code from the other patch:
> 
> static int timerlat_fd_release(struct inode *inode, struct file *file)
> {
>     ...
>     mutex_lock(&interface_lock);
>     <-- this can now run at the same time as tlat_var_reset(), since
> the latter is not done under interface_lock in the latest version
>     ...
> if (tlat_var->kthread)
>         <-- if tlat_var is zeroed here, we still panic on NULL dereference
> hrtimer_cancel(&tlat_var->timer);
>     ...
> }

Right, but my other patch was protecting the osn variable too, which I
don't think is necessary.

> 
> Either way, the results are much better: before, the kernel panicked
> in <10 iterations of the while loop of the reproducer; with the
> cpumask patch and the tlat_var->kthread check patch, it has been
> running on my test VM for a few hours already with no panic.

I think the hrtimer change is missing this:

diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c
index 8543d941b870..8e611dcee68a 100644
--- a/kernel/trace/trace_osnoise.c
+++ b/kernel/trace/trace_osnoise.c
@@ -259,6 +259,9 @@ static inline void tlat_var_reset(void)
 {
 	struct timerlat_variables *tlat_var;
 	int cpu;
+
+	/* Synchronize with the timerlat interfaces */
+	mutex_lock(&interface_lock);
 	/*
 	 * So far, all the values are initialized as 0, so
 	 * zeroing the structure is perfect.
@@ -269,6 +272,7 @@ static inline void tlat_var_reset(void)
 			hrtimer_cancel(&tlat_var->timer);
 		memset(tlat_var, 0, sizeof(*tlat_var));
 	}
+	mutex_unlock(&interface_lock);
 }
 #else /* CONFIG_TIMERLAT_TRACER */
 #define tlat_var_reset()	do {} while (0)

I'll add it and start testing it. I haven't pushed to Linus yet.

-- Steve