Re: [BUG] KCSAN: data-race in process_one_work / process_one_work

[Mirsad Todorovac]

On 8/18/23 10:10, Mirsad Todorovac wrote:
> On 8/17/23 23:59, Tejun Heo wrote:
> > On Thu, Aug 17, 2023 at 09:24:28PM +0200, Mirsad Todorovac wrote:
> > > Hi,
> > >
> > > This is your friendly bug reporter.
> > >
> > > The environment is vanilla torvalds tree kernel on Ubuntu 22.04 LTS and on a Ryzen 7950X
> > > assembled box.
> > >
> > > The kernel reports G taint "properietary module loaded", but I know of no module since
> > > the boot state when the status was "Not tainted".
> > >
> > > Please find the complete dmesg output (or what's left in the ring buffer) and lshw output
> > > attached.
> > >
> > > Here is the dmesg output excerpt:
> > >
> > > [ 6849.221584] ==================================================================
> > > [ 6849.221607] BUG: KCSAN: data-race in process_one_work / process_one_work
> > >
> > > [ 6849.221635] write to 0xffff9b7440151398 of 8 bytes by task 6364 on cpu 7:
> > > [ 6849.221647]  process_one_work+0x504/0x930
> > > [ 6849.221660]  worker_thread+0x311/0x7e0
> > > [ 6849.221673]  kthread+0x18b/0x1d0
> > > [ 6849.221683]  ret_from_fork+0x43/0x70
> > > [ 6849.221695]  ret_from_fork_asm+0x1b/0x30
> > >
> > > [ 6849.221712] read to 0xffff9b7440151398 of 8 bytes by task 5586 on cpu 28:
> > > [ 6849.221724]  process_one_work+0x4e8/0x930
> > > [ 6849.221737]  worker_thread+0x519/0x7e0
> > > [ 6849.221749]  kthread+0x18b/0x1d0
> > > [ 6849.221759]  ret_from_fork+0x43/0x70
> > > [ 6849.221770]  ret_from_fork_asm+0x1b/0x30
> > >
> > > [ 6849.221786] value changed: 0x00000000000154d0 -> 0x00000000000154d1
> > >
> > > [ 6849.221802] Reported by Kernel Concurrency Sanitizer on:
> > > [ 6849.221811] CPU: 28 PID: 5586 Comm: kworker/u64:0 Tainted: G             L     6.5.0-rc6-net-cfg-kcsan-00038-g16931859a650 #35
> > > [ 6849.221825] Hardware name: ASRock X670E PG Lightning/X670E PG Lightning, BIOS 1.21 04/26/2023
> > > [ 6849.221833] Workqueue: events_unbound wq_barrier_func
> > > [ 6849.221846] ==================================================================
> >
> > That looks like the PENDING bit. The setting is atomic. Maybe there's a raw
> > read? Can you map the addresses to lines?
> >
> > Thanks.
>
> Good news:
>
> [ 1863.554079] ==================================================================
> [ 1863.554118] BUG: KCSAN: data-race in process_one_work / process_one_work
>
> [ 1863.554142] write to 0xffff963d99d79998 of 8 bytes by task 5394 on cpu 27:
> [ 1863.554154] process_one_work (kernel/workqueue.c:2598)
> [ 1863.554166] worker_thread (./include/linux/list.h:292 kernel/workqueue.c:2752)
> [ 1863.554177] kthread (kernel/kthread.c:389)
> [ 1863.554186] ret_from_fork (arch/x86/kernel/process.c:145)
> [ 1863.554197] ret_from_fork_asm (arch/x86/entry/entry_64.S:312)
>
> [ 1863.554213] read to 0xffff963d99d79998 of 8 bytes by task 5450 on cpu 12:
> [ 1863.554224] process_one_work (kernel/workqueue.c:2598)
> [ 1863.554235] worker_thread (./include/linux/list.h:292 kernel/workqueue.c:2752)
> [ 1863.554247] kthread (kernel/kthread.c:389)
> [ 1863.554255] ret_from_fork (arch/x86/kernel/process.c:145)
> [ 1863.554266] ret_from_fork_asm (arch/x86/entry/entry_64.S:312)
>
> [ 1863.554280] value changed: 0x0000000000001766 -> 0x000000000000176a
>
> [ 1863.554295] Reported by Kernel Concurrency Sanitizer on:
> [ 1863.554303] CPU: 12 PID: 5450 Comm: kworker/u64:1 Tainted: G             L     6.5.0-rc6+ #44
> [ 1863.554314] Hardware name: ASRock X670E PG Lightning/X670E PG Lightning, BIOS 1.21 04/26/2023
> [ 1863.554322] Workqueue: btrfs-endio btrfs_end_bio_work [btrfs]
> [ 1863.554941] ==================================================================
>
>
>      lockdep_invariant_state(true);
> :    pwq->stats[PWQ_STAT_STARTED]++;
>      trace_workqueue_execute_start(work);
>      worker->current_func(work);

Just as an exercise, this seems to work, but it is probably not very clean and not very scalable:

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 800b4208dba9..237e5800fbad 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -2595,7 +2595,7 @@ __acquires(&pool->lock)
         * workqueues), so hiding them isn't a problem.
         */
        lockdep_invariant_state(true);
-       pwq->stats[PWQ_STAT_STARTED]++;
+       atomic_inc((atomic_t *) &pwq->stats[PWQ_STAT_STARTED]);
        trace_workqueue_execute_start(work);
        worker->current_func(work);
        /*

--
Best regards,
Mirsad Todorovac