[PATCH v1] x86/mm: fix data-race in arch_tlbbatch_add_pending() when reading mm_cpumask

[Xuewen Wang]

KCSAN reports a data-race in __bitmap_or() when reading mm_cpumask(mm)
during arch_tlbbatch_add_pending(). The mm_cpumask can be concurrently
modified by other CPUs via set_bit/clear_bit (e.g., in switch_mm_irqs_off()
and leave_mm()) while cpumask_or() reads it through non-atomic bitmap
operations.

Replace the cpumask_or() call with an explicit per-word loop using
READ_ONCE() to pair with the concurrent atomic bitops on mm_cpumask.
This ensures each word is read atomically without tearing, while
keeping the write side of batch->cpumask unaffected.

Signed-off-by: Xuewen Wang <wangxuewen@xxxxxxxxxx>
---
 arch/x86/include/asm/tlbflush.h | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
index 0545fe75c3fa..f363059c4dec 100644
--- a/arch/x86/include/asm/tlbflush.h
+++ b/arch/x86/include/asm/tlbflush.h
@@ -371,8 +371,16 @@ static inline u64 inc_mm_tlb_gen(struct mm_struct *mm)
 static inline void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *batch,
 		struct mm_struct *mm, unsigned long start, unsigned long end)
 {
+	unsigned int i, nr = BITS_TO_LONGS(nr_cpu_ids);
+	const unsigned long *src = cpumask_bits(mm_cpumask(mm));
+	unsigned long *dst = cpumask_bits(&batch->cpumask);
+
 	inc_mm_tlb_gen(mm);
-	cpumask_or(&batch->cpumask, &batch->cpumask, mm_cpumask(mm));
+
+	/* mm_cpumask can be concurrently modified by atomic bitops; use READ_ONCE per word to pair. */
+	for (i = 0; i < nr; i++)
+		dst[i] |= READ_ONCE(src[i]);
+
 	batch->unmapped_pages = true;
 	mmu_notifier_arch_invalidate_secondary_tlbs(mm, 0, -1UL);
 }
-- 
2.25.1