Re: [PATCH RFC 1/3] mm/tlb: allow architectures to skip redundant TLB sync IPIs
From: Lance Yang
Date: Mon Dec 15 2025 - 00:48:47 EST
On 2025/12/13 16:00, Lance Yang wrote:
From: Lance Yang <lance.yang@xxxxxxxxx>
When unsharing hugetlb PMD page tables, we currently send two IPIs:
one for TLB invalidation, and another to synchronize with concurrent
GUP-fast walkers.
However, if the TLB flush already reaches all CPUs, the second IPI is
redundant. GUP-fast runs with IRQs disabled, so when the TLB flush IPI
completes, any concurrent GUP-fast must have finished.
Add tlb_table_flush_implies_ipi_broadcast() to let architectures indicate
their TLB flush provides full synchronization, enabling the redundant IPI
to be skipped.
The default implementation returns false to maintain current behavior.
Suggested-by: David Hildenbrand (Red Hat) <david@xxxxxxxxxx>
Signed-off-by: Lance Yang <lance.yang@xxxxxxxxx>
---
include/asm-generic/tlb.h | 22 +++++++++++++++++++++-
1 file changed, 21 insertions(+), 1 deletion(-)
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index 324a21f53b64..3f0add95604f 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -248,6 +248,21 @@ static inline void tlb_remove_table(struct mmu_gather *tlb, void *table)
#define tlb_needs_table_invalidate() (true)
#endif
+/*
+ * Architectures can override if their TLB flush already broadcasts IPIs to all
+ * CPUs when freeing or unsharing page tables.
+ *
+ * Return true only when the flush guarantees:
+ * - IPIs reach all CPUs with potentially stale paging-structure cache entries
+ * - Synchronization with IRQ-disabled code like GUP-fast
+ */
+#ifndef tlb_table_flush_implies_ipi_broadcast
+static inline bool tlb_table_flush_implies_ipi_broadcast(void)
+{
+ return false;
+}
+#endif
As the kernel test robot reported[1][2], the compiler is unhappy with
patch #3:
```
mm/khugepaged.c: In function 'collapse_huge_page':
1185 | if (!tlb_table_flush_implies_ipi_broadcast())>> mm/khugepaged.c:1185:14: error: implicit declaration of function 'tlb_table_flush_implies_ipi_broadcast' [-Werror=implicit-function-declaration]
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
```
I'll move tlb_table_flush_implies_ipi_broadcast() outside of
CONFIG_MMU_GATHER_RCU_TABLE_FREE in next version, making the complier
happy on architectures that don't enable that config ;)
[1] https://lore.kernel.org/oe-kbuild-all/202512142105.NXwq6dfP-lkp@xxxxxxxxx/
[2] https://lore.kernel.org/oe-kbuild-all/202512142156.cShiu6PU-lkp@xxxxxxxxx/
+
void tlb_remove_table_sync_one(void);
#else
@@ -829,12 +844,17 @@ static inline void tlb_flush_unshared_tables(struct mmu_gather *tlb)
* We only perform this when we are the last sharer of a page table,
* as the IPI will reach all CPUs: any GUP-fast.
*
+ * However, if the TLB flush already synchronized with other CPUs
+ * (indicated by tlb_table_flush_implies_ipi_broadcast()), we can skip
+ * the additional IPI.
+ *
* Note that on configs where tlb_remove_table_sync_one() is a NOP,
* the expectation is that the tlb_flush_mmu_tlbonly() would have issued
* required IPIs already for us.
*/
if (tlb->fully_unshared_tables) {
- tlb_remove_table_sync_one();
+ if (!tlb_table_flush_implies_ipi_broadcast())
+ tlb_remove_table_sync_one();
tlb->fully_unshared_tables = false;
}
}