[PATCH] sparc64: vdso: Flush the D-cache after updating the time data

[Stian Halseth]

The vDSO time data page is written through the kernel linear mapping
and read through a user mapping placed without colour alignment.  On
sun4u the L1 D-cache is virtually indexed, so when the two differ in
colour the timekeeping CPU's stores leave stale lines behind the user
alias in its own D-cache.  The seqcount does not catch this: the line
holding seq and the lines holding the clock data are refreshed
independently, so a reader sees an even, unchanged seq together with a
mix of old and new fields.

On a Sun Fire V240 (UltraSPARC IIIi, clocksource stick) this makes
clock_gettime(CLOCK_MONOTONIC) via the vDSO wrong for most calls on
the tick CPU, by multiples of the 10 ms tick and up to a second, in
about half of all processes: those that drew the other colour at exec.
The syscall is correct, so a deadline derived from the vDSO can already
be in the past when handed to the kernel; MySQL's InnoDB
log_files_governor thread spun at ~24000 futex calls/s on ETIMEDOUT
this way.

Flush the page after every update, as arm does.  The helper lives in
vma.c because asm/vdso/vsyscall.h is also compiled into the vDSO, where
asm/cacheflush.h is unavailable.  On sun4v flush_dcache_folio() is a
no-op, the caches there being physically indexed.

The sparc-specific vDSO that v7.1-rc1 replaced had the same defect
(reproduced on 6.18), so the bug is as old as the sparc vDSO, but this
fix relies on __arch_sync_vdso_time_data() and applies from v7.1-rc1.

Verified on 7.2.6: 32 probe runs across mapping colours show no vDSO
deviation, and MySQL's governor idles at 0.2% CPU across five restarts
of an unpatched build.

Fixes: 9a08862a5d2e ("vDSO for sparc")
Closes: https://github.com/sparclinux/issues/issues/94
Signed-off-by: Stian Halseth <stian@xxxxxx>
---
 arch/sparc/include/asm/vdso/vsyscall.h | 10 ++++++++++
 arch/sparc/vdso/vma.c                  |  6 ++++++
 2 files changed, 16 insertions(+)

diff --git a/arch/sparc/include/asm/vdso/vsyscall.h b/arch/sparc/include/asm/vdso/vsyscall.h
index 8bfe703fedc5..4852f122ce47 100644
--- a/arch/sparc/include/asm/vdso/vsyscall.h
+++ b/arch/sparc/include/asm/vdso/vsyscall.h
@@ -5,6 +5,16 @@
 
 #define __VDSO_PAGES 4
 
+#ifndef __ASSEMBLER__
+
+/* The user mapping may alias the kernel one in the VIPT D-cache. */
+struct vdso_time_data;
+void __arch_sync_vdso_time_data(struct vdso_time_data *vdata);
+#define __arch_sync_vdso_time_data __arch_sync_vdso_time_data
+
+#endif /* !__ASSEMBLER__ */
+
+/* The asm-generic header needs to be included after the definitions above */
 #include <asm-generic/vdso/vsyscall.h>
 
 #endif /* _ASM_SPARC_VDSO_VSYSCALL_H */
diff --git a/arch/sparc/vdso/vma.c b/arch/sparc/vdso/vma.c
index 60029d60f4d3..fe6a47af6b05 100644
--- a/arch/sparc/vdso/vma.c
+++ b/arch/sparc/vdso/vma.c
@@ -27,6 +27,12 @@
 
 unsigned int __read_mostly vdso_enabled = 1;
 
+/* Called by the timekeeping code after every update of the vDSO data. */
+void __arch_sync_vdso_time_data(struct vdso_time_data *vdata)
+{
+	flush_dcache_page(virt_to_page(vdata));
+}
+
 #ifdef	CONFIG_SPARC64
 static struct vm_special_mapping vdso_mapping64 = {
 	.name = "[vdso]"
-- 
2.43.0