[RFC PATCH] asm/generic: introduce if_nospec and nospec_barrier

[Dan Williams]

The 'if_nospec' primitive marks locations where the kernel is disabling
speculative execution that could potentially access privileged data. It
is expected to be paired with a 'nospec_{ptr,load}' where the user
controlled value is actually consumed. Architectures can optionally
implement a speculation barrier in 'if_nospec' or a speculation sync in
each 'nospec_{ptr,load}' depending what is best/feasible for the
architecture.

The pairing of if_nospec and nospec_load documents which branch
is sensitive to which load(s) in a code block. For example:

	if_nospec(foo < bar) {
		ptr = array_base + foo;
		array_max = array_base + bar;
		baz = nospec_load(ptr, array_base, array_max);
	}

Suggested-by: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Cc: Mark Rutland <mark.rutland@xxxxxxx>
Cc: Greg KH <gregkh@xxxxxxxxxxxxxxxxxxx>
Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
Cc: Alan Cox <alan@xxxxxxxxxxxxxxx>
Cc: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>
Cc: Elena Reshetova <elena.reshetova@xxxxxxxxx>
Signed-off-by: Dan Williams <dan.j.williams@xxxxxxxxx>
---

This proposal builds on Mark's nospec_load RFC here:

    https://lkml.org/lkml/2018/1/3/754

...and combines it with a suggestion from Peter to introduce if_nospec.
The goal is to both document which loads must not be speculated behind
which branches, and allow architectures like x86 that can do a single
barrier after the branch to coexist with architectures like ARM that
want to instrument each load.

 include/asm-generic/barrier.h |   31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/include/asm-generic/barrier.h b/include/asm-generic/barrier.h
index 5eba6ae0c34e..25c1b47f84b7 100644
--- a/include/asm-generic/barrier.h
+++ b/include/asm-generic/barrier.h
@@ -55,6 +55,27 @@
 #endif
 
 /**
+ * if_nospec() - block speculative execution of this branch
+ *
+ * @cond: condition that if true should barrier speculation
+ *
+ * Architectures should override the definition of nospec_barrier() to
+ * inject a speculative execution barrier for any conditional branches
+ * that might speculatively execute reads based on user controlled
+ * value.  Architectures that can more efficiently flush speculation via
+ * nospec_load can leave nospec_barrier as a nop.
+ *
+ * The expectation is that nospec_{load,ptr} are always used inside an
+ * if_nospec block so that architectures that can use a single barrier
+ * after the branch can do that once rather than per access.
+ */
+#define if_nospec(cond)	if (({ bool ret = (cond); nospec_barrier(); ret }))
+
+#ifndef nospec_barrier
+#define nospec_barrier() do { } while (0)
+#endif
+
+/**
  * nospec_ptr() - Ensure a  pointer is bounded, even under speculation.
  *
  * @ptr: the pointer to test
@@ -68,6 +89,11 @@
  * interval both under architectural execution and under speculation,
  * preventing propagation of an out-of-bounds pointer to code which is
  * speculatively executed.
+ *
+ * Architectures that can more efficiently flush speculation via
+ * nospec_barrier can define nospec_ptr as a nop.
+ *
+ * The expectation is that if_nospec and nospec_ptr are always paired.
  */
 #ifndef nospec_ptr
 #define nospec_ptr(ptr, lo, hi)						\
@@ -93,6 +119,11 @@
  * Architectures should override this to ensure that ptr falls in the [lo, hi)
  * interval both under architectural execution and under speculation,
  * preventing speculative out-of-bounds reads.
+ *
+ * Architectures that can more efficiently flush speculation via
+ * nospec_barrier can define nospec_load as a nop.
+ *
+ * The expectation is that if_nospec and nospec_load are always paired.
  */
 #ifndef nospec_load
 #define nospec_load(ptr, lo, hi)					\