[PATCH v2 1/5] generic_pt: allow missing sw bit in DMA_INCOHERENT case

[Daniel Drake]

When working with a iommu with PT_FEAT_DMA_INCOHERENT set, generic_pt
will attempt to use a spare "SW" bit in the hardware page tables to
denote when a thread has flushed the CPU cache after modifying an entry.
This means that other threads know that they are not working with
cached/unflushed data, if they come across the same entry.

In the case where no SW bit is implemented, an undefined reference to
__pt_no_sw_bit() is created, causing a linker error in order to prompt
the developer to implement this feature.

For BCM2712 IOMMU, no SW bit is available at the hardware level, so we now
need to permit this case.

Introduce a PT_FEAT_NO_SW_BIT flag to allow for this: this will avoid the
sw_bit codepaths, so no undefined reference is created. __map_range() will
fall back to defensive flushing every time it reads the PT, ensuring
that all data that may have just been manipulated by another thread gets
flushed and made iommu-visible immediately.

Signed-off-by: Daniel Drake <dan@xxxxxxxxxxxxxxx>
---
 drivers/iommu/generic_pt/iommu_pt.h        | 13 ++++++++-----
 drivers/iommu/generic_pt/pt_fmt_defaults.h |  7 +++++++
 include/linux/generic_pt/common.h          |  7 +++++++
 3 files changed, 22 insertions(+), 5 deletions(-)

diff --git a/drivers/iommu/generic_pt/iommu_pt.h b/drivers/iommu/generic_pt/iommu_pt.h
index c2752151c80a..54f9c311d0ac 100644
--- a/drivers/iommu/generic_pt/iommu_pt.h
+++ b/drivers/iommu/generic_pt/iommu_pt.h
@@ -484,7 +484,8 @@ static inline int pt_iommu_new_table(struct pt_state *pts,
 
 	if (pts_feature(pts, PT_FEAT_DMA_INCOHERENT)) {
 		flush_writes_item(pts);
-		pt_set_sw_bit_release(pts, SW_BIT_CACHE_FLUSH_DONE);
+		if (!pts_feature(pts, PT_FEAT_NO_SW_BIT))
+			pt_set_sw_bit_release(pts, SW_BIT_CACHE_FLUSH_DONE);
 	}
 
 	if (IS_ENABLED(CONFIG_DEBUG_GENERIC_PT)) {
@@ -702,10 +703,12 @@ static int __map_range(struct pt_range *range, void *arg, unsigned int level,
 			 * release of the cache flush so that this can acquire
 			 * visibility at the iommu.
 			 */
-			if (pts_feature(&pts, PT_FEAT_DMA_INCOHERENT) &&
-			    !pt_test_sw_bit_acquire(&pts,
-						    SW_BIT_CACHE_FLUSH_DONE))
-				flush_writes_item(&pts);
+			if (pts_feature(&pts, PT_FEAT_DMA_INCOHERENT)) {
+				if (pts_feature(&pts, PT_FEAT_NO_SW_BIT) ||
+				    !pt_test_sw_bit_acquire(&pts,
+							    SW_BIT_CACHE_FLUSH_DONE))
+					flush_writes_item(&pts);
+			}
 		}
 
 		/*
diff --git a/drivers/iommu/generic_pt/pt_fmt_defaults.h b/drivers/iommu/generic_pt/pt_fmt_defaults.h
index 69fb7c2314ca..72f341645294 100644
--- a/drivers/iommu/generic_pt/pt_fmt_defaults.h
+++ b/drivers/iommu/generic_pt/pt_fmt_defaults.h
@@ -249,6 +249,13 @@ static inline unsigned int pt_max_sw_bit(struct pt_common *common)
 	return 0;
 }
 
+/*
+ * In the DMA_INCOHERENT case, if no SW bit has been defined, produce a linker
+ * error (undefined symbol __pt_no_sw_bit) to alert the developer that this
+ * should be implemented. Alternatively, if this is not supported by the
+ * underlying hardware, PT_FEAT_NO_SW_BIT can be set, enabling sw-bit-less
+ * operation with a defensive flushing performance penalty.
+ */
 extern void __pt_no_sw_bit(void);
 static inline bool pt_test_sw_bit_acquire(struct pt_state *pts,
 					  unsigned int bitnr)
diff --git a/include/linux/generic_pt/common.h b/include/linux/generic_pt/common.h
index 07ef1c8341a4..9e8001f7b069 100644
--- a/include/linux/generic_pt/common.h
+++ b/include/linux/generic_pt/common.h
@@ -139,6 +139,13 @@ enum pt_features {
 	 * sub structure with information about which levels were changed.
 	 */
 	PT_FEAT_DETAILED_GATHER,
+	/**
+	 * @PT_FEAT_NO_SW_BIT: No software bit is available to denote when a
+	 * thread has flushed the CPU cache after modifying an entry. Only
+	 * set this if you cannot implement pt_sw_bit() due to hardware
+	 * limitations.
+	 */
+	PT_FEAT_NO_SW_BIT,
 	/* private: */
 	PT_FEAT_FMT_START,
 };

-- 
2.55.0