Re: [PATCH v6 3/5] powerpc/pseries/eeh: Implement RTAS-based EEH error injection
From: Narayana Murty N
Date: Fri Nov 20 2026 - 02:33:25 EST
Hi Sourah,
On 19/11/26 12:44 AM, Sourabh Jain wrote:
Similar to 2/5, I'm finding this commit message a bit difficult to follow.Agreed. I'll merge 2/5 and 3/5 and rework the commit message.
Could you please rework it to make the changes and their purpose clearer?
On 23/10/26 13:06, Narayana Murty N wrote:I was referring to the error injection session established by
Replace the legacy MMIO stub in pseries_eeh_err_inject() with a fullI don't quite understand this point. What RTAS state are we referring to
PAPR-compliant RTAS error injection path.
Serialize each ibm,open-errinjct / ibm,errinjct / ibm,close-errinjct
session with errinjct_mutex because RTAS maintains state between these
calls.
here? Also, could you clarify how keeping all three calls under a lock helps
maintain the RTAS state mentioned above?
ibm,open-errinjct and identified by the returned session token.
The RTAS lock serializes individual RTAS calls, but not the complete
open/inject/close sequence. The intention of errinjct_mutex is to
prevent concurrent EEH injection requests from interleaving their
sessions.
I'll recheck the PAPR session ownership requirements to determine
whether this additional serialization is necessary.
Agreed. I'll keep the work-area allocation and lifetime detailsRTAS work area is mentioned here, but the next paragraph discusses
Once ibm,open-errinjct succeeds, ibm,close-errinjct is attempted
after parameter preparation or ibm,errinjct failures. The RTAS work
area is allocated after a successful open to avoid holding it while
waiting for the mutex or if open fails.
something else. The details about the RTAS work area only appear in a
later paragraph. I think the flow could be improved by keeping related
information together.
together in the revised commit message.
The intention was to separate the helper infrastructure from the
VFIO EEH error injection exposes a generic userspace ABI. pSeries maps
the generic EEH error types to RTAS ibm,errinjct encodings via
pseries_eeh_type_to_rtas(). EEH_ERR_TYPE_32 and EEH_ERR_TYPE_64 are
unchanged; their values are defined in arch/powerpc/include/uapi/asm/ eeh.h
and are not renumbered.
The RTAS work-area allocator guarantees that returned memory is within
the range accessible by RTAS (below min(ppc64_rma_size,
RTAS_INSTANTIATE_MAX), which is at most RTAS_INSTANTIATE_MAX = 1 GiB),
so the physical address fits in the 32-bit RTAS argument without
truncation.
Remove eeh_pe_inject_mmio_error() and its declaration; this patch
removes its only caller. The only remaining caller of
eeh_break_device() is eeh_dev_break_write() inside CONFIG_DEBUG_FS,
so move eeh_break_device() back inside the CONFIG_DEBUG_FS block to
avoid an unused-function warning when CONFIG_DEBUG_FS=n.
Also introduce validate_errinjct_args() and prepare_errinjct_buffer()
here alongside their callers, so each intermediate commit builds
cleanly with CONFIG_WERROR.
Could you clarify what this intermediate commit is for?
RTAS implementation. However, given the unused static helpers
and the additional complexity introduced by the split, I'll merge
the two patches.
Agreed. I'll move the reference to the corresponding QEMU series
The addr argument to ibm,errinjct is function-dependent. Memory and IO
load/store functions (EEH_ERR_FUNC_LD/ST_MEM_* and EEH_ERR_FUNC_LD/ ST_IO_*)
target BAR-backed PCI bus addresses; these are validated against the PE
via validate_addr_mask_in_pe(). Config-space functions carry PCI config
register offsets and DMA functions carry IOVA/TCE addresses; addr for
these functions is passed directly to RTAS without BAR membership
validation.
For the 32-bit RTAS encoding (RTAS_ERR_TYPE_IOA_BUS_ERROR), addr and
mask are checked to fit in 32 bits before buffer encoding. This is an
RTAS format constraint independent of function type.
Tested with corresponding QEMU patches:
https://lore.kernel.org/all/20251029150618.186803-1- nnmlinux@xxxxxxxxxxxxx/
Wouldn't this be better suited for the cover letter?
to the cover letter.
Yes. I'll introduce pr_debug() directly in the merged patch.
[...]
Signed-off-by: Narayana Murty N <nnmlinux@xxxxxxxxxxxxx>
---
arch/powerpc/include/asm/eeh.h | 2 -
arch/powerpc/kernel/eeh.c | 10 +-
arch/powerpc/platforms/pseries/eeh_pseries.c | 246 +++++++++++++++++--
3 files changed, 228 insertions(+), 30 deletions(-)
diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/ asm/eeh.h
index b7ebb4ac2c71..083ac58e7eb7 100644
--- a/arch/powerpc/include/asm/eeh.h
+++ b/arch/powerpc/include/asm/eeh.h
@@ -310,8 +310,6 @@ int eeh_pe_reset(struct eeh_pe *pe, int option, bool include_passed);
int eeh_pe_configure(struct eeh_pe *pe);
int eeh_pe_inject_err(struct eeh_pe *pe, int type, int func,
unsigned long addr, unsigned long mask);
-int eeh_pe_inject_mmio_error(struct pci_dev *pdev);
-
/**
* EEH_POSSIBLE_ERROR() -- test for possible MMIO failure.
*
diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c
index bb836f02101c..bae7ca2bd399 100644
--- a/arch/powerpc/kernel/eeh.c
+++ b/arch/powerpc/kernel/eeh.c
@@ -1571,6 +1571,8 @@ static int proc_eeh_show(struct seq_file *m, void *v)[...]
}
#endif /* CONFIG_PROC_FS */
+#ifdef CONFIG_DEBUG_FS
+
static int eeh_break_device(struct pci_dev *pdev)
{
struct resource *bar = NULL;
@@ -1664,14 +1666,6 @@ static int eeh_break_device(struct pci_dev *pdev)
return 0;
}
-int eeh_pe_inject_mmio_error(struct pci_dev *pdev)
-{
- return eeh_break_device(pdev);
-}
-
-#ifdef CONFIG_DEBUG_FS
-
-
static struct pci_dev *eeh_debug_lookup_pdev(struct file *filp,
const char __user *user_buf,
size_t count, loff_t *ppos)
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c b/arch/ powerpc/platforms/pseries/eeh_pseries.c
index 28b1fd19e8bc..2a2ec1a237e6 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -35,6 +35,7 @@
#include <asm/machdep.h>
#include <asm/ppc-pci.h>
#include <asm/rtas.h>
+#include <asm/rtas-work-area.h>
/* RTAS tokens */
static int ibm_set_eeh_option;
@@ -915,48 +916,253 @@ static int validate_addr_mask_in_pe(struct eeh_pe *pe, unsigned long addr,
pci_unlock_rescan_remove();
if (!found)
- pr_err("addr=0x%lx mask=0x%lx not within any PCI bus BAR of any device in PE\n",
- addr, mask);
+ pr_debug("addr=0x%lx mask=0x%lx not within any PCI bus BAR of any device in PE\n",
+ addr, mask);
IIUC, the previous patch adds pr_err(), and this patch changes it to pr_debug().
Could we avoid this intermediate change?
No, they are not used by validate_errinjct_args().return found ? 0 : -EINVAL;Are addr and mask really needed as function arguments?
}
/**
- * pseries_eeh_err_inject - Inject specified error to the indicated PE
- * @pe: the indicated PE
- * @type: error type
- * @func: specific error type
- * @addr: address
- * @mask: address mask
- * The routine is called to inject specified error, which is
- * determined by @type and @func, to the indicated PE
+ * validate_errinjct_args - Top-level validation for RTAS error injection arguments
+ * @pe: EEH PE for the target device
+ * @type: generic EEH error type
+ * @func: error function selector
+ * @addr: address argument (type-dependent, may be zero)
+ * @mask: mask argument (type-dependent, may be zero)
I'll simplify the validation and avoid the duplicate error-type
mapping.
Agreed. I will shorten the comment.+ *
+ * Validates all parameters before opening an RTAS injection session.
+ * Returns Linux errno values; does not return raw RTAS status codes.
Wouldn't it be better for the function comment to describe what the
function does, rather than how an RTAS injection session should be opened?
Agreed. I'll keep the helper comment focused.+ *
+ * Return: 0 if all parameters are valid, negative errno otherwise.
*/
-static int pseries_eeh_err_inject(struct eeh_pe *pe, int type, int func,
+static int validate_errinjct_args(struct eeh_pe *pe, int type, int func,
unsigned long addr, unsigned long mask)
{
- struct eeh_dev *pdev;
+ if (!pe || !pe->phb)
+ return -EINVAL;
+
+ if (pseries_eeh_type_to_rtas(type) < 0)
+ return -EINVAL;
- /* Check on PCI error type */
- if (type != EEH_ERR_TYPE_32 && type != EEH_ERR_TYPE_64)
+ if (func < EEH_ERR_FUNC_MIN || func > EEH_ERR_FUNC_MAX)
return -EINVAL;
+ return 0;
+}
+
+/*
+ * pseries_eeh_func_needs_bar_addr - return true for error functions whose
+ * addr argument is a BAR-backed PCI MMIO or IO address.
+ *
+ * Memory and IO load/store functions target specific BAR addresses and
+ * require PCI bus address validation against the PE. Config-space
+ * functions carry PCI config register offsets and DMA functions carry
+ * IOVA/TCE addresses; neither is a BAR address.
This seems like useful information, but perhaps it belongs elsewhere.
The function itself looks fairly simple, and the comment conveys quite a
bit of information that I’m not sure how to relate to what the function actually does.
Agreed. I'll keep the comment focused on encoding the RTAS
+ */Wouldn't this be better documented elsewhere? This function only populates
+static bool pseries_eeh_func_needs_bar_addr(int func)
+{
switch (func) {
case EEH_ERR_FUNC_LD_MEM_ADDR:
case EEH_ERR_FUNC_LD_MEM_DATA:
+ case EEH_ERR_FUNC_LD_IO_ADDR:
+ case EEH_ERR_FUNC_LD_IO_DATA:
case EEH_ERR_FUNC_ST_MEM_ADDR:
case EEH_ERR_FUNC_ST_MEM_DATA:
- /* injects a MMIO error for all pdev's belonging to PE */
- pci_lock_rescan_remove();
- list_for_each_entry(pdev, &pe->edevs, entry)
- eeh_pe_inject_mmio_error(pdev->pdev);
- pci_unlock_rescan_remove();
+ case EEH_ERR_FUNC_ST_IO_ADDR:
+ case EEH_ERR_FUNC_ST_IO_DATA:
+ return true;
+ default:
+ return false;
+ }
+}
+
+/**
+ * prepare_errinjct_buffer() - Build ibm,errinjct work buffer
+ * @buf: RTAS error-injection work buffer
+ * @pe: EEH PE associated with the injection target
+ * @rtas_err_type: RTAS firmware error-injection type
+ * @func: Error function selector
+ * @addr: Address argument interpreted according to @func:
+ * MEM/IO functions: BAR-backed PCI bus address;
+ * CFG functions: PCI configuration-space offset;
+ * DMA functions: DMA/IOVA/TCE address.
+ * @mask: Address mask passed to firmware
+ *
+ * Zeroes @buf and populates it according to the PAPR layout for the
+ * selected IOA bus-error injection type.
+ *
+ * BAR membership validation is performed only for MEM/IO functions
+ * whose @addr is a BAR-backed PCI bus address. CFG and DMA function
+ * addresses are encoded directly without BAR lookup.
+ *
+ * The caller provides an exclusive per-call RTAS work-area buffer.
+ * Firmware session serialization is handled by pseries_eeh_err_inject().
the given work area; it doesn't make the RTAS call.
parameter buffer and move the session-related details to the caller.
Agreed. I'll use errinjct_class.+ *[...]
+ * Return: 0 on success or a negative errno on invalid input.
+ */
+static int prepare_errinjct_buffer(void *buf, struct eeh_pe *pe,
+ int rtas_err_type, int func,
+ unsigned long addr, unsigned long mask)
+{
+ __be64 *buf64 = (__be64 *)buf;
+ __be32 *buf32 = (__be32 *)buf;
+ int rc;
+
+ if (!buf || !pe || !pe->phb)
+ return -EINVAL;
+
+ memset(buf, 0, RTAS_ERRINJCT_BUF_SIZE);
+
+ switch (rtas_err_type) {
+ case RTAS_ERR_TYPE_IOA_BUS_ERROR:
+ /*
+ * 32-bit RTAS format: addr and mask must fit in 32 bits.
+ * This is an RTAS encoding constraint, not BAR validation.
+ */
+ if (upper_32_bits(addr) || upper_32_bits(mask))
+ return -EINVAL;
+
+ if (pseries_eeh_func_needs_bar_addr(func)) {[...]
+ rc = validate_addr_mask_in_pe(pe, addr, mask);
+ if (rc)
+ return rc;
+ }
+
+ buf32[0] = cpu_to_be32((u32)addr);
+ buf32[1] = cpu_to_be32((u32)mask);
+ buf32[2] = cpu_to_be32(pe->addr);
+ buf32[3] = cpu_to_be32(BUID_HI(pe->phb->buid));
+ buf32[4] = cpu_to_be32(BUID_LO(pe->phb->buid));
+ buf32[5] = cpu_to_be32(func);
+ break;
+
+ case RTAS_ERR_TYPE_IOA_BUS_ERROR_64:
+ if (pseries_eeh_func_needs_bar_addr(func)) {
+ rc = validate_addr_mask_in_pe(pe, addr, mask);
+ if (rc)
+ return rc;
+ }
+
+ buf64[0] = cpu_to_be64(addr);
+ buf64[1] = cpu_to_be64(mask);
+ buf32[4] = cpu_to_be32(pe->addr);
+ buf32[5] = cpu_to_be32(BUID_HI(pe->phb->buid));
+ buf32[6] = cpu_to_be32(BUID_LO(pe->phb->buid));
+ buf32[7] = cpu_to_be32(func);
break;
+
default:
- return -ERANGE;
+ pr_err("unsupported RTAS error injection type 0x%x\n", rtas_err_type);
+ return -EINVAL;
}
+ pr_debug("errinjct buffer ready: rtas_err_type=0x%x func=%d addr=0x%lx mask=0x%lx\n",
+ rtas_err_type, func, addr, mask);
return 0;
}
+/* Serializes the open/inject/close RTAS error injection session */
+static DEFINE_MUTEX(errinjct_mutex);
+
+/**
+ * pseries_eeh_err_inject - Inject specified error to the indicated PE
+ * @pe: the indicated PE
+ * @type: generic EEH error type (EEH_ERR_TYPE_32 or EEH_ERR_TYPE_64)
+ * @func: specific error function
+ * @addr: address argument (type-dependent, may be zero)
+ * @mask: address mask (type-dependent, may be zero)
+ *
+ * Implements PAPR-compliant error injection using:
+ * ibm,open-errinjct -> ibm,errinjct -> ibm,close-errinjct
+ *
+ * errinjct_mutex serializes the open/inject/close session sequence
+ * because RTAS maintains state between these calls. A short-lived
+ * RTAS work area is allocated after a successful open and freed before
+ * close.
+ *
+ * Return: 0 on success, negative errno on failure.
+ */
+static int pseries_eeh_err_inject(struct eeh_pe *pe, int type, int func,
+ unsigned long addr, unsigned long mask)
+{
+ struct rtas_work_area *area;
+ phys_addr_t area_phys;
+ void *buf;
+ int open_token, errinjct_token, close_token;
+ int session_token;
+ int rtas_err_type;
The name rtas_err_type seems a bit confusing for the error injection class.
Would err_injct_class or errinjct_class be clearer?
As mentioned above, the intention is to serialize complete error+ int close_rc;Is this lock really necessary?
+ int rc;
+
+ rc = validate_errinjct_args(pe, type, func, addr, mask);
+ if (rc)
+ return rc;
+
+ rtas_err_type = pseries_eeh_type_to_rtas(type);
+ if (rtas_err_type < 0)
+ return -EINVAL;
+
+ open_token = rtas_function_token(RTAS_FN_IBM_OPEN_ERRINJCT);
+ errinjct_token = rtas_function_token(RTAS_FN_IBM_ERRINJCT);
+ close_token = rtas_function_token(RTAS_FN_IBM_CLOSE_ERRINJCT);
+
+ if (open_token == RTAS_UNKNOWN_SERVICE ||
+ errinjct_token == RTAS_UNKNOWN_SERVICE ||
+ close_token == RTAS_UNKNOWN_SERVICE)
+ return -ENODEV;
+
+ mutex_lock(&errinjct_mutex);
injection sessions initiated through the EEH backend rather than
individual RTAS calls.
However, this mutex does not serialize direct userspace RTAS
callers. I'll verify the firmware session ownership semantics
before deciding whether to retain it.
- Sourabh Jain
+
+ do {
+ rc = rtas_call(open_token, 0, 2, &session_token);
+ } while (rtas_busy_delay(rc));
+
+ if (rc) {
+ pr_err("ibm,open-errinjct failed: status=%d\n", rc);
+ rc = rtas_error_rc(rc);
+ goto out_unlock;
+ }
+
+ area = rtas_work_area_alloc(RTAS_ERRINJCT_BUF_SIZE);
+ buf = rtas_work_area_raw_buf(area);
+ area_phys = rtas_work_area_phys(area);
+
+ rc = prepare_errinjct_buffer(buf, pe, rtas_err_type, func, addr, mask);
+ if (rc)
+ goto out_free_area;
+
+ /*
+ * RTAS work areas are allocated below RTAS_INSTANTIATE_MAX, so
+ * area_phys fits in the 32-bit RTAS argument.
+ */
+ do {
+ rc = rtas_call(errinjct_token, 3, 1, NULL,
+ rtas_err_type, session_token,
+ (u32)area_phys);
+ } while (rtas_busy_delay(rc));
+
+ if (rc) {
+ pr_err("ibm,errinjct failed: status=%d\n", rc);
+ rc = rtas_error_rc(rc);
+ }
+
+out_free_area:
+ rtas_work_area_free(area);
+
+ do {
+ close_rc = rtas_call(close_token, 1, 1, NULL, session_token);
+ } while (rtas_busy_delay(close_rc));
+
+ if (close_rc) {
+ pr_warn("ibm,close-errinjct failed: status=%d\n", close_rc);
+ if (!rc)
+ rc = rtas_error_rc(close_rc);
+ }
+
+out_unlock:
+ mutex_unlock(&errinjct_mutex);
+ return rc;
+}
+
static struct eeh_ops pseries_eeh_ops = {
.name = "pseries",
.probe = pseries_eeh_probe,
Thanks,
Narayana Murty.