RE: [PATCH 0/5] cxl: Log downport PCIe AER and CXL RAS error information

[Ariel.Sibley]

> -----Original Message-----
> From: Terry Bowman <Terry.Bowman@xxxxxxx>
> Sent: Friday, October 28, 2022 11:30 AM
> To: Ariel Sibley - C33371 <Ariel.Sibley@xxxxxxxxxxxxx>; alison.schofield@xxxxxxxxx;
> vishal.l.verma@xxxxxxxxx; dave.jiang@xxxxxxxxx; ira.weiny@xxxxxxxxx; bwidawsk@xxxxxxxxxx;
> dan.j.williams@xxxxxxxxx
> Cc: linux-cxl@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; bhelgaas@xxxxxxxxxx; rafael@xxxxxxxxxx;
> lenb@xxxxxxxxxx; Jonathan.Cameron@xxxxxxxxxx; dave@xxxxxxxxxxxx; rrichter@xxxxxxx
> Subject: Re: [PATCH 0/5] cxl: Log downport PCIe AER and CXL RAS error information
> 
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> 
> Hi Ariel,
> 
> On 10/28/22 07:30, Ariel.Sibley@xxxxxxxxxxxxx wrote:
> >> -----Original Message-----
> >> From: Terry Bowman <terry.bowman@xxxxxxx>
> >> Sent: Friday, October 21, 2022 3:56 PM
> >> To: alison.schofield@xxxxxxxxx; vishal.l.verma@xxxxxxxxx; dave.jiang@xxxxxxxxx; ira.weiny@xxxxxxxxx;
> >> bwidawsk@xxxxxxxxxx; dan.j.williams@xxxxxxxxx
> >> Cc: terry.bowman@xxxxxxx; linux-cxl@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; bhelgaas@xxxxxxxxxx;
> >> rafael@xxxxxxxxxx; lenb@xxxxxxxxxx; Jonathan.Cameron@xxxxxxxxxx; dave@xxxxxxxxxxxx; rrichter@xxxxxxx
> >> Subject: [PATCH 0/5] cxl: Log downport PCIe AER and CXL RAS error information
> >>
> >> This patchset adds CXL downport PCI AER and CXL RAS logging to the CXL
> >> error handling. This is necessary for communicating CXL HW issues to users.
> >> The included patches find and cache pointers to the AER and CXL RAS PCIe
> >> capability structures. The cached pointers are then used to display the
> >> error information in a later patch. These changes follow the CXL
> >> specification, Chapter 8 'Control and Status Registers'.[1]
> >>
> >> The first patch enables CXL1.1 RCD support through the ACPI _OSC support
> >> method.
> >>
> >> The 2nd and 3rd patches find and map PCIe AER and CXL RAS capabilities.
> >>
> >> The 4th patch enables AER error reporting.
> >>
> >> The 5th patch adds functionality to log the PCIe AER and RAS capabilities.
> >>
> >> TODO work remains to consolidate the HDM and CXL RAS register mapping
> >> (patch#3). The current CXL RAS register mapping will be replaced to reuse
> >> cxl_probe_component_regs() function as David Jiang and Alison Schofield
> >> upstreamed. Should the same be done for the AER registers (patch#2)? The
> >> AER registers are not in the component register block but are instead in
> >> the downport and upport (RCRB).
> >
> > The RCD's AER registers are not in either the component register block or
> > RCRB. They are in the RCiEP config space.
> >
> > Per CXL 3.0 Section 12.2.1.2 RCD Upstream Port-detected Errors:
> > "2. Upstream Port RCRB shall not implement the AER Extended Capability."
> > ...
> > "4. CXL.io Functions log the received message in their respective AER Extended
> > Capability."
> >
> 
> I based this comment on CXL3.0 8.2.1.1 "RCH Downstream Port RCRB":
> 
> "The RCH Downstream Port RCRB is a 4-KB memory region that contains
> registers based upon the PCIe-defined registers for a root port... The
> RCH Downstream Port supported PCIe capabilities and extended
> capabilities are listed in Table 8-18"
> 
> And Table 8-18 includes 'Advanced Error Reporting
> Extended Capability' with no exceptions.
> 
> The RCD upstream port needs to be removed from my comment. Thank you for
> pointing that out. My understanding is the RCH downstream port does
> include the AER registers.
> 
> Regards,
> Terry
> 

Thanks Terry. Yes, the RCH downstream port RCRB must contain AER registers
due to this text in CXL 3.0 Section 12.2.1.1 RCH Downstream Port-detected Errors:
"1. DPA CXL.io-detected errors are logged in the local AER Extended Capability
in DPA RCRB. Software must ensure that the Root Port Control register in the
DPA AER Extended Capability are not configured to generate interrupts."

However, the text in the Exceptions column in Table 8-18 for AER is "N/A. Software
should ignore.", which I find somewhat confusing given the text in Section
12.2.1.1. However, since the text in 12.2.1.1 is explicit, the AER registers
must be there.