Re: [PATCH v8 11/14] PCI/RCEC: Add RCiEP's linked RCEC to AER/ERR
From: Sean V Kelley
Date: Tue Oct 13 2020 - 12:56:26 EST
On Mon, 2020-10-12 at 17:58 -0500, Bjorn Helgaas wrote:
> On Fri, Oct 09, 2020 at 11:51:39PM +0000, Kelley, Sean V wrote:
> > On Fri, 2020-10-09 at 15:07 -0700, Sean V Kelley wrote:
> > So I tested the following out, including your moving flr to aer.c:
> >
> > - Renamed flr_on_rciep() to flr_on_rc() for RC devices (RC_END and
> > RC_EC)
> >
> > - Moved check on dev->rcec into aer_root_reset() including the FLR.
> >
> > - Reworked pci_walk_bridge() to drop extra dev argument and check
> > locally for the bridge->rcec. Maybe should also check on type when
> > checking on bridge->rcec.
> >
> > Note I didn't use the check on aer_cap existence because I think
> > you
> > had added that for simply being able to skip over for the non-
> > native
> > case and I handle that with the single goto at the beginning which
> > takes you to the FLR.
>
> Right. Well, my thinking was that "root" would be a device with the
> AER Root Error Command and Root Error Status registers, i.e., a Root
> Port or RCEC. IIUC that basically means the APEI case where firmware
> gives us an error record.
Got it.
>
> Isn't the existing v5.9 code buggy in that it unconditionally pokes
> these registers? I think the APEI path can end up here, and firmware
> probably has not granted us control over AER.
Yes, APEI path can end up here even in the absence of AER control.
>
> Somewhat related question: I'm a little skeptical about the fact that
> aer_root_reset() currently does:
>
> - clear ROOT_PORT_INTR_ON_MESG_MASK
> - do reset
> - clear PCI_ERR_ROOT_STATUS
> - enable ROOT_PORT_INTR_ON_MESG_MASK
It's a bit of a mix and growing with RC_END handling.
>
> In the APEI path all this AER register manipulation must be done by
> firmware before passing the error record to the OS. So in the native
> case where the OS does own the AER registers, why can't the OS do
> that
> manipulation in the same order, i.e., all before doing the reset?
And you're right, the mix here imposes additional complexity for native
versus non-native. If you're not actively engaged with the code, it's
not obvious. So, yes moving it out would make more sense.
>
> > So this is rough, compiled, tested with AER injections but that's
> > it...
>
> I couldn't actually apply the patch below because it seems to be
> whitespace-damaged, but I think I like it.
Yes, it was a quick copy-paste to an existing email. Will work with
your branch.
>
> - It would be nice to be able to just call pcie_flr() and not have
> to add flr_on_rc(). I can't remember why we need the
> pcie_has_flr() / pcie_flr() dance. It seems racy and ugly, but I
> have a vague recollection that there actually is some reason for
> it.
I'll have a look.
>
> - I would *rather* consolidate the AER register updates and test
> for
> the non-native case once instead of treating it like a completely
> separate path with a "goto". But maybe not possible. Not a big
> deal either way.
Following your line of reasoning above, I think we can better
consolidate the AER register updates.
>
> - Getting rid of the extra "dev" argument to pci_walk_bridge() is a
> great side-effect. I didn't even notice that.
>
> - If we can simplify that "state == pci_channel_io_frozen" case as
> this does, that is a *big* deal because there are other patches
> just waiting to touch that reset and it will be much simpler if
> there's only one reset_subordinate_devices() call there.
Agreed.
>
> If you do work this up, I'd really appreciate it if you can start
> with
> my pci/err branch so I don't have to re-do all the tweaks I've
> already
> done:
>
>
> https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git/commit/?h=pci/err
>
Will do.
Thanks,
Sean
> Bjorn