Re: PCI IO ECS access is no longer possible for AMD family 17h

[Ingo Molnar]

* Rostyslav Khudolii <ros@xxxxxxxx> wrote:

> Hi,
> 
> > Rostyslav, I would like to ask you, do you have patches / updates for
> > enabling the EnableCf8ExtCfg bit for AMD 17h+ family? I could try to
> > adjust my lspci changes for new machines.
> 
> Pali, sorry for the late reply. Do I understand correctly, that even
> though you have access to the ECS via
> the MMCFG you still want the legacy (direct IO) to work for the
> debugging purposes? I can prepare a
> simple patch that will allow you to do so if that's the case.
> 
> >
> > So what is the practical impact here? Do things start breaking
> > unexpectedly if CONFIG_ACPI_MCFG and CONFIG_PCI_MMCONFIG are disabled?
> > Then I'd suggest fixing that in the Kconfig space, either by adding a
> > dependency on ACPI_MCFG && PCI_MMCONFIG, or by selecting those
> > must-have pieces of infrastructure.
> >
> 
> Ingo, thank you for the reply.
> 
> The way I understand the access to the PCI ECS (via raw_pci_ext_ops)
> works, is the following:
> 1. If CONFIG_ACPI_MCFG or CONFIG_PCI_MMCONFIG are enabled - set the
> raw_pci_ext_ops to use
>     MMCFG to access ECS. See pci_mmcfg_early_init() / pci_mmcfg_late_init();
> 2. If CONFIG_ACPI_MCFG and CONFIG_PCI_MMCONFIG are disabled - set the
> raw_pci_ext_ops to use
>     the 'direct' access to ECS. See pci_direct_init(). The direct
> access is conditional on the PCI_HAS_IO_ECS
>     flag being set.
> 
> On AMD, the kernel enables the ECS IO access via the
> amd_bus_cpu_online() and pci_enable_pci_io_ecs().
> Except those functions have no desired effect on the AMD 17h+ family
> because the register (EnableCf8ExtCfg),
> they access, has been moved. What is important though, is that the
> PCI_HAS_IO_ECS flag is set unconditionally.
> See pci_io_ecs_init() in amd_bus.c
> 
> Therefore I was wondering whether we should add support for the 17h+
> family in those functions to have
> the direct access work for those families as well.

Yeah, I think so, but I'm really just guessing:

> Regarding your suggestion to address this in the Kconfig space - I'm 
> not quite sure I follow, since right now the kernel will use 
> raw_pci_ext_ops whenever access beyond the first 256 bytes is 
> requested. Say we want to make that conditional on CONFIG_ACPI_MCFG 
> and CONFIG_PCI_MMCONFIG, does it also mean then we want to drop 
> support for the 'direct' PCI IO ECS access altogether?

I thought that enabling CONFIG_ACPI_MCFG would solve the problem, and 
other architectures are selecting it pretty broadly:

 arch/arm64/Kconfig:     select ACPI_MCFG if (ACPI && PCI)
 arch/loongarch/Kconfig: select ACPI_MCFG if ACPI
 arch/riscv/Kconfig:     select ACPI_MCFG if (ACPI && PCI)

While x86 allows it to be user-configured, which may result in 
misconfiguration, given that PCI_HAS_IO_ECS is being followed 
unconditionally if a platform provides it?

Thanks,

	Ingo