Re: [PATCH v4 2/3] lib: logic_pio: Reject accesses to unregistered CPU MMIO regions

From: Bjorn Helgaas
Date: Thu Jun 13 2019 - 12:57:03 EST

On Tue, Jun 11, 2019 at 10:12:53PM +0800, John Garry wrote:
> Currently when accessing logical indirect PIO addresses in
> logic_{in, out}{,s}, we first ensure that the region is registered.

I think logic_pio is specifically concerned with I/O port space, so
it's a little bit unfortunate that we named this "PIO".

PIO is a general term for "Programmed I/O", which just means the CPU
is involved in each transfer, as opposed to DMA. The transfers can be
to either MMIO or I/O port space.

So this ends up being a little confusing because I think you mean
"Port I/O", not "Programmed I/O".

> However, no such check exists for CPU MMIO regions. The CPU MMIO regions
> would be registered by the PCI host (when PCI_IOBASE is defined) in
> pci_register_io_range().

IIUC this "CPU MMIO region" is an MMIO region where a memory load or
store from the CPU is converted by a PCI host bridge into an I/O port
transaction on PCI.

Again IIUC, logic_pio supports two kinds of I/O port space accesses:

1) The simple "bridge converts loads/stores to an MMIO region to PCI
I/O port transactions" kind, and

2) The more complicated "somebody supplies logic_pio_host_ops
functions that do arbitrary magic to generate I/O port
transactions on some bus.

And this patch is making the first kind smarter. Previously it would
perform the memory access whenever "addr < MMIO_UPPER_LIMIT", but
after this patch it will only do it if find_io_range() succeeds.

Right? Sorry for restating what probably should have been obvious to
me.

I have two observations here:

1) The simple "bridge converts CPU MMIO space to PCI I/O port space"
flavor is essentially identical to what ia64 (and probably other
architectures) does. This should really be combined somehow.

2) If you made a default set of logic_pio_host_ops that merely did
loads/stores and maybe added a couple fields in the struct
logic_pio_hwaddr, I bet you could unify the two kinds so
logic_inb() would look something like this:

u8 logic_inb(unsigned long addr)
{
struct logic_pio_hwaddr *range = find_io_range(addr);

if (!range)
return (u8) ~0;

return (u8) range->ops->in(range->hostdata, addr, sz);
}

> We have seen scenarios when systems which don't have a PCI host, or they
> do but the PCI host probe fails, certain drivers attempts to still attempt
> to access PCI IO ports; examples are in [1] and [2].
>
> Such is a case on an ARM64 system without a PCI host:
>
> root@(none)$ insmod hwmon/f71805f.ko
> Unable to handle kernel paging request at virtual address ffff7dfffee0002e
> Mem abort info:
> ESR = 0x96000046
> Exception class = DABT (current EL), IL = 32 bits
> SET = 0, FnV = 0
> EA = 0, S1PTW = 0
> Data abort info:
> ISV = 0, ISS = 0x00000046
> CM = 0, WnR = 1
> swapper pgtable: 4k pages, 48-bit VAs, pgdp = (____ptrval____)
> [ffff7dfffee0002e] pgd=000000000141c003, pud=000000000141d003, pmd=0000000000000000
> Internal error: Oops: 96000046 [#1] PREEMPT SMP
> Modules linked in: f71805f(+)
> CPU: 20 PID: 2736 Comm: insmod Not tainted 5.1.0-rc1-00003-g6f1bfec2a620-dirty #99
> Hardware name: Huawei Taishan 2280 /D05, BIOS Hisilicon D05 IT21 Nemo 2.0 RC0 04/18/2018
> pstate: 80000005 (Nzcv daif -PAN -UAO)
> pc : logic_outb+0x54/0xb8
> lr : f71805f_find+0x2c/0x1b8 [f71805f]
> sp : ffff000025fbba90
> x29: ffff000025fbba90 x28: ffff000008b944d0
> x27: ffff000025fbbdf0 x26: 0000000000000100
> x25: ffff801f8c270580 x24: ffff000011420000
> x23: ffff000025fbbb3e x22: ffff000025fbbb40
> x21: ffff000008b991b8 x20: 0000000000000087
> x19: 000000000000002e x18: ffffffffffffffff
> x17: 0000000000000000 x16: 0000000000000000
> x15: ffff00001127d6c8 x14: 0000000000000000
> x13: 0000000000000000 x12: 0000000000000000
> x11: 0000000000010820 x10: 0000841fdac40000
> x9 : 0000000000000001 x8 : 0000000040000000
> x7 : 0000000000210d00 x6 : 0000000000000000
> x5 : ffff801fb6a46040 x4 : ffff841febeaeda0
> x3 : 0000000000ffbffe x2 : ffff000025fbbb40
> x1 : ffff7dfffee0002e x0 : ffff7dfffee00000
> Process insmod (pid: 2736, stack limit = 0x(____ptrval____))
> Call trace:
> logic_outb+0x54/0xb8
> f71805f_find+0x2c/0x1b8 [f71805f]
> f71805f_init+0x38/0xe48 [f71805f]
> do_one_initcall+0x5c/0x198
> do_init_module+0x54/0x1b0
> load_module+0x1dc4/0x2158
> __se_sys_init_module+0x14c/0x1e8
> __arm64_sys_init_module+0x18/0x20
> el0_svc_common+0x5c/0x100
> el0_svc_handler+0x2c/0x80
> el0_svc+0x8/0xc
> Code: d2bfdc00 f2cfbfe0 f2ffffe0 8b000021 (39000034)
> ---[ end trace 10ea80bde051bbfc ]---
> root@(none)$
>
> Well-behaved drivers call request_{muxed_}region() to grab the IO port
> region, but success here still doesn't actually mean that there is some IO
> port mapped in this region.
>
> This patch adds a check to ensure that the CPU MMIO region is registered
> prior to accessing the PCI IO ports.
>
> Any failed checks silently return.

I *think* what you're doing here is making inb/outb/etc work the same
as on x86, i.e., if no device responds to an inb(), the caller gets
~0, and if no device claims an outb() the data gets dropped.

That should be explicit in the commit log.

> [1] https://lore.kernel.org/linux-pci/56F209A9.4040304@xxxxxxxxxx
> [2] https://lore.kernel.org/linux-arm-kernel/e6995b4a-184a-d8d4-f4d4-9ce75d8f47c0@xxxxxxxxxx/
>
> Signed-off-by: John Garry <john.garry@xxxxxxxxxx>
> ---
> lib/logic_pio.c | 60 +++++++++++++++++++++++++++++++++----------------
> 1 file changed, 41 insertions(+), 19 deletions(-)
>
> diff --git a/lib/logic_pio.c b/lib/logic_pio.c
> index 40d9428010e1..47d24f428908 100644
> --- a/lib/logic_pio.c
> +++ b/lib/logic_pio.c
> @@ -126,7 +126,7 @@ static struct logic_pio_hwaddr *find_io_range(unsigned long pio)
> if (in_range(pio, range->io_start, range->size))
> return range;
> }
> - pr_err("PIO entry token %lx invalid\n", pio);
> +
> return NULL;
> }
>
> @@ -197,11 +197,12 @@ unsigned long logic_pio_trans_cpuaddr(resource_size_t addr)
> type logic_in##bw(unsigned long addr) \
> { \
> type ret = (type)~0; \
> + struct logic_pio_hwaddr *range = find_io_range(addr); \
> \
> if (addr < MMIO_UPPER_LIMIT) { \
> - ret = read##bw(PCI_IOBASE + addr); \
> + if (range) \
> + ret = read##bw(PCI_IOBASE + addr); \
> } else if (addr >= MMIO_UPPER_LIMIT && addr < IO_SPACE_LIMIT) { \
> - struct logic_pio_hwaddr *range = find_io_range(addr); \
> size_t sz = sizeof(type); \
> \
> if (range && range->ops) \
> @@ -214,10 +215,12 @@ type logic_in##bw(unsigned long addr) \
> \
> void logic_out##bw(type value, unsigned long addr) \
> { \
> + struct logic_pio_hwaddr *range = find_io_range(addr); \
> + \
> if (addr < MMIO_UPPER_LIMIT) { \
> - write##bw(value, PCI_IOBASE + addr); \
> + if (range) \
> + write##bw(value, PCI_IOBASE + addr); \
> } else if (addr >= MMIO_UPPER_LIMIT && addr < IO_SPACE_LIMIT) { \
> - struct logic_pio_hwaddr *range = find_io_range(addr); \
> size_t sz = sizeof(type); \
> \
> if (range && range->ops) \
> @@ -230,10 +233,12 @@ void logic_out##bw(type value, unsigned long addr) \
> \
> void logic_ins##bw(unsigned long addr, void *buf, unsigned int cnt) \
> { \
> + struct logic_pio_hwaddr *range = find_io_range(addr); \
> + \
> if (addr < MMIO_UPPER_LIMIT) { \
> - reads##bw(PCI_IOBASE + addr, buf, cnt); \
> + if (range) \
> + reads##bw(PCI_IOBASE + addr, buf, cnt); \
> } else if (addr >= MMIO_UPPER_LIMIT && addr < IO_SPACE_LIMIT) { \
> - struct logic_pio_hwaddr *range = find_io_range(addr); \
> size_t sz = sizeof(type); \
> \
> if (range && range->ops) \
> @@ -242,16 +247,17 @@ void logic_ins##bw(unsigned long addr, void *buf, unsigned int cnt) \
> else \
> WARN_ON_ONCE(1); \
> } \
> - \
> } \
> \
> void logic_outs##bw(unsigned long addr, const void *buf, \
> unsigned int cnt) \
> { \
> + struct logic_pio_hwaddr *range = find_io_range(addr); \
> + \
> if (addr < MMIO_UPPER_LIMIT) { \
> - writes##bw(PCI_IOBASE + addr, buf, cnt); \
> + if (range) \
> + writes##bw(PCI_IOBASE + addr, buf, cnt); \
> } else if (addr >= MMIO_UPPER_LIMIT && addr < IO_SPACE_LIMIT) { \
> - struct logic_pio_hwaddr *range = find_io_range(addr); \
> size_t sz = sizeof(type); \
> \
> if (range && range->ops) \
> @@ -269,28 +275,44 @@ type logic_in##bw(unsigned long addr) \
> { \
> type ret = (type)~0; \
> \
> - if (addr < MMIO_UPPER_LIMIT) \
> - ret = read##bw(PCI_IOBASE + addr); \
> + if (addr < MMIO_UPPER_LIMIT) { \
> + struct logic_pio_hwaddr *range = find_io_range(addr); \
> + \
> + if (range) \
> + ret = read##bw(PCI_IOBASE + addr); \
> + } \
> return ret; \
> } \
> \
> -void logic_out##bw(type value, unsigned long addr) \
> +void logic_out##bw(type val, unsigned long addr) \
> { \
> - if (addr < MMIO_UPPER_LIMIT) \
> - write##bw(value, PCI_IOBASE + addr); \
> + if (addr < MMIO_UPPER_LIMIT) { \
> + struct logic_pio_hwaddr *range = find_io_range(addr); \
> + \
> + if (range) \
> + write##bw(val, PCI_IOBASE + addr); \
> + } \
> } \
> \
> void logic_ins##bw(unsigned long addr, void *buf, unsigned int cnt) \
> { \
> - if (addr < MMIO_UPPER_LIMIT) \
> - reads##bw(PCI_IOBASE + addr, buf, cnt); \
> + if (addr < MMIO_UPPER_LIMIT) { \
> + struct logic_pio_hwaddr *range = find_io_range(addr); \
> + \
> + if (range) \
> + reads##bw(PCI_IOBASE + addr, buf, cnt); \
> + } \
> } \
> \
> void logic_outs##bw(unsigned long addr, const void *buf, \
> unsigned int cnt) \
> { \
> - if (addr < MMIO_UPPER_LIMIT) \
> - writes##bw(PCI_IOBASE + addr, buf, cnt); \
> + if (addr < MMIO_UPPER_LIMIT) { \
> + struct logic_pio_hwaddr *range = find_io_range(addr); \
> + \
> + if (range) \
> + writes##bw(PCI_IOBASE + addr, buf, cnt); \
> + } \
> }
> #endif /* CONFIG_INDIRECT_PIO */
>
> --
> 2.17.1
>