Re: [PATCH v5 09/16] kexec: enable KHO support for memory preservation

[Jason Gunthorpe]

On Thu, Apr 03, 2025 at 05:37:06PM +0000, Pratyush Yadav wrote:

> And I think this will help make the 2 seconds much smaller as well later
> down the line since we can now find out if a given page is reserved in a
> few operations, and do it in parallel.

Yes, most certainly
 
> > This should be more like:
> >
> > union {
> >       void *table;
> >       phys_addr_t table_phys;
> > };
> >
> > Since we are not using the low bits right now and it is alot cheaper
> > to convert from va to phys only once during the final step. __va is
> > not exactly fast.
> 
> The descriptor is used on _every_ level of the table, not just the
> top.

Yes

> So if we use virtual addresses, at serialize time we would have to walk
> the whole table and covert all addresses to physical.

Yes

> And __va() does
> not seem to be doing too much. On x86, it expands to:
> 
>     #define __va(x)			((void *)((unsigned long)(x)+PAGE_OFFSET))
> 
> and on ARM64 to:
> 
>     #define __va(x)			((void *)__phys_to_virt((phys_addr_t)(x)))
>     #define __phys_to_virt(x)	((unsigned long)((x) - PHYS_OFFSET) | PAGE_OFFSET)

Hmm, I was sure sparsemem added a bunch of stuff to this path, maybe
I'm thinking of page_to_phys

> >> +struct kho_mem_track {
> >> +	/* Points to L4 KHOMEM descriptor, each order gets its own table. */
> >> +	struct xarray orders;
> >> +};
> >
> > I think it would be easy to add a 5th level and just use bits 63:57 as
> > a 6 bit order. Then you don't need all this stuff either.
> 
> I am guessing you mean to store the order in the table descriptor
> itself, instead of having a different table for each order.

Not quite, I mean to index the per-order sub trees by using the high
order bits. You still end up with N seperate bitmap trees, but instead
of using an xarray to hold their top pointers you hold them in a 5th
level.

> Though now that I think of it, it is probably much simpler to just use
> khomem_desc_t orders[NR_PAGE_ORDERS] instead of the xarray.

Which is basically this, but encoding the index to orders in the address

Jason