Re: [Xen-devel] [Patch V3 14/15] xen: allow more than 512 GB of RAM for 64 bit pv-domains

[Juergen Gross]

On 05/27/2015 07:05 PM, David Vrabel wrote:
> On 27/05/15 17:25, David Vrabel wrote:
> > On 20/04/15 06:23, Juergen Gross wrote:
> > > 64 bit pv-domains under Xen are limited to 512 GB of RAM today. The
> > > main reason has been the 3 level p2m tree, which was replaced by the
> > > virtual mapped linear p2m list. Parallel to the p2m list which is
> > > being used by the kernel itself there is a 3 level mfn tree for usage
> > > by the Xen tools and eventually for crash dump analysis. For this tree
> > > the linear p2m list can serve as a replacement, too. As the kernel
> > > can't know whether the tools are capable of dealing with the p2m list
> > > instead of the mfn tree, the limit of 512 GB can't be dropped in all
> > > cases.
> > >
> > > This patch replaces the hard limit by a kernel parameter which tells
> > > the kernel to obey the 512 GB limit or not. The default is selected by
> > > a configuration parameter which specifies whether the 512 GB limit
> > > should be active per default for domUs (domain save/restore/migration
> > > and crash dump analysis are affected).
> > >
> > > Memory above the domain limit is returned to the hypervisor instead of
> > > being identity mapped, which was wrong anyway.
> > >
> > > The kernel configuration parameter to specify the maximum size of a
> > > domain can be deleted, as it is not relevant any more.
> >
> > Something in this patch breaks the hvc console in my test domU.
> >
> > kernel BUG at /local/davidvr/work/k.org/tip/drivers/tty/hvc/hvc_xen.c:153
> >
> > Which suggests the hvc driver mapped the wrong console ring frame.
>
> Sorry, it's patch #13 (xen: move p2m list if conflicting with e820 map)
> that seems to be bad.

I think I've found the reason: the console frame isn't being marked as
"reserved" any more. With moving the p2m list I had to change the call
of memblock_reserve() for it. Before that patch this call covered the
p2m list, the start_info page, the xenstore page and the console page.
I added a memblock_reserve() for start_info, but failed to do so for
xenstore and console.

I'll modify the patch and respin.

I have to check why I didn't hit this issue. Maybe my test machine was
too large and the memory in question didn't get reused until my test
was finished.


Thanks for testing,

Juergen

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/