Re: [PATCH] VMware balloon: force compiling as a module

From: Dmitry Torokhov
Date: Wed Jun 30 2010 - 17:39:51 EST

On Wednesday, June 30, 2010 02:26:40 pm Chetan Loke wrote:
> Hello Dmitry,
> On Wed, Jun 30, 2010 at 3:27 PM, Dmitry Torokhov <dtor@xxxxxxxxxx> wrote:
> > Hi Chetan,
> >
> > On Wednesday, June 30, 2010 11:42:53 am Chetan Loke wrote:
> >> Q1)Does vmtools handle pvscsi correctly?
> >
> > Yes, as long as it compiled as a module or installer will not overwrite
> > distribution-supplied version unless user explicitly requests installer
> > to clobber it.
> perfect.
> > So far distributions have not tried building their kernels with pvscsi
> > or vmxnet3 built-in, but did so with our ballon driver, which prompted
> > this particular change.
> We are building iso's which will then be used to build/create an ESX
> appliance. So we would need the pvscsi driver from the start.

Well, with typical setup, even though pvscsi is a module, as long as it
is in initramfs it will still be loaded automatically. If you are building
truly custom appliance and require pvscsi built-in you'll have to modify
the tools installer script.

> vNICs
> will be populated post-install. At which point vmxnet[2/3] will
> kick-in via vmtools.

Depending on what you base your appliance vmxnet3 might be already in
the kernel along with pvscsi.

> >> Q2)In case if a VM wants to be a good citizen, is there a way for a
> >> guest to know about the balloon-event?
> >
> > I am not sure I follow. Ballooning supposed to be as transparent as
> > possible...
> This is too product specific. I will send you an email separately.


> >> Q3)What if an app mlock's its memory resources and driver's have
> >> pinned down their pages then how does inflation work?
> >
> > We will inflate as much as we can. Obviously if there are no more
> > memory balloon may not grow to its full target size.
> >
> > Balloon driver communicates to the hypervisor the total amount of
> > memory in the guest, we may want to adjust that number by subtracting
> > memory allocated by the kernel, mlocked memory and so on, but it is
> > not done currently.
> Ok.
> I'm stuck with one question -
> A) Ballooning will trigger guest's native memory management policy.
> A.1) So this could mean guest might swap it's pages on it's vdisk,
> correct?


> Consider this setup -
> B) VM1..VMn have backing store(data and OS partitions) on LUNs(SAN).
> Further, data LUNs are mounted as RDMs. I chose RDMs just to keep it
> simple.
> C) Say there's memory pressure. How? Well, few VM's are blasting I/O
> to the LUNs. Plus, a backup triggered. Plus, whatever else happened.
> C.1) VM's now seem to need more and more memory.
> C.2) hypervisors block-layer/other-layers also need more memory.
> C.3) Hypervisor's memory-management algorithm kicks-in.
> ......
> C.3.x) Ballooning triggers - now some VM's (excluding the ones
> from C.1) are giving up memory and if A.1) above is true then the
> guest's pages will be swapped out on the LUNs via
> hypervisor's SCSI-LLDD. But look at C.2) above. Is
> this a soft-deadlock?

If there is no memory something will have to give up. If you look at
the ballon driver you will see that when it switches from non-sleeping
to sleeping allocations or otherwise starts getting allocation errors
it will throttle the inflation rates to give the box a "breather" and
not choke it completely right then and there.

> Oh, it's a linux-guest and if C.1) timesout then the guest will send
> aborts and eventually a LUN reset ;).
> In this particular case, if my suspicion is valid and if all the
> signatures match(swap is out on the SAN, block-congestion etc) then
> the balloon driver could just bail out.

Yes, it is not guaranteed that ballon will reach this target, and in
this case host itself might start swapping causing severe performance

Realistically it all boils down to this: even though you may overcommit
you still have to adequately provision your hosts so they could handle
the load.


To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at