Re: [benchmark] 1% performance overhead of paravirt_ops on nativekernels

From: Ingo Molnar
Date: Sat May 30 2009 - 06:24:26 EST

* Nick Piggin <npiggin@xxxxxxx> wrote:

> FWIW, we had to disable paravirt in our default SLES11 kernel.
> (admittedly this was before some of the recent improvements were
> made). But there are only so many 1% performance regressions you
> can introduce before customers won't upgrade (or vendors won't
> publish benchmarks with the new software).
> But OTOH, almost any bit feature is going to cost performance. I don't
> think this is something new (as noted with CONFIG_SECURITY). [...]

Yes in a way, but the difference is that:

- i noted CONFIG_SECURITY as the _worst current example_. It is the
largest-overhead feature known to me in this area, and i
benchmark the kernel a lot. CONFIG_PARAVIRT has _even more_
overhead so it takes the (dubious) top spot in this category.

- CONFIG_SECURITY, in the distros where it's enabled (most of them)
it is actually being relied on by the default user-space. It's
being configured and every default install of the distro has a
real (or at least perceived) advantage from it.

Not so with CONFIG_PARAVIRT. That feature is almost fully parasitic
to native environments: currently it brings no advantage on native
hardware _at all_ (and 95% of the users dont care about Xen).

Still it's impractical for a distro to disable it because adding a
separate kernel package has high costs too and PARAVIRT=y is needed
for the weird execution environment that Xen requires to run Linux
with acceptable speed.

It's as if we paid a full 1% overhead from the requirements of say
Centaur CPUs, on all other CPUs (Intel and AMD). That would be
inacceptable to owners of Intel and AMD CPUs: and so would it be
inacceptable to distros to have a separate kernel package for it.

A distro is unable to hold the tide of creeping bloat - they dont
have the long-term view (and probably shouldnt have it - a distro
should care about and maximize the here-and-now utility of Linux
mostly). Upstream maintainers are the ones to say "NO" to such crap,
even if it's unpopular. In this current case that's me i guess.

Note what _is_ acceptable and what _is_ doable is to be a bit more
inventive when dumping this optional, currently-high-overhead
paravirt feature on us. My message to Xen folks is: use dynamic
patching, fix your hypervisor and just use plain old-fashioned
_restraint_ and common sense when engineering things, and for
heaven's sake, _care_ about the native kernel's performance because
in the long run it's your bread and butter too.

1% overhead (paid by _everyone_ who runs that distro kernel) for
something 95%+ of the users _wont ever use_ is _not acceptable_.

So unless this overhead is brought down significantly i see the dom0
patches as outright harmful: upstream dom0 makes PARAVIRT=Y even
_harder_ for distros to disable, and there will be even _less_
incentive for Xen folks to get the native bloat they caused under

Note that specific pieces of the dom0 patches, like the io-apic
driver patches, are still acceptable of course, as they improve the
overall quality of the kernel. Helping dom0 too as a 'side effect'
is fully acceptable as long as everyone else is helped too.

And if the performance problems are largely fixed (say 0.1%-0.2%
overhead in the benchmarks i did would be acceptable IMO), and if
the patches are all squeaky-clean, i suspect we can do upstream dom0
as well.

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at