Feedback on 4.9 performance after PTI fixes

[Willy Tarreau]

Hi,

I managed to take a bit of time to run some more tests on PTI both
native and hosted in KVM, on stable versions built with
CONFIG_PAGE_TABLE_ISOLATION=y. Here it's 4.9.75, used both on the
host and the VM. I could compare pti=on/off both in the host and the
VM. A single CPU was exposed in the VM.

It was running on my laptop (core i7 3320M at 2.6 GHz, 3.3 GHz single
core turbo).

The test was run on haproxy's ability to forward connections. The
results are below :

   Host  |  Guest  | conn/s  | ratio_to_host |  ratio_to_VM | Notes
---------+---------+---------+---------------+--------------+----------------
 pti=off |   -     |  27400  |     100.0%    |      -       | host reference
 pti=off | pti=off |  24200  |      88.3%    |    100.0%    | VM reference 
 pti=off | pti=on  |  13300  |      48.5%    |     55.0%    |
 pti=on  |   -     |  23800  |      86.9%    |      -	    | protected host
 pti=on  | pti=off |  23100  |      84.3%    |     95.5%    |
 pti=on  | pti=on  |  13300  |      48.5%    |     55.0%    |

The ratio_to_host column shows the performance relative to the host
with pti=off. The ratio_to_VM column shows the performance relative to
the VM running with pti=off in a host also having pti=off (ie:
performance before upgrading the systems).

On this test we see a few things :
  - the performance impact on the native host is around 13%

  - the highest performance impact on VMs comes from having PTI on the
    guest kernel (-45%). At this point it makes no difference whether
    the host kernel has it or not.

  - the host kernel's protection has a very limited impact on the guest
    system's performance (-4.5%), which is probably nice for some cloud
    users who might want to take the risk of turning the protection off
    on their VMs.

The impact inside VMs is quite big but it's not where we usuall install
processes sensitive to syscall performance. I could find an even higher
impact on a packet generation program dropping from 2.5 Mpps to 600kpps
in the VM after the fix, but it doesn't make much sense to do this in
VMs so I don't really care.

I have not yet tried the retpoline patches.

Regards,
Willy