Re: [linus:master] [x86/bugs] a3af84b0fa: stress-ng.seccomp.ops_per_sec 95.7% regression

[Chengfeng Lin]

Hi Pawan, Dave and Daniel,

I found another workload affected by a3af84b0fa00 while running MMTests:
Redis ZPOPMIN throughput on v7.3 is about 13% lower than on v7.1.3.
Unlike the repeated filter loading discussed here, the slowdown remains
after the tasks that triggered the predictor flush (IBPB) have exited.
Can this cost be reduced while preserving the security protection?

The machine is an i7-12700KF with 32 GiB RAM. All runs used CPU microcode
0x3e, and I did not disable any security mitigations. The test uses
unmodified Redis 6.2.6 and MMTests' redis-benchmark-small recipe:

  redis-benchmark --csv -c 1 -r 50000 -n 100000 -P 256

There is one client, no CPU pinning, and all 20 logical CPUs are available.
After each boot, I ran the full MMTests recipe nine times. Each run kept
all 20 commands, including ZADD before ZPOPMIN, and the recipe's five
internal repetitions:

                           v7.1.3 first       v7.3      v7.1.3 return
  ZPOPMIN (requests/s)        999,366         871,397      1,000,947
  GET (requests/s)          3,548,521       3,551,132      3,547,933
  Complete test time (s)       61.396          61.704         61.301

ZPOPMIN is down 12.87% relative to the mean of the two v7.1.3 baselines.
Within each boot, its coefficient of variation is below 0.75%; the two
baselines differ by 0.16%.
GET is essentially unchanged (+0.08%). The complete test, including
startup and cleanup, takes 0.58% longer. The ZPOPMIN sequence drains the
set without refilling it, so its request rate includes empty results,
not just successful member removals.

Bisection identified a3af84b0fa00. To test the effect of a flush without
changing kernel code, I used its parent, 96cce16e26dd, where Redis was
still fast. On each CPU in turn, a short-lived helper enabled
indirect-branch protection for itself using:

  prctl(PR_SET_SPECULATION_CTRL, PR_SPEC_INDIRECT_BRANCH,
        PR_SPEC_FORCE_DISABLE, 0L, 0L)

After all helpers exited, I started Redis from a controller whose
protection setting was unchanged. Redis did not inherit the helpers'
setting.

ZPOPMIN throughput fell by 11.06% and 11.69% on two separate boots.
After another reboot, I ran the same helpers without changing their
protection settings. Throughput returned to baseline (a 1.33% difference).
Separate diagnostic runs found no BPF JIT allocations or associated IBPB
calls during the Redis test. They did show more samples of mispredicted
returns in kernel code, as on the slow kernel.

The tested v7.3 already includes a9b1f19a6a67, which reduces how often
BPF code allocation triggers an IBPB. This result concerns the cost
after an IBPB, rather than repeatedly loading filters during the
benchmark. I have not determined why the return-prediction penalty
persists after the helpers exit.

The helper used in these tests is attached as ibpb-pulse.c. On a dedicated
test machine, build it before rebooting:

  cc -O2 -Wall -Wextra -o ibpb-pulse ibpb-pulse.c

Boot the parent kernel, 96cce16e26dd, then run as a non-root user:

  for cpu in $(seq 0 19); do ./ibpb-pulse "$cpu" protect || exit 1; done

Then run the Redis test above against a fresh, dedicated Redis instance.
For the control, reboot into the same kernel and replace protect with
sham. The separate boot avoids carrying over the effect of the protect run.
The helper is limited to CPUs 0-19 and checks the initial protection state.
I would be happy to test changes that preserve the security protection.

Thanks,
Chengfeng
#define _GNU_SOURCE
#include <errno.h>
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/prctl.h>
#include <unistd.h>

/* Opt IN to protection for this disposable process only. Never enable
 * speculation, touch an MSR, change a system setting or run a benchmark. */
int main(int argc, char **argv)
{
    char *end;
    cpu_set_t mask, actual;
    if (argc != 3 || (strcmp(argv[2], "sham") && strcmp(argv[2], "protect")) ||
        getuid() == 0)
        return 2;
    errno = 0;
    long cpu = strtol(argv[1], &end, 10);
    if (errno || *end || cpu < 0 || cpu >= 20)
        return 3;
    CPU_ZERO(&mask);
    CPU_SET(cpu, &mask);
    if (sched_setaffinity(0, sizeof(mask), &mask) ||
        sched_getaffinity(0, sizeof(actual), &actual) ||
        CPU_COUNT(&actual) != 1 || !CPU_ISSET(cpu, &actual) || sched_getcpu() != cpu)
        return 4;
    int before = prctl(PR_GET_SPECULATION_CTRL, PR_SPEC_INDIRECT_BRANCH, 0L, 0L, 0L);
    if (before != (PR_SPEC_PRCTL | PR_SPEC_ENABLE)) {
        fprintf(stderr, "unexpected initial speculation mask %d errno %d\n", before, errno);
        return 5;
    }
    int protect = !strcmp(argv[2], "protect");
    if (protect && prctl(PR_SET_SPECULATION_CTRL, PR_SPEC_INDIRECT_BRANCH,
                         PR_SPEC_FORCE_DISABLE, 0L, 0L)) {
        perror("protection opt-in");
        return 6;
    }
    int after = prctl(PR_GET_SPECULATION_CTRL, PR_SPEC_INDIRECT_BRANCH, 0L, 0L, 0L);
    if (after != (protect ? (int)(PR_SPEC_PRCTL | PR_SPEC_FORCE_DISABLE) : before) ||
        sched_getcpu() != cpu)
        return 7;
    printf("{\"cpu\":%ld,\"pid\":%ld,\"uid\":%ld,\"mode\":\"%s\","
           "\"before\":%d,\"after\":%d}\n",
           cpu, (long)getpid(), (long)getuid(), argv[2], before, after);
    return 0;
}