Re: [PATCH RFC 0/4] selftests: harness: Provide global metadata pointer to allow clean teardown from selftest libraries

[Ackerley Tng]

Sean Christopherson <seanjc@xxxxxxxxxx> writes:

> On Tue, May 19, 2026, Ackerley Tng wrote:
>> Sean Christopherson <seanjc@xxxxxxxxxx> writes:
>>
>> > On Tue, Apr 14, 2026, Ackerley Tng wrote:
>> >> Sean suggested using setjmp and longjmp [1] to back to the top level
>> >> TEST_F(). I looked at [1] and found myself wishing to use TEST_F() the from
>> >> kselftest harness directly.
>> >
>> > Can you elaborate?  If you have a need for direct TEST_F() in KVM selftests, odds
>> > are good someone/something else will have a similar need, sooner or later.
>> >
>>
>> I guess I like the consistency of working with TEST_F(), there's also
>> TEST_F_TIMEOUT() and friends and all the usefulness of the rest of the
>> kselftest_harness as you described, all of which will probably need
>> re-wrapping if we proceed with the approach of wrapping.
>
> FWIW, utilizing the TIMEOUT functionality could get tricky.  KVM tests often have
> highly variable runtimes based on the underlying environment.  E.g. as an extreme
> case, see the never ending game of whack-a-mole we've been playing with flavors
> of KVM-Unit-Tests' access test[*].
>
> I can definitely see it being useful, e.g. for tests where we *know* the runtime
> is O(ms), just want to call out that this is yet another case where KVM tests tend
> to have more annoying requirements than other selftests.
>
> [*] https://lore.kernel.org/all/20260317225327.4068448-1-yosry@xxxxxxxxxx
>

I guess today's KVM selftests are the equivalent of "never timeout", so
that could be a separate enhancement to kselftest_harness.

We could use kselftest_harness where it is useful in KVM selftests, not
as a replacement :)

>> >> Another option would be to expose the current teardown function pointer
>> >> globally instead of the pointer to the entire metadata struct.
>> >
>> > I'm strongly opposed to any idea effectively requires special casing KVM selftests
>> > in the common harness.  In my experience, the common harness is already quite
>> > brittle, in large part because there is no singular maintainer(s) that is responsible
>> > for ensuring changes work for all downstream users.  Adding odd bits of code that
>> > is only ever used by a handful of KVM selftests is only going to increase the
>> > probability that that code is broken.
>>
>> If Kees likes the idea of exposing a pointer to the metadata globally,
>> does that make KVM selftests less "special"?
>
> Yes.  I don't particuarly care about the code, I just don't want to be in a
> situation where KVM is doing something "weird" relative to the rest of the world.
>
>> If the community likes the global metadata pointer, I could update the
>> harness to adopt the global metadata pointer and then KVM wouldn't be
>> special at all.

On the PUCK call today Sean asked if this would help with the
TEST_REQUIRE(requirement) problem [1], where we want to skip this test
if a requirement is not met.

Having a metadata pointer helps specifically for the KVM_ONE_VCPU_TEST()
wrapper. We can't define a generic TEST_REQUIRE(), but SKIP() would
work, since you can define the SKIP() action to be return, which would
return out of the __suite##_##test() function and effectively skip the
test.

The SKIP() action needs to be handled all the way up to the top level
TEST(), so there's no generic TEST_REQUIRE() to be defined.

Defining TEST_REQUIRE() as SKIP(<call teardown>, abort(), "message")
would work but that only helps with teardown and doesn't continue with
the rest of the tests. To continue with the other tests, I can't really
think of a good option other than setjmp/longjmp.


It seems like the problem KVM_ONE_VCPU_TEST() solves is a FIXTURE
configuration problem. The vCPU could be set up in the FIXTURE_SETUP(),
but there's no good way to parametrize the fixture setup code. Did I
understand correctly?

I'll have that issue in the guest_memfd conversion tests too, leading to
lots of kselftest_harness wrappers. e.g. INIT_PRIVATE [2], then
INIT_SHARED [3]. There is FIXTURE_VARIANT_ADD, but the test code isn't
the same for the INIT_PRIVATE and INIT_SHARED tests.


Is the kselftest_harness-native way to parametrize fixture setup to
create different fixtures? Like FIXTURE(gmem_conversions_init_private),
FIXTURE(gmem_conversions_init_shared) and
FIXTURE(gmem_conversions_init_shared_4_pages),
FIXTURE(gmem_conversions_init_shared_8_pages)?

I guess FIXTURE_SETUP() does have access to the fixture name so setup
parameters could also be encoded in the fixture name.

[1] https://lore.kernel.org/all/ZjUwqEXPA5QVItyX@xxxxxxxxxx/
[2] https://lore.kernel.org/all/20260507-gmem-inplace-conversion-v6-28-91ab5a8b19a4@xxxxxxxxxx/
[3] https://lore.kernel.org/all/20260507-gmem-inplace-conversion-v6-29-91ab5a8b19a4@xxxxxxxxxx/