Re: [RFC PATCH 04/20] compiler_types.h: Don't instrument __always_inline functions for coverage

[Peter Zijlstra]

On Wed, Oct 28, 2026 at 07:59:28AM -0400, Sasha Levin wrote:
> noinstr code may only call other noinstr functions and __always_inline
> functions. gcov and Clang's source-based code coverage insert their
> instrumentation before inlining, so the instrumentation of an
> __always_inline function ends up in every noinstr function it gets
> inlined into. With Clang's source-based coverage on x86, 2500 of the
> 3449 relocations in .noinstr.text and 333 of the 451 in .cpuidle.text
> of vmlinux.o point to coverage counters or MC/DC bitmaps. GCC's gcov
> does the same. Since these are plain memory updates rather than calls,
> objtool's noinstr validation doesn't notice them.
> 
> KCOV doesn't have this problem, as it instruments the code after
> inlining and checks the attributes of the function that the code ends
> up in.
> 
> Make __always_inline imply __no_profile when a code coverage
> measurement implementation is enabled, so that no noinstr code carries
> coverage instrumentation. The cost is that __always_inline functions
> don't show up in the coverage data at all, including at their callers
> outside of noinstr code. For an x86_64 configuration built with Clang 22
> and MC/DC, that removes 556 of 60788 functions (0.9%), 29261 of 1598305
> regions (1.8%), 11067 of 591108 branches (1.9%) and 445 of 36561 MC/DC
> conditions (1.2%) from the coverage data.

This is pretty vile. But also, since people were trying to use MC/DC for
validation efforts and the like, how does this impact them? If I
understand correctly their previous emails on the topic, this means they
have to go manually audit every single __always_inline function.

Now; I understand this is a tool limitation and there really must not be
instrumentation in noinstr code, since that can run before things like
page-tables are complete (consider the PTI case). So this is the best
option.

Still, I'm left wondering if these tools should be fixed. For instance,
while the front-end emits this instrumentation, the backend could easily
remove it again when it finds itself in the context of a noinstr
function after inlining.

> Leave unoptimized objects alone: GCC refuses to inline an
> __always_inline function across a no_profile_instrument_function
> mismatch at -O0, and those objects don't contain noinstr code.

We have -O0 code? I thought we hard relied on building with at least
-O1, we often get 'fixes' to enable code to build with -O0.

> Signed-off-by: Sasha Levin <sashal@xxxxxxxxxx>
> ---
>  include/linux/compiler_types.h | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
> index c5921f1390079..c3298d29a9425 100644
> --- a/include/linux/compiler_types.h
> +++ b/include/linux/compiler_types.h
> @@ -246,6 +246,19 @@ struct ftrace_likely_data {
>   */
>  #define __inline__ inline
>  
> +/*
> + * noinstr code may only call noinstr and __always_inline functions. Code
> + * coverage instrumentation is added before inlining, so the counters of an
> + * __always_inline function would end up in the noinstr code it gets inlined
> + * into: don't instrument __always_inline functions. GCC refuses to inline
> + * across a no_profile_instrument_function mismatch without optimization, so
> + * leave the rare unoptimized objects alone; they don't contain noinstr code.
> + */
> +#if defined(CONFIG_COVERAGE_KERNEL) && defined(__OPTIMIZE__)
> +#undef __always_inline
> +#define __always_inline inline __attribute__((__always_inline__)) __no_profile
> +#endif
> +
>  /*
>   * GCC does not warn about unused static inline functions for -Wunused-function.
>   * Suppress the warning in clang as well by using __maybe_unused, but enable it
> -- 
> 2.53.0
>