Re: [RFC PATCH] kbuild: Add option to generate a Compilation Database

From: Masahiro Yamada
Date: Tue Jun 18 2019 - 11:22:10 EST


On Fri, Jun 7, 2019 at 5:30 AM Raul E Rangel <rrangel@xxxxxxxxxxxx> wrote:
>
> Clang tooling requires a compilation database to figure out the build
> options for each file. This enables tools like clang-tidy and
> clang-check.
>
> See https://clang.llvm.org/docs/HowToSetupToolingForLLVM.html for more
> information.
>
> Normally cmake is used to generate the compilation database, but the
> linux kernel uses make. Another option is using
> [BEAR](https://github.com/rizsotto/Bear) which instruments
> exec to find clang invocations and generate the database that way.
>
> Clang 4.0.0 added the -MJ option to generate the json for each
> compilation unit. https://reviews.llvm.org/D27140
>
> This patch takes advantage of the -MJ option. So it only works for
> Clang.
>
> Signed-off-by: Raul E Rangel <rrangel@xxxxxxxxxxxx>
> ---
> I have a couple TODOs in the code that I would like some feedback on.
> Specifically why extra-y doesn't seem to work in the root Makefile.
> Also, is there a way to add the correct list of prerequisites to the
> compile_commands.json target?
>
> Thanks,
> Raul
>
>
> Makefile | 20 ++++++++++++++++++++
> lib/Kconfig.debug | 7 +++++++
> scripts/Makefile.build | 9 ++++++++-
> 3 files changed, 35 insertions(+), 1 deletion(-)
>
> diff --git a/Makefile b/Makefile
> index a61a95b6b38f7..06067ee18ff64 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1663,6 +1663,26 @@ quiet_cmd_tags = GEN $@
> tags TAGS cscope gtags: FORCE
> $(call cmd,tags)
>
> +# Compilation Database
> +# ---------------------------------------------------------------------------
> +# Generates a compilation database that can be used with the LLVM tools
> +ifdef CONFIG_COMPILATION_DATABASE
> +
> +quiet_cmd_compilation_db = GEN $@
> +cmd_compilation_db = (echo '['; \
> + find "$(@D)" -mindepth 2 -iname '*.json' -print0 | xargs -0 cat; \
> + echo ']') > "$(@D)/$(@F)"


Using 'find' has the same problem as
scripts/gen_compile_commands.py does.

Unless we start build from the clean source tree,
compile_commands.json will contain stale entries
from the previous building.

The motivation is
to replace scripts/gen_compile_commands.py entirely?

Generally, we do not need two ways to do the same thing...





> +# Make sure the database is built when calling `make` without a target.
> +# TODO: Using extra-y doesn't seem to work.
> +_all: $(obj)/compile_commands.json
> +
> +# TODO: Is there a variable that contains all the object files created by
> +# cmd_cc_o_c? Depending on `all` is kind of a hack
> +$(obj)/compile_commands.json: all FORCE
> + $(call if_changed,compilation_db)
> +endif
> +
> # Scripts to check various things for consistency
> # ---------------------------------------------------------------------------
>
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index eae43952902eb..46fceb1fff3d9 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -238,6 +238,13 @@ config GDB_SCRIPTS
> instance. See Documentation/dev-tools/gdb-kernel-debugging.rst
> for further details.
>
> +config COMPILATION_DATABASE
> + bool "Generate a compilation database"
> + depends on CLANG_VERSION >= 40000
> + help
> + This creates a JSON Compilation Database (compile_commands.json)
> + that is used by the clang tooling (clang-tidy, clang-check, etc).
> +
> config ENABLE_MUST_CHECK
> bool "Enable __must_check logic"
> default y
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index ae9cf740633e1..0017bf397292d 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -145,8 +145,15 @@ $(obj)/%.ll: $(src)/%.c FORCE
> # The C file is compiled and updated dependency information is generated.
> # (See cmd_cc_o_c + relevant part of rule_cc_o_c)
>
> +ifdef CONFIG_COMPILATION_DATABASE
> +# TODO: Should we store the json in a temp variable and only copy it to the
> +# final name when the content is different? In theory we could avoid having to
> +# generate the compilation db if the json did not change.
> +compdb_flags = -MJ $(@D)/.$(@F).json
> +endif
> +
> quiet_cmd_cc_o_c = CC $(quiet_modtag) $@
> - cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $<
> + cmd_cc_o_c = $(CC) $(c_flags) $(compdb_flags) -c -o $@ $<
>
> ifdef CONFIG_MODVERSIONS
> # When module versioning is enabled the following steps are executed:
> --
> 2.22.0.rc1.311.g5d7573a151-goog
>


--
Best Regards
Masahiro Yamada