Re: [PATCH] scripts: `make rust-analyzer` for out-of-tree modules

[Alice Ferrazzi]

On Thu, Jan 19, 2023 at 12:02:20AM +0800, Vinay Varma wrote:
> Adds support for out-of-tree rust modules to use the `rust-analyzer`
> make target to generate the rust-project.json file.
> 
> The change involves adding an optional parameter `external_src` to the
> `generate_rust_analyzer.py` which expects the path to the out-of-tree
> module's source directory. When this parameter is passed, I have chosen
> not to add the non-core modules (samples and drivers) into the result
> since these are not expected to be used in third party modules. Related
> changes are also made to the Makefile and rust/Makefile allowing the
> `rust-analyzer` target to be used for out-of-tree modules as well.
> 
> Signed-off-by: Vinay Varma <varmavinaym@xxxxxxxxx>
> ---
>  Makefile                          | 12 +++++++-----
>  rust/Makefile                     |  6 ++++--
>  scripts/generate_rust_analyzer.py | 14 +++++++++-----
>  3 files changed, 20 insertions(+), 12 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index f41ec8c8426b..a055a316d2a4 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1831,11 +1831,6 @@ rustfmt:
>  rustfmtcheck: rustfmt_flags = --check
>  rustfmtcheck: rustfmt
>  
> -# IDE support targets
> -PHONY += rust-analyzer
> -rust-analyzer:
> -	$(Q)$(MAKE) $(build)=rust $@
> -
>  # Misc
>  # ---------------------------------------------------------------------------
>  
> @@ -1888,6 +1883,7 @@ help:
>  	@echo  '  modules         - default target, build the module(s)'
>  	@echo  '  modules_install - install the module'
>  	@echo  '  clean           - remove generated files in module directory only'
> +	@echo  '  rust-analyzer	  - generate rust-project.json rust-analyzer support file'
>  	@echo  ''
>  
>  endif # KBUILD_EXTMOD
> @@ -2022,6 +2018,12 @@ quiet_cmd_tags = GEN     $@
>  tags TAGS cscope gtags: FORCE
>  	$(call cmd,tags)
>  
> +# IDE support targets
> +PHONY += rust-analyzer
> +rust-analyzer:
> +	$(Q)$(MAKE) $(build)=rust $@
> +
> +
>  # Script to generate missing namespace dependencies
>  # ---------------------------------------------------------------------------
>  
> diff --git a/rust/Makefile b/rust/Makefile
> index 8f598a904f38..41c1435cd8d4 100644
> --- a/rust/Makefile
> +++ b/rust/Makefile
> @@ -389,8 +389,10 @@ quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L
>  	$(if $(rustc_objcopy),;$(OBJCOPY) $(rustc_objcopy) $@)
>  
>  rust-analyzer:
> -	$(Q)$(srctree)/scripts/generate_rust_analyzer.py $(srctree) $(objtree) \
> -		$(RUST_LIB_SRC) > $(objtree)/rust-project.json
> +	$(Q)$(srctree)/scripts/generate_rust_analyzer.py \
> +		$(abs_srctree) $(abs_objtree) \
> +		$(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \
> +		$(if $(KBUILD_EXTMOD),$(extmod_prefix),$(objtree))/rust-project.json
>  
>  $(obj)/core.o: private skip_clippy = 1
>  $(obj)/core.o: private skip_flags = -Dunreachable_pub
> diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
> index ecc7ea9a4dcf..1546b80db554 100755
> --- a/scripts/generate_rust_analyzer.py
> +++ b/scripts/generate_rust_analyzer.py
> @@ -8,8 +8,9 @@ import json
>  import logging
>  import pathlib
>  import sys
> +import os
>  
> -def generate_crates(srctree, objtree, sysroot_src):
> +def generate_crates(srctree, objtree, sysroot_src, external_src):
>      # Generate the configuration list.
>      cfg = []
>      with open(objtree / "include" / "generated" / "rustc_cfg") as fd:
> @@ -65,7 +66,7 @@ def generate_crates(srctree, objtree, sysroot_src):
>          [],
>          is_proc_macro=True,
>      )
> -    crates[-1]["proc_macro_dylib_path"] = "rust/libmacros.so"
> +    crates[-1]["proc_macro_dylib_path"] = f"{objtree}/rust/libmacros.so"
>  
>      append_crate(
>          "build_error",
> @@ -98,13 +99,15 @@ def generate_crates(srctree, objtree, sysroot_src):
>      # Then, the rest outside of `rust/`.
>      #
>      # We explicitly mention the top-level folders we want to cover.
> -    for folder in ("samples", "drivers"):
> +    extra_src_dirs = ["samples", "drivers"] if external_src is None else [external_src]
> +
> +    for folder in extra_src_dirs:
>          for path in (srctree / folder).rglob("*.rs"):
>              logging.info("Checking %s", path)
>              name = path.name.replace(".rs", "")
>  
>              # Skip those that are not crate roots.
> -            if f"{name}.o" not in open(path.parent / "Makefile").read():
> +            if os.path.exists(path.parent / "Makefile") and f"{name}.o" not in open(path.parent / "Makefile").read():
>                  continue
>  
>              logging.info("Adding %s", name)
> @@ -123,6 +126,7 @@ def main():
>      parser.add_argument("srctree", type=pathlib.Path)
>      parser.add_argument("objtree", type=pathlib.Path)
>      parser.add_argument("sysroot_src", type=pathlib.Path)
> +    parser.add_argument("exttree", type=pathlib.Path, nargs='?')

I would prefer to don't mix '' and "" unless necessary

>      args = parser.parse_args()
>  
>      logging.basicConfig(
> @@ -131,7 +135,7 @@ def main():
>      )
>  
>      rust_project = {
> -        "crates": generate_crates(args.srctree, args.objtree, args.sysroot_src),
> +        "crates": generate_crates(args.srctree, args.objtree, args.sysroot_src, args.exttree),
>          "sysroot_src": str(args.sysroot_src),
>      }
>  
> -- 
> 2.39.0

I could test it on my enviroment with rust-out-of-tree-module and rustc
1.66
it correctly generated a json file rust-project.json with correct path

As following from github conversation:
Reviewed-by: Alice Ferrazzi <alice.ferrazzi@xxxxxxxxxxxxxxxx>
Tested-by: Alice Ferrazzi <alice.ferrazzi@xxxxxxxxxxxxxxxx>

Thanks,
Alicef