Re: External modules with O=... (was: Re: [PATCH] kbuild: Fix include path in scripts/Makefile.modpost)

[Nicolas Schier]

On Wed, Sep 11, 2024 at 10:38:58AM -0500, Lucas De Marchi wrote:
> On Wed, Sep 11, 2024 at 04:28:30PM GMT, Masahiro Yamada wrote:
> > On Wed, Sep 11, 2024 at 1:36 PM Lucas De Marchi
> > <lucas.demarchi@xxxxxxxxx> wrote:
> > > 
> > > On Tue, Sep 10, 2024 at 09:43:14PM GMT, Lucas De Marchi wrote:
> > > >On Wed, Sep 11, 2024 at 09:10:09AM GMT, Jing Leng wrote:
> > > >>>-----Original Messages-----
> > > >>>From: "Lucas De Marchi" <lucas.demarchi@xxxxxxxxx>
> > > >>>Send time:Tuesday, 09/10/2024 22:00:29
> > > >>>To: "Masahiro Yamada" <masahiroy@xxxxxxxxxx>
> > > >>>Cc: 3090101217@xxxxxxxxxx, "Michal Marek" <michal.lkml@xxxxxxxxxxx>, "Nick
> > > >>> Desaulniers" <ndesaulniers@xxxxxxxxxx>, "Linux Kbuild mailing list" <linux-kbuild@xxxxxxxxxxxxxxx>, "Linux Kernel Mailing List" <linux-kernel@xxxxxxxxxxxxxxx>, "Jing Leng" <jleng@xxxxxxxxxxxxx>
> > > >>>Subject: External modules with O=... (was: Re: [PATCH] kbuild: Fix include path in scripts/Makefile.modpost)
> > > >>>
> > > >>>Hi, I was pointed to this thread since I'm trying something similar
> > > >>>in kmod's testsuite. See below.
> > > >>>
> > > >>>On Tue, May 24, 2022 at 02:52:45AM GMT, Masahiro Yamada wrote:
> > > >>>>On Tue, May 17, 2022 at 7:51 PM <3090101217@xxxxxxxxxx> wrote:
> > > >>>>>
> > > >>>>> From: Jing Leng <jleng@xxxxxxxxxxxxx>
> > > >>>>>
> > > >>>>> When building an external module, if users don't need to separate the
> > > >>>>> compilation output and source code, they run the following command:
> > > >>>>> "make -C $(LINUX_SRC_DIR) M=$(PWD)". At this point, "$(KBUILD_EXTMOD)"
> > > >>>>> and "$(src)" are the same.
> > > >>>>>
> > > >>>>> If they need to separate them, they run "make -C $(KERNEL_SRC_DIR)
> > > >>>>> O=$(KERNEL_OUT_DIR) M=$(OUT_DIR) src=$(PWD)". Before running the
> > > >>>>> command, they need to copy "Kbuild" or "Makefile" to "$(OUT_DIR)" to
> > > >>>>> prevent compilation failure.
> > > >>>>>
> > > >>>>> So the kernel should change the included path to avoid the copy operation.
> > > >>>>>
> > > >>>>> Signed-off-by: Jing Leng <jleng@xxxxxxxxxxxxx>
> > > >>>>> ---
> > > >>>>>  scripts/Makefile.modpost | 3 +--
> > > >>>>>  1 file changed, 1 insertion(+), 2 deletions(-)
> > > >>>>>
> > > >>>>> diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
> > > >>>>> index 48585c4d04ad..0273bf7375e2 100644
> > > >>>>> --- a/scripts/Makefile.modpost
> > > >>>>> +++ b/scripts/Makefile.modpost
> > > >>>>> @@ -87,8 +87,7 @@ obj := $(KBUILD_EXTMOD)
> > > >>>>>  src := $(obj)
> > > >>>>>
> > > >>>>>  # Include the module's Makefile to find KBUILD_EXTRA_SYMBOLS
> > > >>>>> -include $(if $(wildcard $(KBUILD_EXTMOD)/Kbuild), \
> > > >>>>> -             $(KBUILD_EXTMOD)/Kbuild, $(KBUILD_EXTMOD)/Makefile)
> > > >>>>> +include $(if $(wildcard $(src)/Kbuild), $(src)/Kbuild, $(src)/Makefile)
> > > >>>>>
> > > >>>>>  # modpost option for external modules
> > > >>>>>  MODPOST += -e
> > > >>>>> --
> > > >>>>> 2.17.1
> > > >>>>>
> > > >>>>
> > > >>>>
> > > >>>>I do not think "M=$(OUT_DIR) src=$(PWD)" is the official way,
> > > >>>>but this patch is a clean up.
> > > >>>
> > > >>>I tried what is in this patch and also tried to find an official way in
> > > >>>the docs.
> > 
> > 
> > There is no official way.
> > 
> > > >>>
> > > >>>In kmod's testsuite we build dummy kernel modules to exercise the API.
> > > >>>https://git.kernel.org/pub/scm/utils/kernel/kmod/kmod.git/tree/testsuite/module-playground
> > > >>>
> > > >>>This works:
> > > >>>     make -C /lib/modules/$(uname -r)/build M=$PWD
> > > >>>
> > > >>>This doesn't:
> > > >>>     make -C /lib/modules/$(uname -r)/build M=$PWD O=/tmp/kmod_test_modules
> > 
> > 
> > 
> > O= points the output directory of the kernel,
> > not the output directory of the external modules.
> > 
> > 
> > /lib/modules/$(uname -r)/build
> > is the clean source tree.
> > 
> > /tmp/kmod_test_modules
> > contains the build artifacts of vmlinux and
> > in-tree modules.
> > 
> > Then, the command you gave would work.
> > 
> > 
> > 
> > 
> > 
> > 
> > > >>>
> > > >>>I also tried the variants above with setting src, but all of them give
> > > >>>me errors - I used 6.10 and 6.11-rc7 for these tests.
> > > >>>
> > > >>>Is there a way to do this?
> > > >>>
> > > >>>thanks
> > > >>>Lucas De Marchi
> > > >>>
> > > >>>>
> > > >>>>Applied to linux-kbuild. Thanks.
> > > >>>>
> > > >>>>
> > > >>>>--
> > > >>>>Best Regards
> > > >>>>Masahiro Yamada
> > > >>
> > > >>Hi Masahiro,
> > > >
> > > >I guess you meant Lucas :)
> > > >
> > > >>
> > > >>I think your intention is to separate the source code from the compiled output.
> > > >>The correct command should be:
> > > >>   make -C /lib/modules/$(uname -r)/build src=$PWD M=/tmp/kmod_test_modules
> > > >
> > > >oh, looks like this works. Apparently my mistake was trying to set O=
> > > >like I normally do for in-tree modules.
> > > 
> > > spoke too early... It worked because I was in another machine pointing
> > > to a 6.8 kernel. It seems like something broke between 6.9 and 6.10.
> > > 
> > > Running a quick bisect, it's pointing to this commit:
> > > 9a0ebe5011f4 ("kbuild: use $(obj)/ instead of $(src)/ for common pattern rules")
> > 
> > 
> > Overriding 'src' from the command is not allowed. That's why.
> > 
> > 
> > > Error like below:
> > > 
> > > $ make -j$(nproc) -C ~/p/linux-dim/src MddPWD/build srcx=$PWD
> > > make: Entering directory '/home/ldmartin/p/linux-dim/src'
> > > make[2]: *** No rule to make target '/home/ldmartin/p/kmod/testsuite/module-playground/build/mod-simple.o', needed by '/home/ldmartin/p/kmod/testsuite/module-playground/build/'.  Stop.
> > > make[1]: *** [/home/ldmartin/p/linux-dim/src/Makefile:1922: /home/ldmartin/p/kmod/testsuite/module-playground/build] Error 2
> > > make: *** [Makefile:240: __sub-make] Error 2
> > > make: Leaving directory '/home/ldmartin/p/linux-dim/src'
> > 
> > 
> > I suggested M=relative-path + VPATH=
> > but I do not know what you want to achieve.
> > 
> > https://lore.kernel.org/linux-kbuild/CAK7LNATGGibmjZzYX_A2SkJthmOPbKw2K3R7JYuHTWzgGL2Zjg@xxxxxxxxxxxxxx/
> 
> but that is only for in-tree modules, not external modules, right?
> Quick reproducer on what I expected would work:
> 
> 	git clone https://git.kernel.org/pub/scm/utils/kernel/kmod/kmod.git
> 	cd kmod/testsuite/module-playground
> 	make -C $KDIR M=$PWD O=$PWD/build
> 
> So... did I get the below statement right?
> 
> 	There never was an official way to build external modules with
> 	source and build in different directories. Up until 6.9 we
> 	could pass `-C <kernel_srcdir> M=<mod_builddir> src=<mod_srcdir>`
> 	for unofficial way. After that there's no alternative.

yes, AFAICT your summary describes the situation quite precisely.

Kind regards,
Nicolas