Re: External modules with O=... (was: Re: [PATCH] kbuild: Fix include path in scripts/Makefile.modpost)
From: Lucas De Marchi
Date: Wed Sep 11 2024 - 11:39:44 EST
On Wed, Sep 11, 2024 at 04:28:30PM GMT, Masahiro Yamada wrote:
On Wed, Sep 11, 2024 at 1:36 PM Lucas De Marchi
<lucas.demarchi@xxxxxxxxx> wrote:
On Tue, Sep 10, 2024 at 09:43:14PM GMT, Lucas De Marchi wrote:
>On Wed, Sep 11, 2024 at 09:10:09AM GMT, Jing Leng wrote:
>>>-----Original Messages-----
>>>From: "Lucas De Marchi" <lucas.demarchi@xxxxxxxxx>
>>>Send time:Tuesday, 09/10/2024 22:00:29
>>>To: "Masahiro Yamada" <masahiroy@xxxxxxxxxx>
>>>Cc: 3090101217@xxxxxxxxxx, "Michal Marek" <michal.lkml@xxxxxxxxxxx>, "Nick
>>> Desaulniers" <ndesaulniers@xxxxxxxxxx>, "Linux Kbuild mailing list" <linux-kbuild@xxxxxxxxxxxxxxx>, "Linux Kernel Mailing List" <linux-kernel@xxxxxxxxxxxxxxx>, "Jing Leng" <jleng@xxxxxxxxxxxxx>
>>>Subject: External modules with O=... (was: Re: [PATCH] kbuild: Fix include path in scripts/Makefile.modpost)
>>>
>>>Hi, I was pointed to this thread since I'm trying something similar
>>>in kmod's testsuite. See below.
>>>
>>>On Tue, May 24, 2022 at 02:52:45AM GMT, Masahiro Yamada wrote:
>>>>On Tue, May 17, 2022 at 7:51 PM <3090101217@xxxxxxxxxx> wrote:
>>>>>
>>>>> From: Jing Leng <jleng@xxxxxxxxxxxxx>
>>>>>
>>>>> When building an external module, if users don't need to separate the
>>>>> compilation output and source code, they run the following command:
>>>>> "make -C $(LINUX_SRC_DIR) M=$(PWD)". At this point, "$(KBUILD_EXTMOD)"
>>>>> and "$(src)" are the same.
>>>>>
>>>>> If they need to separate them, they run "make -C $(KERNEL_SRC_DIR)
>>>>> O=$(KERNEL_OUT_DIR) M=$(OUT_DIR) src=$(PWD)". Before running the
>>>>> command, they need to copy "Kbuild" or "Makefile" to "$(OUT_DIR)" to
>>>>> prevent compilation failure.
>>>>>
>>>>> So the kernel should change the included path to avoid the copy operation.
>>>>>
>>>>> Signed-off-by: Jing Leng <jleng@xxxxxxxxxxxxx>
>>>>> ---
>>>>> scripts/Makefile.modpost | 3 +--
>>>>> 1 file changed, 1 insertion(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
>>>>> index 48585c4d04ad..0273bf7375e2 100644
>>>>> --- a/scripts/Makefile.modpost
>>>>> +++ b/scripts/Makefile.modpost
>>>>> @@ -87,8 +87,7 @@ obj := $(KBUILD_EXTMOD)
>>>>> src := $(obj)
>>>>>
>>>>> # Include the module's Makefile to find KBUILD_EXTRA_SYMBOLS
>>>>> -include $(if $(wildcard $(KBUILD_EXTMOD)/Kbuild), \
>>>>> - $(KBUILD_EXTMOD)/Kbuild, $(KBUILD_EXTMOD)/Makefile)
>>>>> +include $(if $(wildcard $(src)/Kbuild), $(src)/Kbuild, $(src)/Makefile)
>>>>>
>>>>> # modpost option for external modules
>>>>> MODPOST += -e
>>>>> --
>>>>> 2.17.1
>>>>>
>>>>
>>>>
>>>>I do not think "M=$(OUT_DIR) src=$(PWD)" is the official way,
>>>>but this patch is a clean up.
>>>
>>>I tried what is in this patch and also tried to find an official way in
>>>the docs.
There is no official way.
>>>
>>>In kmod's testsuite we build dummy kernel modules to exercise the API.
>>>https://git.kernel.org/pub/scm/utils/kernel/kmod/kmod.git/tree/testsuite/module-playground
>>>
>>>This works:
>>> make -C /lib/modules/$(uname -r)/build M=$PWD
>>>
>>>This doesn't:
>>> make -C /lib/modules/$(uname -r)/build M=$PWD O=/tmp/kmod_test_modules
O= points the output directory of the kernel,
not the output directory of the external modules.
/lib/modules/$(uname -r)/build
is the clean source tree.
/tmp/kmod_test_modules
contains the build artifacts of vmlinux and
in-tree modules.
Then, the command you gave would work.
>>>
>>>I also tried the variants above with setting src, but all of them give
>>>me errors - I used 6.10 and 6.11-rc7 for these tests.
>>>
>>>Is there a way to do this?
>>>
>>>thanks
>>>Lucas De Marchi
>>>
>>>>
>>>>Applied to linux-kbuild. Thanks.
>>>>
>>>>
>>>>--
>>>>Best Regards
>>>>Masahiro Yamada
>>
>>Hi Masahiro,
>
>I guess you meant Lucas :)
>
>>
>>I think your intention is to separate the source code from the compiled output.
>>The correct command should be:
>> make -C /lib/modules/$(uname -r)/build src=$PWD M=/tmp/kmod_test_modules
>
>oh, looks like this works. Apparently my mistake was trying to set O=
>like I normally do for in-tree modules.
spoke too early... It worked because I was in another machine pointing
to a 6.8 kernel. It seems like something broke between 6.9 and 6.10.
Running a quick bisect, it's pointing to this commit:
9a0ebe5011f4 ("kbuild: use $(obj)/ instead of $(src)/ for common pattern rules")
Overriding 'src' from the command is not allowed. That's why.
Error like below:
$ make -j$(nproc) -C ~/p/linux-dim/src MddPWD/build srcx=$PWD
make: Entering directory '/home/ldmartin/p/linux-dim/src'
make[2]: *** No rule to make target '/home/ldmartin/p/kmod/testsuite/module-playground/build/mod-simple.o', needed by '/home/ldmartin/p/kmod/testsuite/module-playground/build/'. Stop.
make[1]: *** [/home/ldmartin/p/linux-dim/src/Makefile:1922: /home/ldmartin/p/kmod/testsuite/module-playground/build] Error 2
make: *** [Makefile:240: __sub-make] Error 2
make: Leaving directory '/home/ldmartin/p/linux-dim/src'
I suggested M=relative-path + VPATH=
but I do not know what you want to achieve.
https://lore.kernel.org/linux-kbuild/CAK7LNATGGibmjZzYX_A2SkJthmOPbKw2K3R7JYuHTWzgGL2Zjg@xxxxxxxxxxxxxx/
but that is only for in-tree modules, not external modules, right?
Quick reproducer on what I expected would work:
git clone https://git.kernel.org/pub/scm/utils/kernel/kmod/kmod.git
cd kmod/testsuite/module-playground
make -C $KDIR M=$PWD O=$PWD/build
So... did I get the below statement right?
There never was an official way to build external modules with
source and build in different directories. Up until 6.9 we
could pass `-C <kernel_srcdir> M=<mod_builddir> src=<mod_srcdir>`
for unofficial way. After that there's no alternative.
thanks
Lucas De Marchi