Re: Kernel build system broken in 5.8?

From: Martin Burnicki
Date: Mon Sep 07 2020 - 06:03:42 EST

Next message: Enric Balletbo i Serra: "Re: [PATCH v3] platform: cros_ec: Reduce ligthbar get version command"
Previous message: Bartosz Golaszewski: "Re: [PATCH 03/23] lib: uaccess: provide getline_from_user()"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Hi,

sorry for the late reply. I neede some time figure out what was going
wrong, beside some other tasks.

My driver can be used with all kernels from 2.6 up to the current versions.

In early 2.6 kernels you had to run "make ... modules_add"
to install a module that had been built out-of-tree.

If you ran "make ... modules_install" then *only* the current
out-of-tree module was installed, but *all other* kernel modules were
deleted, leaving the system unusable after the next reboot.

So since many years, my Makefile includes a test to determine if the
make target "modules_add" exists:

make -n ... modules_add

Please note the '-n' in the make command. If this "make" call exited
successfully, "make modules_add" was used to install the module, and
"modules_install" was called otherwise, e.g. for current kernels.

This has been working for many years, up to kernel 5.7.

With kernel 5.8.0-1-default shipped by openSUSE for their Tumbleweed
distro, this test corrupts the pre-configured kernel sources. This
becomes obvious if I run the command "make modules_add" as non-root:

---------------------------------------------------------------------
mbgclock> make -C /lib/modules/`uname -r`/build/ -n modules_add V=1
make: Entering directory '/usr/src/linux-5.8.0-1-obj/x86_64/default'
make -C /usr/src/linux-5.8.0-1-obj/x86_64/default -f
/usr/src/linux-5.8.0-1/Makefile modules_add
mkdir: cannot create directory '.tmp_25862': Permission denied
... ('cannot create' repeated 21 times)
arch/x86/Makefile:147: CONFIG_X86_X32 enabled but no binutils support
mkdir: cannot create directory '.tmp_25907': Permission denied
... ('cannot create' repeated 35 times)
make -f /usr/src/linux-5.8.0-1/Makefile syncconfig
mkdir: cannot create directory '.tmp_26006': Permission denied
... ('cannot create' repeated 13 times)
if [ -f /usr/src/linux-5.8.0-1/.config -o \
-d /usr/src/linux-5.8.0-1/include/config -o \
-d /usr/src/linux-5.8.0-1/arch/x86/include/generated ]; then \
echo >&2 "***"; \
echo >&2 "*** The source tree is not clean, please run\
'make mrproper'"; \
echo >&2 "*** in /usr/src/linux-5.8.0-1";\
echo >&2 "***"; \
false; \
fi
ln -fsn /usr/src/linux-5.8.0-1 source
ln: failed to create symbolic link 'source': Permission denied
make[2]: *** [/usr/src/linux-5.8.0-1/Makefile:557: outputmakefile] Error 1
make[1]: *** [/usr/src/linux-5.8.0-1/Makefile:716:
include/config/auto.conf.cmd] Error 2
make[1]: *** [include/config/auto.conf.cmd] Deleting file
'include/generated/autoconf.h'
make[1]: unlink: include/generated/autoconf.h: Permission denied
make: *** [../../../linux-5.8.0-1/Makefile:185: __sub-make] Error 2
make: Leaving directory '/usr/src/linux-5.8.0-1-obj/x86_64/default'
---------------------------------------------------------------------

One of the last line shows that 'include/generated/autoconf.h' is going
to be deleted, but this just didn't happen because I ran the command
without root permissions.

Of course, autoconf.h *is* deleted if I run the command with root
permission, as required if you really want to install the kernel module.

This happens even though "make" has been called with '-n', even though
the target "modules_add" is invalid, and even worse, it happens if I
call make with *any* invalid target, e.g. "make -C ... M=... xyz".

This is really, really ugly, IMO, and the same problem occurs with a 5.8
kernel for Ubuntu.

Fortunately this doesn't happen if I call the kernel build system
against a vanilla kernel. I've specifically cloned and prepared the
kernel git repo as root to see any "permission denied" errors when I
call the kernel build system as non-root user:

sudo git clone <...>/linux-stable
cd linux-stable
sudo git checkout v5.8
sudo make defconfig
sudo make modules_prepare

---------------------------------------------------------------------
mbgclock> make -C /data-1/projects/linux-stable -n modules_add V=1
make: Entering directory '/data-1/projects/linux-stable'
mkdir: cannot create directory ‘.tmp_20352’: Permission denied
... ('cannot create' repeated ~50 times)
make: *** No rule to make target 'modules_add'. Stop.
make: Leaving directory '/data-1/projects/linux-stable'
---------------------------------------------------------------------

So this returns with error, as expected because the make target is
invalid, and it tries to create temporary files inside the kernel source
tree, but at least it doesn't try to delete any files like autoconf.h.

If I repeat the test above with linux-stable v5.7, I just get the
expected error, but there is no single "permission denied" error due to
the fact that tmp files/dirs can't be created.

Anyway, for my own project I've implemented a workaround that the test
with make target "modules_add" is only made if the kernel major version
is < 3, which fixes these problems for me.

Thanks,

Martin

Masahiro Yamada wrote:
> On Wed, Aug 19, 2020 at 5:50 AM Martin Burnicki
> <martin.burnicki@xxxxxxxxxxx> wrote:
>> Hello Masahiro,
>>
>> Masahiro Yamada wrote:
>
> Can you reproduce the issue in the linux-stable source tree
> instead of the ubuntu kernel?
>
> You can get the stable kernel repository, like follows:
>
> $ git clone
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
> $ cd linux-stable
> $ git checkout v5.8.1

OK, but this shouldn't work without further steps, should it?

If I clone the repo from the root account, the files are "read-only" for
a normal user, and if I try to build the driver as normal user, I get an
error, which is expected in this case:

-----------------------------------------------------------------------
$ BUILD_DIR=../linux-stable-tmp make
make -C ../linux-stable-tmp M=/data-1/projects/hello-drvr
make[1]: Entering directory '/data-1/projects/linux-stable-tmp'

ERROR: Kernel configuration is invalid.
include/generated/autoconf.h or include/config/auto.conf are
missing.
Run 'make oldconfig && make prepare' on kernel src to fix it.

Makefile:648: include/config/auto.conf: No such file or directory
make[1]: *** [Makefile:719: include/config/auto.conf] Error 1
make[1]: Leaving directory '/data-1/projects/linux-stable-tmp'
make: *** [Makefile:8: __default] Error 2
-----------------------------------------------------------------------

'make defconfig' (run as root):

-----------------------------------------------------------------------
HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/confdata.o
HOSTCC scripts/kconfig/expr.o
LEX scripts/kconfig/lexer.lex.c
YACC scripts/kconfig/parser.tab.[ch]
HOSTCC scripts/kconfig/lexer.lex.o
HOSTCC scripts/kconfig/parser.tab.o
HOSTCC scripts/kconfig/preprocess.o
HOSTCC scripts/kconfig/symbol.o
HOSTCC scripts/kconfig/util.o
HOSTLD scripts/kconfig/conf
*** Default configuration is based on 'x86_64_defconfig'
#
# configuration written to .config
#
-----------------------------------------------------------------------

Unlike mentioned in the output quoted above, this command generates lots
of files below include/config, the file include/generated/autoconf.h,
and a number of files below kconfig/.

pc-martin:/data-1/projects/linux-stable-tmp # make modules_prepare
SYSTBL arch/x86/include/generated/asm/syscalls_32.h
SYSHDR arch/x86/include/generated/asm/unistd_32_ia32.h
SYSHDR arch/x86/include/generated/asm/unistd_64_x32.h
SYSTBL arch/x86/include/generated/asm/syscalls_64.h
SYSHDR arch/x86/include/generated/uapi/asm/unistd_32.h
SYSHDR arch/x86/include/generated/uapi/asm/unistd_64.h
SYSHDR arch/x86/include/generated/uapi/asm/unistd_x32.h
HOSTCC arch/x86/tools/relocs_32.o
HOSTCC arch/x86/tools/relocs_64.o
HOSTCC arch/x86/tools/relocs_common.o
HOSTLD arch/x86/tools/relocs
HOSTCC scripts/selinux/genheaders/genheaders
HOSTCC scripts/selinux/mdp/mdp
HOSTCC scripts/kallsyms
HOSTCC scripts/sorttable
HOSTCC scripts/asn1_compiler
HOSTCC scripts/extract-cert
WRAP arch/x86/include/generated/uapi/asm/bpf_perf_event.h
WRAP arch/x86/include/generated/uapi/asm/errno.h
WRAP arch/x86/include/generated/uapi/asm/fcntl.h
WRAP arch/x86/include/generated/uapi/asm/ioctl.h
WRAP arch/x86/include/generated/uapi/asm/ioctls.h
WRAP arch/x86/include/generated/uapi/asm/ipcbuf.h
WRAP arch/x86/include/generated/uapi/asm/param.h
WRAP arch/x86/include/generated/uapi/asm/poll.h
WRAP arch/x86/include/generated/uapi/asm/resource.h
WRAP arch/x86/include/generated/uapi/asm/socket.h
WRAP arch/x86/include/generated/uapi/asm/sockios.h
WRAP arch/x86/include/generated/uapi/asm/termbits.h
WRAP arch/x86/include/generated/uapi/asm/termios.h
WRAP arch/x86/include/generated/uapi/asm/types.h
WRAP arch/x86/include/generated/asm/early_ioremap.h
WRAP arch/x86/include/generated/asm/export.h
WRAP arch/x86/include/generated/asm/mcs_spinlock.h
WRAP arch/x86/include/generated/asm/dma-contiguous.h
WRAP arch/x86/include/generated/asm/irq_regs.h
WRAP arch/x86/include/generated/asm/mm-arch-hooks.h
WRAP arch/x86/include/generated/asm/mmiowb.h
UPD include/config/kernel.release
UPD include/generated/uapi/linux/version.h
UPD include/generated/utsrelease.h
CC scripts/mod/empty.o
HOSTCC scripts/mod/mk_elfconfig
MKELF scripts/mod/elfconfig.h
HOSTCC scripts/mod/modpost.o
CC scripts/mod/devicetable-offsets.s
UPD scripts/mod/devicetable-offsets.h
HOSTCC scripts/mod/file2alias.o
HOSTCC scripts/mod/sumversion.o
HOSTLD scripts/mod/modpost
CC kernel/bounds.s
UPD include/generated/bounds.h
UPD include/generated/timeconst.h
CC arch/x86/kernel/asm-offsets.s
UPD include/generated/asm-offsets.h
CALL scripts/checksyscalls.sh
CALL scripts/atomic/check-atomics.sh
DESCEND objtool
HOSTCC /data-1/projects/linux-stable-tmp/tools/objtool/fixdep.o
HOSTLD /data-1/projects/linux-stable-tmp/tools/objtool/fixdep-in.o
LINK /data-1/projects/linux-stable-tmp/tools/objtool/fixdep
CC /data-1/projects/linux-stable-tmp/tools/objtool/exec-cmd.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/help.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/pager.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/parse-options.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/run-command.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/sigchain.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/subcmd-config.o
LD /data-1/projects/linux-stable-tmp/tools/objtool/libsubcmd-in.o
AR /data-1/projects/linux-stable-tmp/tools/objtool/libsubcmd.a
MKDIR /data-1/projects/linux-stable-tmp/tools/objtool/arch/x86/lib/
GEN
/data-1/projects/linux-stable-tmp/tools/objtool/arch/x86/lib/inat-tables.c
CC /data-1/projects/linux-stable-tmp/tools/objtool/arch/x86/decode.o
LD
/data-1/projects/linux-stable-tmp/tools/objtool/arch/x86/objtool-in.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/weak.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/check.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/special.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/orc_gen.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/orc_dump.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/builtin-check.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/builtin-orc.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/elf.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/objtool.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/libstring.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/libctype.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/str_error_r.o
CC /data-1/projects/linux-stable-tmp/tools/objtool/librbtree.o
LD /data-1/projects/linux-stable-tmp/tools/objtool/objtool-in.o
LINK /data-1/projects/linux-stable-tmp/tools/objtool/objtool

>>
>>> On Wed, Aug 19, 2020 at 1:14 AM Martin Burnicki
>>> <martin.burnicki@xxxxxxxxxxx> wrote:
>> [...]
>>>> I usually build the kernel module as standard user, and only install the
>>>> new module as root, e.g.:
>>>>
>>>> git checkout devel # The 'devel' branch is appropriate for testing
>>>> cd mbgclock # the subdirectory of the kernel module
>>>> git clean -fd; make # or make V=1 for verbose output
>>>> sudo make install
>>>
>>>
>>> This is a bad way for reporting a bug
>>> because your project is too big.
>>
>> Sorry for this. I wasn't sure which was the best way to get a contact to
>> folks who are more familiar with details of the kernel build system, and
>> what may have been changed.
>
> Sorry, if my response was too harsh.
>
>> A possibility could have
>>
>>> mbgclock/Makefile include the top Makefile,
>>> which is more than 1000 lines.
>>
>> If you run 'make' in the mbgclock subdirectory then the kernel build
>> system is just called as usual, i.e.
>>
>> make -C <path-to-the-kernel-build-system> M=$PWD <some EXTRA_CFLAGS>
>
>
> I know.
>
> I tried this, and then
> "sudo make modules_install -C <path-to-the-kernel-build-system> M=$PWD"
> but I did not see the issue.
>
> So, I have no idea what the root-cause is
> at this point.
>
>
>> This has been working without problems for many years, for many popular
>> distros, and all kernels from 2.6 up to 5.7 (except that for older
>> kernels SUBDIRS=$PWD was used instead of M=$PWD).
>>
>>> Please provide steps to reproduce it
>>> with a tiny test module.
>>
>> I was just hoping that someone had an idea like "Oh, this is because xyz
>> has changed in kernel 5.18" when he read what I've observed.
>>
>> In the past I've seen cases where folks e.g. from RedHat or SuSE had
>> backported patches from newer kernels to older ones, causing build
>> errors in my driver code, and I was able to apply workarounds to my
>> source code.
>>
>> However, this case is different because obviously the kernel source tree
>> is modified under certain circumstances. Is my assumption correct that
>> this should not happen if I install Linux header packages for a distro,
>> that have been pre-configured according to the running kernel?
>
> Right.
> External modules should never ever attempt
> to modify the kernel tree since it is often read-only.
>
> I did not change that behavior
> (unless I was doing a horrible mistake)
>
>
>
>>
>> Anyway, I'm going to try to set up an example which makes it easier to
>> duplicate the problem.
>>
>>> You state 5.7.15 was fine.
>>
>> Yes.
>>
>>> So, could you also run git-bisect, please?
>>
>> Sorry, I've just recently started using git, and yet I'm not too
>> familiar with some of the commands. How could git-bisect help in this case?
>
>
>
> "git bisect" is really useful to find the first bad commit.
> If you have not tried it yet,
> it is worth learning the usage.
>
>
>
> Can you reproduce the issue in the linux-stable source tree
> instead of the ubuntu kernel?
>
>
> You can get the stable kernel repository, like follows:
>
> $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
> $ cd linux-stable
> $ git checkout v5.8.1
>
>
> Once you find how to reproduce the bug
> in the git repository,
> it is pretty easy to start bisecting
> with "git bisect start <bad> <good>"
>
> For example,
>
> git bisect start v5.8.1 v5.7.15
>
> Then, repeat "git bisect good" or "git bisect bad",
> depending on whether the bug happens.
>
>
>
>
>
>
>>>> Also, I was under the impression that the kernel source tree should be
>>>> read-only, and kept clean, but this doesn't seem to be the case anymore.
>>>>
>>>>
>>>> For example:
>>>>
>>>> On Ubuntu 19.10 with kernel 5.8.1-050801-generic from
>>>> https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.8.1/
>>>
>>>
>>> I cannot try it since that is not what
>>> I install on my machine, sorry.
>>
>> What distro are you using? Or do you only run kernels you have compiled
>> yourself? ;-)
>
>
> I also use Ubuntu, but my one is 5.4.0-42-generic.
>
>
> I am not sure if 5.8.1-050801-generic
> is exactly the same as v5.8.1, though.
>
>
>
>
>>
>>> Please provide steps to reproduce it
>>> for a person using a different environment.
>>
>> I'm going to try to do this. Do you have any hints what this should look
>> like?
>
>
> I'd like to know the steps to reproduce the bug
> in the kernel source tree instead of the distro's one.
> (See above to get the stable kernel tree)
>
>
>
> We could use your project to debug this, but
> if this is purely a problem of the kernel build system,
> it is easier and faster to use a small hello-world module.
>
>
> For example, the minimal Makefile
> of an external module will look like follows:
>
>
>
> ------------------------>8---------------------------
> KERNEL=$(HOME)/workspace/linux-stable
>
> .PHONY: __default
> __default:
> $(MAKE) -C $(KERNEL) M=$(CURDIR)
>
> .PHONY: $(MAKECMDGOALS)
> $(MAKECMDGOALS):
> $(MAKE) -C $(KERNEL) M=$(CURDIR) $@
>
> obj-m := helloworld.o
> ------------------------>8---------------------------
>
>
>
>
>
>
>>
>> Should this topic be continued on linux-kernel@, or better on
>> linux-kbuild@, or elsewhere?
>
>
> Please include both lists in To or Cc.
>
> linux-kernel@ is a large amount.
>
> I often miss mails if they are only
> sent to linux-kernel@.
>
>
>
>
> Thanks.
>
>
>
>
>>
>>
>> Thanks,
>>
>> Martin
>> --
>> Martin Burnicki
>>
>> Senior Software Engineer
>>
>> MEINBERG Funkuhren GmbH & Co. KG
>> Email: martin.burnicki@xxxxxxxxxxx
>> Phone: +49 5281 9309-414
>> Linkedin: https://www.linkedin.com/in/martinburnicki/
>>
>> Lange Wand 9, 31812 Bad Pyrmont, Germany
>> Amtsgericht Hannover 17HRA 100322
>> Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
>> Andre Hartmann, Heiko Gerstung
>> Websites: https://www.meinberg.de https://www.meinbergglobal.com
>> Training: https://www.meinberg.academy
>>
>
>

--
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burnicki@xxxxxxxxxxx
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de https://www.meinbergglobal.com
Training: https://www.meinberg.academy

Next message: Enric Balletbo i Serra: "Re: [PATCH v3] platform: cros_ec: Reduce ligthbar get version command"
Previous message: Bartosz Golaszewski: "Re: [PATCH 03/23] lib: uaccess: provide getline_from_user()"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]