Re: [PATCH v3 1/7] lib: string: add functions to case-convert strings

From: Markus Mayer
Date: Wed Jul 13 2016 - 18:27:05 EST

Next message: Paul E. McKenney: "Re: Severe performance regression w/ 4.4+ on Android due to cgroup locking changes"
Previous message: Luis de Bethencourt: "Re: [RFC PATCH] dynamic_debug: only add header when used"
In reply to: Luis de Bethencourt: "Re: [PATCH v3 1/7] lib: string: add functions to case-convert strings"
Next in thread: Luis de Bethencourt: "Re: [PATCH v3 1/7] lib: string: add functions to case-convert strings"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On 13 July 2016 at 10:19, Luis de Bethencourt <luisbg@xxxxxxxxxxxxxxx> wrote:
> On 11/07/16 23:46, Markus Mayer wrote:
>
> Hi Markus,
>
> Amazing. I see this happening as well, but I know it shouldn't.
>
> The reason the #ifndef guards in headers are there is precisely to allow
> circular dependencies.
>
> The problem in your output reads as:
> strstr() is in string.h
> #include string.h -> that includes kernel.h -> that includes string.h
>
> The third should do nothing based on _LINUX_STRING_H_ being defined already
> and all code inside the #ifndef in string.h not being executed.
> Yet it shouldn't block the first include above since that macro isn't defined,
> which is what the error suggests since it doesn't have strstr()
> If _LINUX_STRING_H is defined, strstr() should be available.
>
> Investigating this issue, it only happens when CONFIG_DYNAMIC_DEBUG is not
> set and line 170 of dynamic_debug.h runs, but just above we have an
> include of string.h.
>
> Very strange that #include <linux/string.h> isn't doing its job.
>
> The first thing I tried is to understand where dynamic_debug.h is used and
> removed the unneeded ones:
> --------------------------------------------------------------------------
> diff --git a/include/linux/kernel.h b/include/linux/kernel.h
> --- a/include/linux/kernel.h
> +++ b/include/linux/kernel.h
> @@ -11,7 +11,6 @@
> #include <linux/log2.h>
> #include <linux/typecheck.h>
> #include <linux/printk.h>
> -#include <linux/dynamic_debug.h>
> #include <asm/byteorder.h>
> #include <uapi/linux/kernel.h>
>
> diff --git a/include/linux/printk.h b/include/linux/printk.h
> --- a/include/linux/printk.h
> +++ b/include/linux/printk.h
> @@ -307,10 +307,11 @@ asmlinkage __printf(1, 2) __cold void __pr_info(const char *fmt, ...);
> no_printk(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
> #endif
>
> -#include <linux/dynamic_debug.h>
>
> /* If you are writing a driver, please use dev_dbg instead */
> #if defined(CONFIG_DYNAMIC_DEBUG)
> +#include <linux/dynamic_debug.h>
> +
> /* dynamic_pr_debug() uses pr_fmt() internally so we don't need it here */
> #define pr_debug(fmt, ...) \
> dynamic_pr_debug(fmt, ##__VA_ARGS__)
> diff --git a/kernel/module.c b/kernel/module.c
> index beaebea..e70a2fa 100644
> --- a/kernel/module.c
> +++ b/kernel/module.c
> @@ -60,6 +60,7 @@
> #include <linux/jump_label.h>
> #include <linux/pfn.h>
> #include <linux/bsearch.h>
> +#include <linux/dynamic_debug.h>
> #include <uapi/linux/module.h>
> #include "module-internal.h"
> --------------------------------------------------------------------------
>
> This diff [0] fixes the issue but it is a workaround for the original
> issue about string.h not being properly included in dynamic_debug.h
>
> Puzzled by this and can't figure out what is happening wrong.
>
> The second thing I tried was adding
> #warning "Linking to string header"
> in include/linux/string.h, and I don't see any include path mentioning
> kernel.h, where do you see the circular dependency? I might be missing
> something.

I did some more poking around and this is what I found.

For starters, the problem happens with kernel/bounds.c. Without
worrying about SIZE_MAX or making any other changes, I added a
#warning line to kernel.h and string.h to see the include sequence.

$ aarch64-linux-gcc -Wp,-MD,kernel/.bounds.s.d -nostdinc -isystem
/opt/toolchain/stbgcc-4.8-1.5/bin/../lib/gcc/aarch64-linux-gnu/4.8.5/include
-I./arch/arm64/include -Iarch/arm64/include/generated/uapi
-Iarch/arm64/include/generated -Iinclude -I./arch/arm64/include/uapi
-Iarch/arm64/include/generated/uapi -I./include/uapi
-Iinclude/generated/uapi -include ./include/linux/kconfig.h
-D__KERNEL__ -mlittle-endian -Wall -Wundef -Wstrict-prototypes
-Wno-trigraphs -fno-strict-aliasing -fno-common
-Werror-implicit-function-declaration -Wno-format-security -std=gnu89
-mgeneral-regs-only -fno-asynchronous-unwind-tables
-fno-delete-null-pointer-checks -O2 --param=allow-store-data-races=0
-Wframe-larger-than=2048 -fno-stack-protector
-Wno-unused-but-set-variable -fno-omit-frame-pointer
-fno-optimize-sibling-calls -fno-var-tracking-assignments -g
-Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow
-fconserve-stack -Werror=implicit-int -Werror=strict-prototypes
-DKBUILD_BASENAME='"bounds"' -DKBUILD_MODNAME='"bounds"'
-fverbose-asm -E -o kernel/bounds.i kernel/bounds.c
In file included from include/asm-generic/bug.h:13:0,
from ./arch/arm64/include/asm/bug.h:62,
from include/linux/bug.h:4,
from include/linux/page-flags.h:9,
from kernel/bounds.c:9:
include/linux/kernel.h:4:2: warning: #warning In kernel.h [-Wcpp]
#warning In kernel.h
^
In file included from include/linux/dynamic_debug.h:111:0,
from include/linux/printk.h:289,
from include/linux/kernel.h:14,
from include/asm-generic/bug.h:13,
from ./arch/arm64/include/asm/bug.h:62,
from include/linux/bug.h:4,
from include/linux/page-flags.h:9,
from kernel/bounds.c:9:
include/linux/string.h:4:2: warning: #warning In string.h [-Wcpp]
#warning In string.h
^

So, kernel.h gets pulled in first. string.h gets pulled in by kernel.h
(via dynamic_debug.h as you pointed out). The build still succeeds at
this point (because I didn't change string.h yet).

Now, if I reference SIZE_MAX in string.h, there's a problem: string.h
gets pulled in at the top of kernel.h, before kernel.h defines
SIZE_MAX. The compiler complains ("error: âSIZE_MAXâ undeclared").

If I add "#include <linux/kernel.h>" to the top of string.h in an
attempt to get SIZE_MAX sooner, nothing happens, because kernel.h has
already been included (and therefore the include guard is defined,
meaning that the new attempt to include it doesn't do anything). So, I
am leaving it out at this point.

To fix the SIZE_MAX issue, I quickly hacked up kernel.h, so SIZE_MAX
gets defined before string.h gets pulled in.

$ git diff
diff --git a/include/linux/kernel.h b/include/linux/kernel.h
index 94aa10f..6cd5269 100644
--- a/include/linux/kernel.h
+++ b/include/linux/kernel.h
@@ -10,8 +10,6 @@
#include <linux/bitops.h>
#include <linux/log2.h>
#include <linux/typecheck.h>
-#include <linux/printk.h>
-#include <linux/dynamic_debug.h>
#include <asm/byteorder.h>
#include <uapi/linux/kernel.h>

@@ -29,6 +27,9 @@
#define ULLONG_MAX (~0ULL)
#define SIZE_MAX (~(size_t)0)

+#include <linux/printk.h>
+#include <linux/dynamic_debug.h>
+
#define U8_MAX ((u8)~0U)
#define S8_MAX ((s8)(U8_MAX>>1))
#define S8_MIN ((s8)(-S8_MAX - 1))

This solves bounds.c. But it doesn't help with other files that don't
automatically pull in kernel.h before string.h. They still complain
about SIZE_MAX being undeclared. So, #include <linux/kernel.h> *does*
need to go into string.h. At this point, we run into the next problem.

$ aarch64-linux-gcc -Wp,-MD,scripts/mod/.devicetable-offsets.s.d
-nostdinc -isystem
/opt/toolchain/stbgcc-4.8-1.5/bin/../lib/gcc/aarch64-linux-gnu/4.8.5/include
-I./arch/arm64/include -Iarch/arm64/include/generated/uapi
-Iarch/arm64/include/generated -Iinclude -I./arch/arm64/include/uapi
-Iarch/arm64/include/generated/uapi -I./include/uapi
-Iinclude/generated/uapi -include ./include/linux/kconfig.h
-D__KERNEL__ -mlittle-endian -Wall -Wundef -Wstrict-prototypes
-Wno-trigraphs -fno-strict-aliasing -fno-common
-Werror-implicit-function-declaration -Wno-format-security -std=gnu89
-mgeneral-regs-only -fno-asynchronous-unwind-tables
-fno-delete-null-pointer-checks -O2 --param=allow-store-data-races=0
-Wframe-larger-than=2048 -fno-stack-protector
-Wno-unused-but-set-variable -fno-omit-frame-pointer
-fno-optimize-sibling-calls -fno-var-tracking-assignments -g
-Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow
-fconserve-stack -Werror=implicit-int -Werror=strict-prototypes
-DKBUILD_BASENAME='"devicetable_offsets"'
-DKBUILD_MODNAME='"devicetable_offsets"' -fverbose-asm -S -o
scripts/mod/devicetable-offsets.s scripts/mod/devicetable-offsets.c
In file included from include/uapi/linux/uuid.h:21:0,
from include/linux/uuid.h:19,
from include/linux/mod_devicetable.h:12,
from scripts/mod/devicetable-offsets.c:2:
include/linux/string.h:4:2: warning: #warning In string.h [-Wcpp]
#warning In string.h
^
In file included from include/linux/string.h:7:0,
from include/uapi/linux/uuid.h:21,
from include/linux/uuid.h:19,
from include/linux/mod_devicetable.h:12,
from scripts/mod/devicetable-offsets.c:2:
include/linux/kernel.h:4:2: warning: #warning In kernel.h [-Wcpp]
#warning In kernel.h
^
In file included from include/linux/printk.h:289:0,
from include/linux/kernel.h:31,
from include/linux/string.h:7,
from include/uapi/linux/uuid.h:21,
from include/linux/uuid.h:19,
from include/linux/mod_devicetable.h:12,
from scripts/mod/devicetable-offsets.c:2:
include/linux/dynamic_debug.h: In function âddebug_dyndbg_module_param_cbâ:
include/linux/dynamic_debug.h:122:2: error: implicit declaration of
function âstrstrâ [-Werror=implicit-function-declaration]
if (strstr(param, "dyndbg")) {
^
include/linux/dynamic_debug.h:122:6: warning: incompatible implicit
declaration of built-in function âstrstrâ [enabled by default]
if (strstr(param, "dyndbg")) {
^
cc1: some warnings being treated as errors

So, now we have the following sequence:

string.h -> kernel.h -> dynamic_debug.h -> oops, we don't know strstr() yet.

Yes, dynamic_debug.h does include string.h, but because of the include
guard, that second attempt at including string.h is stubbed out and
doesn't do anything.

To get past that issue, I can move "#include <linux/kernel.h>" all the
way down and only include it directly before my new functions. That
way strstr() is declared when dynamic_debug.h wants it.

diff --git a/include/linux/string.h b/include/linux/string.h
index 6cc85dc..eb2d9a8 100644
--- a/include/linux/string.h
+++ b/include/linux/string.h
@@ -171,6 +171,8 @@ static inline const char *kbasename(const char *path)
return tail ? tail + 1 : path;
}

+#include <linux/kernel.h>
+
/**
* strcpytoupper - Copy string and convert to uppercase.
* @dst: The buffer to store the result.

This is some scary stuff, but this way it seems to build. Not too sure
how to fix this properly and without risking some major headaches when
one least expects them.

Regards,
-Markus

> Thanks,
> Luis

Next message: Paul E. McKenney: "Re: Severe performance regression w/ 4.4+ on Android due to cgroup locking changes"
Previous message: Luis de Bethencourt: "Re: [RFC PATCH] dynamic_debug: only add header when used"
In reply to: Luis de Bethencourt: "Re: [PATCH v3 1/7] lib: string: add functions to case-convert strings"
Next in thread: Luis de Bethencourt: "Re: [PATCH v3 1/7] lib: string: add functions to case-convert strings"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]