Re: [PATCH v5 5/8] kbuild: add fine grained build dependencies for exported symbols

[Nicolas Pitre]

On Sat, 5 Mar 2016, Michal Marek wrote:

> On Fri, Mar 04, 2016 at 11:53:53PM +0100, Michal Marek wrote:
> > Dne 4.3.2016 v 23:51 Michal Marek napsal(a):
> > > Dne 4.3.2016 v 06:40 Nicolas Pitre napsal(a):
> > >> +cmd_and_fixdep =                                                             \
> > >> +	$(echo-cmd) $(cmd_$(1));                                             \
> > >> +	$(ksym_dep_filter) |                                                 \
> > >> +		scripts/basic/fixdep -e $(depfile) $@ '$(make-cmd)'          \
> > >> +			> $(dot-target).tmp;	                             \
> > >> +	rm -f $(depfile);                                                    \
> > >> +	mv -f $(dot-target).tmp $(dot-target).cmd;
> > > 
> > > While trying this, I got a SIGBUS from fixdep once. My theory is that
> > > the depfile is mmap()ed by fixdep and modified by the preprocesor run at
> > > the same time. I could not reproduce this so far (still trying). But if
> > > it's really this race, the fix would be to disable dependency generation
> > > in the preprocessor by passing -Wp,MD,/dev/null or somesuch. But we
> > > never had this problem with genksyms, which is weird. It could as well
> > > be that my build machine's memory is faulty :(.
> > 
> > Actually, genksyms does not ran in parallel. neither before nor after
>                               run
> > this patch.
> 
> I reproduced the SIGBUS after a few iterations, and it crashes in
> parse_dep_file(). I'm now testing this
> 
> diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
> index 0be7e09ba381..4fdd8348acf6 100644
> --- a/scripts/Kbuild.include
> +++ b/scripts/Kbuild.include
> @@ -270,10 +270,12 @@ else
>  
>  # Filter out exported kernel symbol names from the preprocessor output.
>  # See also __KSYM_DEPS__ in include/linux/export.h.
> +# We disable the depfile generation here, so as not to overwrite the existing
> +# depfile while fixdep is parsing it
>  ksym_dep_filter =                                                            \
>  	case "$(1)" in                                                       \
> -	cc_*_c) $(CPP) $(c_flags) -D__KSYM_DEPS__ $< ;;                      \
> -	as_*_S) $(CPP) $(a_flags) -D__KSYM_DEPS__ $< ;;                      \
> +	cc_*_c) $(CPP) $(filter-out -Wp$(comma)-M%, $(c_flags)) -D__KSYM_DEPS__ $< ;; \
> +	as_*_S) $(CPP) $(filter-out -Wp$(comma)-M%, $(a_flags)) -D__KSYM_DEPS__ $< ;; \
>  	cpp_lds_S) : ;;                                                      \
>  	*) echo "Don't know how to preprocess $(1)"; false ;;                \
>  	esac | sed -rn 's/^.*=== __KSYM_(.*) ===.*$$/KSYM_\1/p'

This makes perfect sense even if I can't reproduce on my side.

Are you willing to fold this patch in, or do you prefer that I repost?


Nicolas