Re: [PATCH RFC] kbuild: create a list of all built DTB files

[Chen-Yu Tsai]

On Sun, Feb 25, 2024 at 4:21 PM Masahiro Yamada <masahiroy@xxxxxxxxxx> wrote:
>
> On Fri, Feb 23, 2024 at 6:23 PM Chen-Yu Tsai <wenst@xxxxxxxxxxxx> wrote:
> >
> > It is useful to have a list of all composite *.dtb files, along with
> > their individual components, generated from the current build.
> >
> > With this commit, 'make dtbs' creates arch/*/boot/dts/dtbs-components,
> > which lists the composite dtb files created in the current build. It
> > maintains the order of the dtb-y additions in Makefiles although the
> > order is not important for DTBs.
> >
> > This compliments the list of all *.dtb and *.dtbo files in dtbs-list,
> > which only includes the files directly added to dtb-y.
> >
> > For example, consider this case:
> >
> >     foo-dtbs := foo_base.dtb foo_overlay.dtbo
> >     dtb-y := bar.dtb foo.dtb
> >
> > In this example, the new list will include foo.dtb with foo_base.dtb and
> > foo_overlay.dtbo on the same line, but not bar.dtb.
> >
> > Signed-off-by: Chen-Yu Tsai <wenst@xxxxxxxxxxxx>
> > ---
> > Hi,
> >
> > I hacked up this new thing to list out the individual components of each
> > composite dtb. I think this information would be useful for FIT image
> > generation or other toolchains to consume. For example, instead of
> > including each dtb, a toolchain could realize that some are put together
> > using others, and if the bootloader supports it, put together commands
> > to reassemble the end result from the original parts.
> >
> > This is based on and complements Masahiro-san's recent dtbs-list work.
>
>
>
> This is another format of my previous per-dtb "*.dtlst"
> (but I did not pick up 3/4, 4/4 because I did not know what we need after all).
>
> This should be discussed together with how Simon's script will look like.
>
> I can understand your Makefile code, but I still do not know
> how the entire overlay stuff will work in a big picture.

How would you like to proceed? I can through together some changes on top
of Simon's patches as an initial proposal if that helps?

I can use your format if you prefer.


ChenYu

> >
> >  .gitignore             |  1 +
> >  scripts/Makefile.build | 16 ++++++++++++++++
> >  scripts/Makefile.lib   |  8 ++++++--
> >  3 files changed, 23 insertions(+), 2 deletions(-)
> >
> > diff --git a/.gitignore b/.gitignore
> > index c59dc60ba62e..bb5b3bbca4ef 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -52,6 +52,7 @@
> >  *.xz
> >  *.zst
> >  Module.symvers
> > +dtbs-components
> >  dtbs-list
> >  modules.order
> >
> > diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> > index 4971f54c855e..ba85c2385c9e 100644
> > --- a/scripts/Makefile.build
> > +++ b/scripts/Makefile.build
> > @@ -72,6 +72,7 @@ endif
> >  subdir-builtin := $(sort $(filter %/built-in.a, $(real-obj-y)))
> >  subdir-modorder := $(sort $(filter %/modules.order, $(obj-m)))
> >  subdir-dtbslist := $(sort $(filter %/dtbs-list, $(dtb-y)))
> > +subdir-dtbscomp := $(sort $(filter %/dtbs-components, $(multi-dtb-y)))
> >
> >  targets-for-builtin := $(extra-y)
> >
> > @@ -390,6 +391,7 @@ $(obj)/%.asn1.c $(obj)/%.asn1.h: $(src)/%.asn1 $(objtree)/scripts/asn1_compiler
> >  $(subdir-builtin): $(obj)/%/built-in.a: $(obj)/% ;
> >  $(subdir-modorder): $(obj)/%/modules.order: $(obj)/% ;
> >  $(subdir-dtbslist): $(obj)/%/dtbs-list: $(obj)/% ;
> > +$(subdir-dtbscomp): $(obj)/%/dtbs-components: $(obj)/% ;
> >
> >  #
> >  # Rule to compile a set of .o files into one .a file (without symbol table)
> > @@ -422,6 +424,20 @@ $(obj)/modules.order: $(obj-m) FORCE
> >  $(obj)/dtbs-list: $(dtb-y) FORCE
> >         $(call if_changed,gen_order)
> >
> > +#
> > +# Rule to create dtbs-components
> > +#
> > +# This is a list of composite dtb(s), along with each dtb's components,
> > +# from the current Makefile and its sub-directories.
> > +
> > +cmd_gen_dtb_components = { $(foreach m, $(real-prereqs), \
> > +               $(if $(filter %/$(notdir $@), $m), cat $m, \
> > +                       echo $m: $(addprefix $(obj)/,$($(notdir $(m:%.dtb=%-dtbs))))); \
> > +       ) :; } > $@
> > +
> > +$(obj)/dtbs-components: $(multi-dtb-y) FORCE
> > +       $(call if_changed,gen_dtb_components)
> > +
> >  #
> >  # Rule to compile a set of .o files into one .a file (with symbol table)
> >  #
> > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > index dbcac396329e..7c2127a84ac2 100644
> > --- a/scripts/Makefile.lib
> > +++ b/scripts/Makefile.lib
> > @@ -61,7 +61,6 @@ real-search = $(foreach m, $1, $(if $(call suffix-search, $m, $2, $3 -), $(call
> >  multi-obj-y := $(call multi-search, $(obj-y), .o, -objs -y)
> >  multi-obj-m := $(call multi-search, $(obj-m), .o, -objs -y -m)
> >  multi-obj-ym := $(multi-obj-y) $(multi-obj-m)
> > -
> >  # Replace multi-part objects by their individual parts,
> >  # including built-in.a from subdirectories
> >  real-obj-y := $(call real-search, $(obj-y), .o, -objs -y)
> > @@ -91,6 +90,11 @@ real-dtb-y := $(call real-search, $(dtb-y), .dtb, -dtbs)
> >  # Base DTB that overlay is applied onto
> >  base-dtb-y := $(filter %.dtb, $(call real-search, $(multi-dtb-y), .dtb, -dtbs))
> >
> > +ifdef need-dtbslist
> > +multi-dtb-y    += $(addsuffix /dtbs-components, $(subdir-ym))
> > +always-y       += dtbs-components
> > +endif
> > +
> >  always-y                       += $(dtb-y)
> >
> >  # Add subdir path
> > @@ -406,7 +410,7 @@ cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ;
> >  quiet_cmd_fdtoverlay = DTOVL   $@
> >        cmd_fdtoverlay = $(objtree)/scripts/dtc/fdtoverlay -o $@ -i $(real-prereqs)
> >
> > -$(multi-dtb-y): FORCE
> > +$(filter-out %/dtbs-components, multi-dtb-y): FORCE
> >         $(call if_changed,fdtoverlay)
> >  $(call multi_depend, $(multi-dtb-y), .dtb, -dtbs)
> >
> > --
> > 2.44.0.rc0.258.g7320e95886-goog
> >
>
>
> --
> Best Regards
> Masahiro Yamada