Re: [PATCH 2/4] fdtfactor: Add tool to split DTBs into a base DTB and overlays

[Simon Glass]

Hi Doug,

On 2026-10-15T18:21:33, Douglas Anderson <dianders@xxxxxxxxxxxx> wrote:
> fdtfactor: Add tool to split DTBs into a base DTB and overlays
>
> When shipping a single firmware or kernel image that supports many
> closely related board variants or SKUs, bundling a full compiled .dtb
> for every variant results in substantial duplication.
>
> Add `fdtfactor`, a Python tool built on `pylibfdt` that takes a set of
> compiled device tree blobs (.dtb) compiled with `-@ -L` (`/__symbols__`
> and `/__local_fixups__`) and splits them into a single common
> `base.dtb` and per-board `.dtbo` overlays:
>
> - Common nodes present across all input DTBs are placed in `base.dtb`,
>   with each common property assigned the value shared by at least two
>   boards that maximizes byte savings across overlays.
> - Using `/__local_fixups__`, phandle references inside property values
>   are normalized to target node paths so common properties referencing
>   common nodes match even when numeric phandles differ across input
>   DTBs.
> - Any common node that carries a phandle in any input DTB receives its
>   phandle in `base.dtb` so overlays only emit `phandle` properties on
> [...]
>
>  Makefile                   |    2 +-
>  README.md                  |   58 +++
>  fdtfactor                  | 1218 ++++++++++++++++++++++++++++++++++++++++++++
>  meson.build                |    1 +
>  tests/Makefile.tests       |    2 +-
>  tests/fdtfactor_board1.dts |   69 +++
>  tests/fdtfactor_board2.dts |   73 +++
>  tests/fdtfactor_board3.dts |   96 ++++
>  tests/run_tests.sh         |   61 ++-
>  tests/testutils.sh         |    1 +
>  10 files changed, 1578 insertions(+), 3 deletions(-)

> diff --git a/fdtfactor b/fdtfactor
> @@ -11,31 +11,195 @@ original device tree.
> +    if comp == 'xz':
> +        return lzma.compress(raw_bytes, format=lzma.FORMAT_XZ)

Python's default integrity check for FORMAT_XZ is CRC64, but embedded
xz decoders often support only CRC32. The kernel's lib/xz is one of
them, which is why scripts/xz_wrap.sh passes --check=crc32 to xz. So a
firmware-friendly .dtb.xz input could produce a .dtbo.xz that the same
firmware cannot read. Please can you pass check=lzma.CHECK_CRC32 here?

> diff --git a/fdtfactor b/fdtfactor
> @@ -1167,20 +1349,24 @@ def factor_dtb_files(args):
>                  d, base_dtb_blob, dtbo_blob, args.strip_local_fixups
>              )
>
> +    base_dtb_file_bytes = compress_blob(base_dtb_blob, base_dtb_comp)

If you keep --verify in the tool (David suggests it belongs in the
test suite), it would be worth running the compressed bytes back
through decompress_blob() and comparing them, since at present only
the uncompressed blobs are checked. That is cheap and would catch a
problem with the lz4 command or the compression options.

> diff --git a/tests/run_tests.sh b/tests/run_tests.sh
> @@ -1147,6 +1147,36 @@ fdtoverlay_tests() {
> +	    run_wrap_test $PYTHON "$FDTFACTOR" -v --verify \
> +		-b "fdtfactor_base.test.dtb.$ext" \

In these tests the base output always uses the same compression as the
inputs, and the input extension always matches the file contents. Two
extra cases would be good, perhaps as a follow-up: a compressed blob
in a plain '.dtb' file (to exercise the magic-byte detection) and, if
-b stays, an uncompressed -b alongside compressed inputs.

Regards,
Simon