Re: [RFC PATCH v2 10/74] tests: Add basic metadata tests

[Herve Codina]

On Sat, 31 Oct 2026 14:17:47 +1100
David Gibson <david@xxxxxxxxxxxxxxxxxxxxx> wrote:

> On Fri, Oct 30, 2026 at 12:01:17PM +0100, Herve Codina wrote:
> > On Fri, 30 Oct 2026 16:24:52 +1100
> > David Gibson <david@xxxxxxxxxxxxxxxxxxxxx> wrote:
> >   
> > > On Wed, Aug 26, 2026 at 11:48:39AM +0200, Herve Codina wrote:  
> > > > This first test is related to local phandle references used in
> > > > properties (FDT_PROPDATA_PHANDLE dtb tag).
> > > > 
> > > > The test pattern used is
> > > >   - Generate a dts (xxx.dts.dts) from an input dts
> > > >   - Check this generated dts against expected contents
> > > >   - Generate a dtb (xxx.dtb) from the same input dts
> > > >   - Check this generated dtb against expected contents
> > > >   - Generate a dts (xxx.dtb.dts) from the generated dtb
> > > >   - Check this generated dts against expected contents
> > > >   - Generate a dtb (xxx.dtb.dts.dtb) from the generated dts
> > > >   - Check this generated dtb, expect the same contents as for xxx.dtb
> > > > 
> > > > Even if only one meta-data feature is currently tested in this tests
> > > > introduction, use a loop in order to ease future addition consisting in
> > > > adding new input dts as soon as new meta-data feature are supported.
> > > > 
> > > > Signed-off-by: Herve Codina <herve.codina@xxxxxxxxxxx>    
> > > 
> > > As noted elsewhere, I'd prefer to see new helper tools for comparing
> > > metadata in binary rather than using text comparisons.  
> > 
> > Right, be ready to see treegen.c growing a lot.
> > 
> > Indeed, to compare in binary, I need to generate the expected binary (I don't
> > want to store it directly in git).  
> 
> Sure.
> 
> > I already plan to split the treegen.c file in several files. Indeed, with
> > unknown "structured" tags, a bunch of expected dtbs (instead to text files)
> > are going to be generated.
> > 
> > The plan is then to have one .c files per expected dtbs set and so the
> > concept of dtbs harnesses.
> > 
> > The problem I see here with known meta-data, is that we:
> >   1) Need to keep consistency between the dts file (input source use in test)
> >      and the treegen source file (which generate the expected dtb).
> > 
> >   2) If new meta-data tags are added in the future, treegen source files
> >      will need an update. This update will be more complex than looking at
> >      expected text files (fdtdump or other), and commiting the new expected
> >      files if the diff is explained.  
> 
> Hm, only when/if the new tags are required / automatic.
> 
> > I've proposed 'fdtanalyze' as a new tool [1] to convert a dtb to a "friendly"
> > text file. The output of 'fdtanalyze' could be something like that:
> > --- 8< ---
> >   (0x1-FDT_BEGIN_NODE): "my_node"
> >     (0x3-FDT_PROP): ...
> >     (0x3-FDT_PROP): ...
> >   (0x2-FDT_END_NODE)
> >   ...
> >   (0x9-FDT_END)
> > --- 8< ---
> > 
> > No complex analysis but you don't seem to like it neither.  
> 
> It's better than relying on fdtdump, but at this point it's just
> making a text format that's isomorphic to the binary and I don't see a
> lot of point.

During test implementation:
- Create an empty expected text file
- Run the test. The comparison an test generated text against expected test fails
- Look at the diff. If it is ok, copy the generated text file to the expected test
  file.

Maintenance in the future:
- You've done some modification and the test fails
- Look at the text file diff. If the failure is a false failure due to modification,
  just copy the generated text file to the expected test file.

With treegen dtb, the process is quite similar but not so obvious. You need
to modified the source file that generated the expected dtb. You cannot update
the dtb directly. Indeed, you can adjust the source which generates the dtb, not
the dtb itself.

Depending on the complexity of the source file to generate the dtb, updating this
source file is more or less easy.

In the next iteration of the structured tag series, a treegen source will be
available to generate dtbs related to test modification when unknown tags are
present.

With this treegen source, we will see if using generated dtbs for expected dtbs
is ok or not. You will judge.

Best regards,
Hervé