Re: [PATCH v4 06/14] pylibfdt: Speed up FdtRo.properties() with a C helper
From: Doug Anderson
Date: Fri Nov 20 2026 - 12:50:45 EST
Hi Simon,
On Fri, Nov 20, 2026 at 7:22 AM Simon Glass <sjg@xxxxxxxxxxxx> wrote:
>
> > diff --git a/pylibfdt/libfdt.i b/pylibfdt/libfdt.i
> > @@ -24,6 +24,61 @@ static int fdt_property_stub(void *fdt, const char *name, const void *val,
> > + name_obj = PyUnicode_InternFromString(pname);
> > + if (!name_obj) {
> > + Py_DECREF(list);
> > + return NULL;
> > + }
>
> Just to check, how much of the 1.8x comes from interning? My
> understanding is that interned strings are immortal in Python 3.12 (I
> am not sure about later versions), so every distinct property name
> seen in any blob stays allocated for the life of the process. If the
> gain is small, PyUnicode_FromString() would be safer.
Interning isn't where the 1.8x comes from. Switching to
`PyUnicode_FromString()` keeps the same ~1.8x speedup in
`FdtRo.properties()`. Switched to `PyUnicode_FromString()` for v5.
> > diff --git a/pylibfdt/libfdt.i b/pylibfdt/libfdt.i
> > @@ -477,10 +532,9 @@ class FdtRo(object):
> > - poff = self.first_property_offset(nodeoffset, quiet=QUIET_NOTFOUND)
> > - while poff >= 0:
> > - yield self.get_property_by_offset(poff)
> > - poff = self.next_property_offset(poff, quiet=QUIET_NOTFOUND)
> > + return iter(check_err(
> > + fdt_node_properties_stub(self._fdt, nodeoffset, Property)
> > + ))
>
> This changes the behaviour a little, since properties() is no longer a
> generator. The FdtException is now raised by the call itself rather
> than on the first next(), and an error part-way through a node (e.g. a
> truncated blob) now produces no properties at all, where before the
> caller saw the earlier ones first.
>
> I think that is OK, but please can you mention it in the commit
> message and update the docstring, which still says 'Yields'.
Done. Updated the docstring (`Yields:` -> `Returns:` plus a note on
eager evaluation) and noted the behavior change in the commit message.
> > diff --git a/tests/pylibfdt_tests.py b/tests/pylibfdt_tests.py
> > @@ -270,6 +270,9 @@ class PyLibfdtBasicTests(unittest.TestCase):
> > + with self.assertRaises(FdtException) as e:
> > + list(self.fdt.properties(-1))
> > + self.assertEqual(e.exception.err, -libfdt.BADOFFSET)
>
> There is already an identical check a little further up in this file
> (just before testGetName()) and, since it wraps the call in list(), it
> passes with both the old and new code.
>
> It would be worth replacing it with a test for the new error path,
> i.e. a failure after the first property, such as a blob truncated in
> the middle of a node's properties.
Done. Replaced the duplicate check in `testPropertiesIterator` with a
test that truncates `size_dt_struct` after `/subnode@1`'s first property
and verifies `fdt_trunc.properties(node)` raises `-libfdt.TRUNCATED`
directly on the call.
-Doug