RE: [PATCH 07/10] docs: kdoc: better handle source when producing YAML output

[Loktionov, Aleksandr]

> -----Original Message-----
> From: Mauro Carvalho Chehab <mchehab+huawei@xxxxxxxxxx>
> Sent: Monday, March 23, 2026 10:11 AM
> To: Jonathan Corbet <corbet@xxxxxxx>; Linux Doc Mailing List <linux-
> doc@xxxxxxxxxxxxxxx>; Mauro Carvalho Chehab <mchehab@xxxxxxxxxx>
> Cc: Mauro Carvalho Chehab <mchehab+huawei@xxxxxxxxxx>; linux-
> kernel@xxxxxxxxxxxxxxx; Loktionov, Aleksandr
> <aleksandr.loktionov@xxxxxxxxx>; Randy Dunlap <rdunlap@xxxxxxxxxxxxx>
> Subject: [PATCH 07/10] docs: kdoc: better handle source when producing
> YAML output
> 
> The current logic was storing symbols source code on a list, not
> linked to the actual KdocItem. While this works fine when kernel-doc
> markups are OK, on places where there is a "/**"
> without a valid kernel-doc markup, it ends that the 1:1 match between
> source code and KdocItem doesn't happen, causing problems to generate
> the YAML output.
> 
> Fix it by storing the source code directly into the KdocItem
> structure.
> 
> This shouldn't affect performance or memory footprint, except when --
> yaml option is used.
> 
> While here, add a __repr__() function for KdocItem, as it helps
> debugging it.
> 

Not sure, do we need Fixes: tag, what do you think?

> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@xxxxxxxxxx>
> ---
>  tools/lib/python/kdoc/kdoc_files.py     |   8 +-
>  tools/lib/python/kdoc/kdoc_item.py      |   6 +-
>  tools/lib/python/kdoc/kdoc_parser.py    | 100 ++++++++++++-----------
> -
>  tools/lib/python/kdoc/kdoc_yaml_file.py |  28 +++----
>  tools/unittests/test_kdoc_parser.py     |   9 +++
>  5 files changed, 79 insertions(+), 72 deletions(-)
> 
> diff --git a/tools/lib/python/kdoc/kdoc_files.py
> b/tools/lib/python/kdoc/kdoc_files.py
> index 5a299ed44d62..2428cfc4e843 100644
> --- a/tools/lib/python/kdoc/kdoc_files.py
> +++ b/tools/lib/python/kdoc/kdoc_files.py
> @@ -203,10 +203,6 @@ class KernelFiles():
> 
>          self.results[fname] = entries
> 

...

>              self.reset_state(ln)
> 
>          elif doc_content.search(line):
> @@ -1596,15 +1606,6 @@ class KernelDoc:
>          state.DOCBLOCK:			process_docblock,
>          }
> 
> -    def get_source(self):
> -        """
> -        Return the file content of the lines handled by kernel-doc at
> the
> -        latest parse_kdoc() run.
> -
> -        Returns none if KernelDoc() was not initialized with
> store_src,
> -        """
> -        return self.source
> -
>      def parse_kdoc(self):
>          """
>          Open and process each line of a C source file.
> @@ -1618,8 +1619,8 @@ class KernelDoc:
>          prev = ""
>          prev_ln = None
>          export_table = set()
> -        self.source = []
>          self.state = state.NORMAL
> +        source = ""
> 
>          try:
>              with open(self.fname, "r", encoding="utf8", @@ -1646,7
> +1647,11 @@ class KernelDoc:
>                                            ln, state.name[self.state],
>                                            line)
> 
> -                    prev_state = self.state
> +                    if self.store_src:
> +                        if source and self.state == state.NORMAL:
> +                            source = ""
> +                        elif self.state != state.NORMAL:
> +                            source += line + "\n"
> 
>                      # This is an optimization over the original
> script.
>                      # There, when export_file was used for the same
> file, @@ -1655,16 +1660,11 @@ class KernelDoc:
>                      #
>                      if (self.state != state.NORMAL) or \
>                         not self.process_export(export_table, line):
> +                        prev_state = self.state
>                          # Hand this line to the appropriate state
> handler
> -                        self.state_actions[self.state](self, ln,
> line)
> -
> -                    if self.store_src and prev_state != self.state or
> self.state != state.NORMAL:
> -                        if self.state == state.NAME:
> -                            # A "/**" was detected. Add a new source
> element
> -                            self.source.append({"ln": ln, "data":
> line + "\n"})
> -                        else:
> -                            # Append to the existing one
> -                            self.source[-1]["data"] += line + "\n"
> +                        self.state_actions[self.state](self, ln,
> line, source)
> +                        if prev_state == state.NORMAL and self.state
> != state.NORMAL:

It looks this block is not guarded by `if self.store_src`,
and even when store_src=False (i.e., --yaml was NOT passed)

> +                            source += line + "\n"
It populates `source` unconditionally regardless of --yaml.
Isn't it?


> 
>              self.emit_unused_warnings()
> 
> diff --git a/tools/lib/python/kdoc/kdoc_yaml_file.py
> b/tools/lib/python/kdoc/kdoc_yaml_file.py
> index 18737abb1176..1e2ae7c59d70 100644
> --- a/tools/lib/python/kdoc/kdoc_yaml_file.py
> +++ b/tools/lib/python/kdoc/kdoc_yaml_file.py
> @@ -85,7 +85,7 @@ class KDocTestFile():
> 
>          return d
> 

...

>                      result = clean_whitespc(d[key], relax_whitespace)
>                      value = clean_whitespc(value, relax_whitespace)
> 
> --
> 2.53.0