Re: [PATCH v2] docs: automarkup.py: Allow automatic cross-reference inside C namespace

[Nícolas F. R. A. Prado]

On Wed Nov 18, 2020 at 5:37 PM -03, Jonathan Corbet wrote:
>
> On Tue, 17 Nov 2020 02:12:01 +0000
> Nícolas F. R. A. Prado <nfraprado@xxxxxxxxxxxxxx> wrote:
>
> > Sphinx 3.1 introduced namespaces for C cross-references. With this,
> > each C domain type/function declaration is put inside the namespace that
> > was active at the time of its declaration.
> >
> > Add support for automatic cross-referencing inside C namespaces by
> > checking whether the corresponding source file had a C namespace Sphinx
> > directive, and if so, try cross-referencing inside of it before going to
> > the global scope.
> >
> > This assumes there's only one namespace (if any) per rst file.
> >
> > Signed-off-by: Nícolas F. R. A. Prado <nfraprado@xxxxxxxxxxxxxx>
> > ---
> >
> > To those following from v1:
> >
> > I ended up doing the simplest solution possible, which is to just directly read
> > the rst source corresponding to the doc page right before doing the automarkup.
> > It's not very efficient in the sense that the source is being read
> > twice (first by Sphinx, then by this), but it sidesteps the "data sharing
> > between processes" issue, so parallel_read_safe can be reenabled, and I didn't
> > notice any performance hit from this patch (as opposed to the big hit from v1).
> > Works with both Sphinx 2 and 3.
>
> The solution does lack elegance, but it is a solution, which is more than
> we had before :)

Exactly :P

> That said, rather than re-opening and re-reading the
> file, why not just connect to the source-read event, which will happily
> hand you the document source that it has already read?

Unfortunately that wouldn't work. What would happen is that Sphinx would spawn a
python process to handle the source-read event for file X, then later it would
spawn another python process to handle doctree-resolved for the same file X.
Being two different processes, data can't easily be shared between them, which
is why I originally disabled parallel_read_safe, to force everything into a
single process, enabling data to be stored in the source-read phase to be later
used at the doctree-resolved phase.

What we need is a single process that both reads the source and uses that info
to make the auto markup. With parallel_read_safe enabled, that is only possible
by doing everything in a single Sphinx event (namely, doctree-resolved), and
therefore the read needs to be done manually outside of Sphinx.

Thanks,
Nícolas

>
> Thanks,
>
> jon