Re: [RFC] doc: fix code snippet build warnings

[Markus Heiser]

> Am 16.01.2018 um 11:22 schrieb Jani Nikula <jani.nikula@xxxxxxxxxxxxxxx>:
> 
> On Wed, 10 Jan 2018, Jonathan Corbet <corbet@xxxxxxx> wrote:
>> On Wed, 10 Jan 2018 15:04:53 +1100
>> "Tobin C. Harding" <me@xxxxxxxx> wrote:
>> 
>>> Posting as RFC in the hope that someone knows how to massage sphinx
>>> correctly to fix this patch.
>>> 
>>> Currently function kernel-doc contains a multi-line code snippet. This
>>> is causing sphinx to emit 5 build warnings
>>> 
>>> 	WARNING: Unexpected indentation.
>>> 	WARNING: Unexpected indentation.
>>> 	WARNING: Block quote ends without a blank line; unexpected unindent.
>>> 	WARNING: Block quote ends without a blank line; unexpected unindent.
>>> 	WARNING: Inline literal start-string without end-string.
>>> 
>>> And the snippet is not rendering correctly in HTML.
>>> 
>>> We can stop shpinx complaining by using '::' instead of the currently
>>> used '``' however this still does not render correctly in HTML. The
>>> rendering is [arguably] better but still incorrect. Sphinx renders two
>>> function calls thus:
>>> 
>>> 	:c:func:`rcu_read_lock()`;
>>> 
>>> The rest of the snippet does however have correct spacing.
>> 
>> The behavior when `` was used is not surprising, that really just does a
>> font change.  Once you went with a literal block (with "::") though, the
>> situation changes a bit.  That really should work.
>> 
>> I looked a bit.  This isn't a sphinx (or "shpinx" :) problem, the bug is
>> in kernel-doc.  Once we go into the literal mode, it shouldn't be
>> screwing around with the text anymore.  Of course, kernel-doc doesn't
>> understand enough RST to know that.  I'm a little nervous about trying to
>> teach it more, but maybe we have to do that; we should certainly be able
>> to put code snippets into the docs and have them come through unmolested.
> 
> I think eventually the Right Thing would be to move most of kernel-doc's
> mucking of the comments (i.e. highlights) into kernel-doc the Sphinx
> extension. I've toyed with the idea, but it's not as trivial as I would
> like it to be.
> 
> Basically it involves flagging all the kernel-doc nodes during the read
> phase, and registering handlers to post process the doctrees. At that
> stage, it's no longer simple regexp replaces, it's replacing doctree
> nodes with ones that have reference nodes within them. It's more
> complicated,

Hi Jani, my thoughts about:  

1.) The reST syntax [1] (the parser part) is not *extendable*, we
can only add new roles [2] or directives.

2.) the kernel-doc syntax is weak and ambiguous. This
remains mainly in tagging only with a start-tag or only with a end-tag 
e.g:

* sectioning:   "Return:" --> end-tag just ":"
* fields:       "@arg1:"  --> better
* highlight/ref: start tag [@%$&] / no end tag

even if I don't know how (1.), if the highlight-syntax becomes a part
of the reST syntax we have to quote [@%$&] elsewhere. This will break
with valid reST documents which is unacceptable.

Thats why I think we need to pre-parse highlighting in the kernel-doc
Perl script, even if it is a bit hackish (how should it be otherwise,
if the syntax is already hackish). I haven't had the time to implement
such a highlight parser right now, but this would be the first shot of
mine. 

An alternative might be to use roles [2] but this means we have to
break with the the kernel-doc (highlight) syntax which is IMO also
unacceptable ... tricky situation :o

[1] http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html
[2] http://docutils.sourceforge.net/docs/howto/rst-roles.html


> but at that stage we can ignore stuff that should stay
> verbatim. I think it's also possible to check that the reference targets
> actually exist. In short, at that phase we have all the knowledge about
> the rst structure, parsed by Sphinx, and we don't have to duplicate that
> knowledge in kernel-doc the perl script.
> 
> Note that this has nothing to do with swithing to Python based
> kernel-doc suggested by Markus previously.

;)

-- Markus --