Re: [PATCH 00/10] Documentation/Sphinx

[Jani Nikula]

On Mon, 30 May 2016, Markus Heiser <markus.heiser@xxxxxxxxxxx> wrote:
> Am 30.05.2016 um 16:46 schrieb Jani Nikula <jani.nikula@xxxxxxxxx>:
>> I am not proposing to merge the documents that I've converted mostly as
>> samples in the branch. I needed something to demonstrate the build is
>> sane.
>
>> The authors of the DocBook documents should make the conversions as they
>> see fit, when they see fit, with the tools they see fit, probably with
>> some manual work on top.
>
> OK

To be clear, the "sphinx-for-docs-next" branch of [1], [2] is what I
propose to merge at this time. There's the Sphinx configuration, kernel
build integration, Sphinx kernel-doc extension, tons of kernel-doc
updates, etc. There is no DocBook tmpl conversion; all of that is left
to the authors (owners, maintainers) of the documents, but this enables
them to focus on that part.

I was planning on sending out the patches after some feedback here.

[1] git://people.freedesktop.org/~jani/drm
[2] https://cgit.freedesktop.org/~jani/drm/log/?h=sphinx-for-docs-next

> With DocBook, it was hard to separate a file into small chunks (see media for
> how it is done). With Sphinx, it is common to split a document in small chunks 
> (along parts, chapters, sections ...) Thats why I recommend chunking documents
> (from the beginning).

Agreed, but up to the authors.

>> One of the goals was to have nice cross-referencing between the
>> documents (e.g. from GPU to kernel or device driver API). And it works.
>
> For this, Sphinx-doc brings intersphinx: http://www.sphinx-doc.org/en/stable/ext/intersphinx.html

If the kernel is split to several intersphinx "prefixes", we won't know
the prefix of the link targets when we're generating the references in
kernel-doc. Also, can be deferred to follow-up work if someone figures
out the how.

> I can't recommend to use rst2pdf (it is less maintained), use default
> sphinx LaTeX toolchain.

I think we'll use whatever works, rst2pdf seemed to work for now, but we
can change if needed.

>> I find it totally unacceptable to require explicitly marking kernel-doc
>> comments or source files as being reStructuredText.
>> Note that it's all opt-in already. If you add a .rst file that includes
>> kernel-doc via the kernel-doc extension, you better make sure the
>> comments parse as reStructuredText and render nicely. I'm willing to do
>> much of the job for all the things that I care about.
>
> We have a different POV ... I try to build up a documentation project,
> which could use all given kernel-doc markups without any change, where
> reST is an "addition". Your approach is to fix kernel-doc comments 
> if they are referred by a kernl-doc directive in a .rst document.
> There is nothing wrong about your approach, but I try to build
> a whole source code documentation like the one I started here: 
> http://return42.github.io/sphkerneldoc/linux_src_doc/index.html

That looks nice, but I'll argue it would not be much worse even if you
assumed it's all rst.

The bigger point is, if you expect people to tag each source file or
kernel-doc comment with "rst", you'll end up with a mess where some
places have that tag, some not, but it's not conclusive about whether
they actually *are* rst or not. (And you've had tons of patch churn to
add those tags to get there.)

The kernel-doc comments are written by humans who will screw it up
anyway. (Apologies for the distrust, fellow developers, but I've been
reading too many of your fine kernel-doc comments lately.) People will
happily cargo cult rst and current kernel-doc and javadoc and doxygen
and whatnot in a fruit salad. The only thing that will help in the end
is keeping the rules simple and consistent and having the feedback from
the tools.

> I worry a little bit in that reST will be only one more toolchain 
> beside DocBook .. in the long term, kernel's documentation 
> should get rid of all the DocBook artifacts and for this a more
> comprehensive solution is needed.

We agree on the end goal, eradicate DocBook. I must say that in my
experiments, apart from the media docs, almost everything converts
surprisingly nicely or IMO "good enough" with just the tmplcvt script in
this series. Do remember that this is a one time conversion. It needs to
be good enough that there's not too much manual editing involved, but it
doesn't need to be perfect. Some degree of editing will be required no
matter what, not least because the DocBook has also been written by
humans, and the battle against the GIGO principle is a lost one.

>> And with Jani's big pile of kernel-doc
>> patches we now also have someone who understands that perl script,
>> which is awesome.
>
> ? 

I must also question any sentence that implies I understand perl. ;)

>> In short I want to go nuts improving the docs themselves and stop
>> discussing the tooling to build them. Can we please make this happen?
>> 
>> Note that Jani's already started to throw out our old ascidoc hacks in
>> the topic/kerneldoc branch in the drm-intel.git repo, and we'll switch
>> over the autobuilder for the 01.org docs as soon as that's done. We're
>> committed, I want this ;-)

It's done, we're feeding this to our integration tree and dogfooding.


BR,
Jani.


-- 
Jani Nikula, Intel Open Source Technology Center