[PATCH 00/10] Documentation/Sphinx

[Jani Nikula]

Hi Jon, all -

I've had a few moments of spare time to look into Sphinx. This is a sort
of status report on the progress.

I suppose we all thought it would be easiest to use docproc for the
Sphinx toolchain, for starters. I tried it, hard, and even sent a bunch
of docproc prep patches. However I ran into some problems. Out-of-tree
builds were a nightmare, as Sphinx has trouble to pick up some of the
files from $(srctree) and some from $(objtree). Having dependencies on
the source files work without rebuilding everything was getting tricky
too.

I decided to look into writing a Sphinx extension for kernel-doc, and it
turns out to be a really nice solution. We can keep all the .rst files
in $(srctree), we don't have to explicitly specify the .rst files to
process, there are no intermediate files, and Sphinx runs the kernel-doc
script using the extension based on the directives in the .rst
files. The extension tells Sphinx about the dependencies on the source
files, and Sphinx handles rebuilding as needed. Out-of-tree builds just
work. Sites like https://readthedocs.org/ can build the documentation,
including kernel-doc, without extra tweaks. As a whole, the build
becomes much simpler.

There are a few tradeoffs, of course. First, this requires that the
EXPORT_SYMBOL markers are placed immediately after the function being
exported, as kernel-doc will only look at one file at a time. This is
the recommendation anyway. See the corresponding patch for further
details ("kernel-doc: support printing exported and non-exported
symbols"). Second, we lose support for the !C docproc directive to check
that all kernel-doc comments in a file are used. This is probably
something we'd like to have back in the future, but at this time I think
it's an acceptable tradeoff wrt the gains.

With this, we can put any .rst files (including ones that have
kernel-doc directives) anywhere under Documentation, add a link to them
in Documentation/index.rst table of contents, and it will just work. It
can't get much simpler than that.

At this time I've put most effort into the configuration and build side
of things, solving the problems described above, and handling missing
tools and packages gracefully. There are still issues to be ironed out
in a) the kernel-doc script rst output and b) the xml template to rst
conversion. These are somewhat orthogonal from each other and the build,
and I expect some hand-editing will be required in the end.

The patches are available in the "sphinx" branch of [1], and I've set up
a project at Read the Docs to build that into documentation [2] (mostly
to test this approach also works and so I don't have to host this
anywhere).

Any comments are welcome, but please do remember that I've focused on
polishing the toolchain and build, not the output quality, with release
early, release often in mind.

BR,
Jani.


[1] git://people.freedesktop.org/~jani/drm
[2] https://kernel.readthedocs.io/



Jani Nikula (9):
  kernel-doc: fix use of uninitialized value
  kernel-doc: support printing exported and non-exported symbols
  Documentation/sphinx: add basic working Sphinx configuration and build
  Documentation: add .gitignore
  Documentation/sphinx: add Sphinx kernel-doc directive extension
  Documentation/sphinx: configure the kernel-doc extension
  Documentation: add kernel hacking rst
  Documentation: add kernel api rst
  Documentation: moar files

Jonathan Corbet (1):
  sphinx: cheesy script to convert .tmpl files

 Documentation/.gitignore                  |    1 +
 Documentation/DocBook/Makefile            |    7 +-
 Documentation/Makefile.sphinx             |   63 +
 Documentation/conf.py                     |  384 +++++
 Documentation/crypto-API.rst              | 1870 +++++++++++++++++++++
 Documentation/filesystems.rst             |  314 ++++
 Documentation/gpu.rst                     | 2556 +++++++++++++++++++++++++++++
 Documentation/index.rst                   |   30 +
 Documentation/kernel-api.rst              |  419 +++++
 Documentation/kernel-hacking.rst          |  795 +++++++++
 Documentation/sphinx/convert_template.sed |   14 +
 Documentation/sphinx/kernel-doc.py        |   99 ++
 Documentation/sphinx/post_convert.sed     |   19 +
 Documentation/sphinx/tmplcvt              |   19 +
 Makefile                                  |    5 +-
 scripts/kernel-doc                        |   32 +-
 16 files changed, 6619 insertions(+), 8 deletions(-)
 create mode 100644 Documentation/.gitignore
 create mode 100644 Documentation/Makefile.sphinx
 create mode 100644 Documentation/conf.py
 create mode 100644 Documentation/crypto-API.rst
 create mode 100644 Documentation/filesystems.rst
 create mode 100644 Documentation/gpu.rst
 create mode 100644 Documentation/index.rst
 create mode 100644 Documentation/kernel-api.rst
 create mode 100644 Documentation/kernel-hacking.rst
 create mode 100644 Documentation/sphinx/convert_template.sed
 create mode 100644 Documentation/sphinx/kernel-doc.py
 create mode 100644 Documentation/sphinx/post_convert.sed
 create mode 100755 Documentation/sphinx/tmplcvt

-- 
2.1.4