refactoring livepatch documentation was Re: [PATCH 1/2] docs/livepatch: Add new compiler considerations doc

[Joe Lawrence]

On 8/6/20 8:03 AM, Petr Mladek wrote:
> On Wed 2020-07-22 15:51:39, Josh Poimboeuf wrote:
> > On Wed, Jul 22, 2020 at 01:03:03PM -0400, Joe Lawrence wrote:
> > > On 7/21/20 7:04 PM, Josh Poimboeuf wrote:
> > > > On Tue, Jul 21, 2020 at 12:14:06PM -0400, Joe Lawrence wrote:
> > > > > Compiler optimizations can have serious implications on livepatching.
> > > > > Create a document that outlines common optimization patterns and safe
> > > > > ways to livepatch them.
> > > > >
> > > > > Signed-off-by: Joe Lawrence <joe.lawrence@xxxxxxxxxx>
> > > >
> > > > There's a lot of good info here, but I wonder if it should be
> > > > reorganized a bit and instead called "how to create a livepatch module",
> > > > because that's really the point of it all.
> > >
> > > That would be nice.  Would you consider a stand-alone compiler-optimizations
> > > doc an incremental step towards that end?  Note that the other files
> > > (callbacks, shadow-vars, system-state) in their current form might be as
> > > confusing to the newbie.
> >
> > It's an incremental step towards _something_.  Whether that's a cohesive
> > patch creation guide, or just a growing hodgepodge of random documents,
> > it may be too early to say :-)
>
> Yes, it would be nice to have a cohesive documentation. But scattered
> pieces are better than nothing.
>
> > > > I'm thinking a newcomer reading this might be lost.  It's not
> > > > necessarily clear that there are currently two completely different
> > > > approaches to creating a livepatch module, each with their own quirks
> > > > and benefits/drawbacks.  There is one mention of a "source-based
> > > > livepatch author" but no explanation of what that means.
> > >
> > > Yes, the initial draft was light on source-based patching since I only
> > > really tinker with it for samples/kselftests.  The doc was the result of an
> > > experienced livepatch developer and Sunday afternoon w/the compiler. I'm
> > > sure it reads as such. :)
> >
> > Are experienced livepatch developers the intended audience?  If so I
> > question what value this document has in its current form.  Presumably
> > experienced livepatch developers would already know this stuff.
>
> IMHO, this document is useful even for newbies. They might at
> least get a clue about these catches. It is better than nothing.
>
> I do not want to discourage Joe from creating even better
> documentation. But if he does not have interest or time
> to work on it, I am happy even for this piece.

Hi Petr, Josh,

The compiler optimization pitfall document can wait for refactored 
livepatch documentation if that puts it into better context, 
particularly for newbies.  I don't mind either way.  FWIW, I don't 
profess to be an authoritative source its content -- we've dealt some of 
these issues in kpatch, so it was interesting to see how they affect 
livepatches that don't rely on binary comparison.


Toward the larger goal, I've changed the thread subject to talk about 
how we may rearrange and supplement our current documentation.  This is 
a first pass at a possible refactoring...


1. Provide a better index page to connect the other files/docs, like
https://www.kernel.org/doc/html/latest/core-api/index.html but obviously 
not that extensive.  Right now we have only a Table of Contents tree 
without any commentary.

2. Rearrange and refactor sections:

livepatch.rst
  Keep just about everything
  Add a history section to explain ksplice, kgraft, kpatch for the
    uninitiated?
  Add a section on source based vs. binary diff livepatch creation,
    this may be worth its own top-level section

Livepatch API
  Basic API
  Callbacks
  Shadow variables
  Cumulative patches
  System state

KLP Relocations
  Right now this is a bit academic AFAIK kpatch is the only tool
  currently making use of them.  So maybe this document becomes a
  more general purpose doc explaining how to reference unexported
  symbols?  (ie, how does kgraft currently do it, particularly
  w/kallsyms going unexported?)

  Eventually this could contain klp-convert howto if it ever gets
  merged.

Compiler considerations
  TBD

I suppose this doesn't create a "Livepatching creation for dummies" 
guide, but my feeling is that there are so many potential (hidden) 
pitfalls that such guide would be dangerous.

If someone were to ask me today how to start building a livepatch, I 
would probably point them at the samples to demonstrate the basic 
concept and API, but then implore them to read through the documentation 
to understand how quickly complicated it can become.

-- Joe