Re: [PATCH v4 1/2] Provide in-kernel headers for making it easy to extend the kernel

[Joel Fernandes]

Hi Masami,
Sorry for late reply as I was traveling.

On Wed, Mar 13, 2019 at 10:18:43AM +0900, Masami Hiramatsu wrote:
> On Mon, 11 Mar 2019 18:38:49 -0700
> Joel Fernandes <joelaf@xxxxxxxxxx> wrote:
> 
> > On Mon, Mar 11, 2019 at 6:28 PM Steven Rostedt <rostedt@xxxxxxxxxxx> wrote:
> > >
> > > On Mon, 11 Mar 2019 20:39:12 -0400
> > > Joel Fernandes <joel@xxxxxxxxxxxxxxxxx> wrote:
> > >
> > > > I think even though the kernel-headers can't have information about all data
> > > > structures, they do already contain a lot of data structure definitions we
> > > > need already. And anything needed can/should arguably be moved to include/ if
> > > > they are really needed for kernel extension by something "external" to the
> > > > kernel such as kernel modules or eBPF, right?
> > >
> > > That's not my worry. I would like to be able to easily walk data
> > > structures from within the kernel, without having to do a lot of work
> > > in userspace to get that information. The kprobe_events could then be
> > > passed type casts or such to access data fields of arguments to
> > > functions and such.
> > 
> > Ok.
> > 
> > > > In any case, such a solution such as what Steve suggested, still cannot do
> > > > what we can with headers - such as build kernel modules on the fly using the
> > > > C-compiler without any auto-generation of C code from any debug artifiacts.
> > > > Think systemtap working with the module-backend without any need for
> > > > linux-headers package on the file system. So such a solution would still be a
> > > > bit orthogonal in scope to what this proposed solution can solve IMO.
> > > >
> > >
> > > With the information I would like to have, it would be trivial to read
> > > the data to create the header files needed for modules.
> > 
> > But there are macros and other #define things too. We lose all of them
> > and can't recreate them from just DWARF (AFAIK). Including
> > include/generated/autoconf.h which #defines the CONFIG options. For
> > that we either need headers, or full kernel's sources with build
> > artifacts.
> 
> What kind of macros would you concern?
> 
> > I do see a use case for the debug info you are talking about as you
> > mentioned for the kprobe_events argument list types, and I already
> > thought about it. But it does not seem to work for all the use cases I
> > am referring to here.
> 
> But the eBPF is based on kprobe-events. What kind of usage would you
> expected? (with macros??)

eBPF C programs are compiled with kernel headers. They can execute inline
functions or refer to macros in the kernel headers. They are similar to
kernel modules where you build a C program that then later is executed in
kernel context. It goes through the whole compiler pipeline. This is slightly
different usage from pure kprobe-events.  Also eBPF kprobe programs need
LINUX_VERSION_CODE (or similarly named) macro which it provides to the bpf(2)
syscall when loading kprobe programs. This is because eBPF implementation in
the kernel checks if the eBPF programs that use kprobes are being loaded
against the right kernel.

thanks,

- Joel