Re: [PATCH v3 1/2] Provide in-kernel headers for making it easy to extend the kernel

From: Dietmar Eggemann
Date: Thu Feb 28 2019 - 11:05:09 EST


Hi Joel,

On 2/28/19 3:47 PM, Joel Fernandes wrote:
On Thu, Feb 28, 2019 at 01:53:43PM +0000, Qais Yousef wrote:
Hi Joel

On 02/27/19 14:37, Joel Fernandes (Google) wrote:

[...]

Ah good catch, I made this change for "file_list=${@:2}" in my tree but
forgot to push it. Below is the updated patch. Sorry and I'll refresh the
series with the change after we finish the discussion in the other thread.
Meanwhile the updated patch is as follows...

---8<-----------------------

From: "Joel Fernandes (Google)" <joel@xxxxxxxxxxxxxxxxx>
Subject: [PATCH v3.1] Provide in-kernel headers for making it easy to extend the kernel

Introduce in-kernel headers and other artifacts which are made available
as an archive through proc (/proc/kheaders.tar.xz file). This archive makes
it possible to build kernel modules, run eBPF programs, and other
tracing programs that need to extend the kernel for tracing purposes
without any dependency on the file system having headers and build
artifacts.

On Android and embedded systems, it is common to switch kernels but not
have kernel headers available on the file system. Raw kernel headers
also cannot be copied into the filesystem like they can be on other
distros, due to licensing and other issues. There's no linux-headers
package on Android. Further once a different kernel is booted, any
headers stored on the file system will no longer be useful. By storing
the headers as a compressed archive within the kernel, we can avoid these
issues that have been a hindrance for a long time.

The feature is also buildable as a module just in case the user desires
it not being part of the kernel image. This makes it possible to load
and unload the headers on demand. A tracing program, or a kernel module
builder can load the module, do its operations, and then unload the
module to save kernel memory. The total memory needed is 3.8MB.

The code to read the headers is based on /proc/config.gz code and uses
the same technique to embed the headers.

This version gives me the header files on a v5.0-rc8 kernel on my arm64 box but does not compile anymore on v4.20:

kernel/kheaders.c:25:22: error: expected identifier or â(â before string constant
#define KH_MAGIC_END "IKHD_ED"
^
kernel/kheaders_data.h:1:1: note: in expansion of macro âKH_MAGIC_ENDâ
KH_MAGIC_END;
^~~~~~~~~~~~
kernel/kheaders.c: In function âikheaders_read_currentâ:
kernel/kheaders.c:38:12: error: âkernel_headers_dataâ undeclared (first use in this function); did you mean âkernel_headers_data_sizeâ?
kernel_headers_data + KH_MAGIC_SIZE,
^~~~~~~~~~~~~~~~~~~
kernel_headers_data_size
kernel/kheaders.c:38:12: note: each undeclared identifier is reported only once for each function it appears in
kernel/kheaders.c: In function âikheaders_initâ:
kernel/kheaders.c:31:10: error: âkernel_headers_dataâ undeclared (first use in this function); did you mean âkernel_headers_data_sizeâ?
(sizeof(kernel_headers_data) - 1 - KH_MAGIC_SIZE * 2)
^
kernel/kheaders.c:57:23: note: in expansion of macro âkernel_headers_data_sizeâ
proc_set_size(entry, kernel_headers_data_size);
^~~~~~~~~~~~~~~~~~~~~~~~
kernel/kheaders.c: In function âikheaders_read_currentâ:
kernel/kheaders.c:40:1: warning: control reaches end of non-void function [-Wreturn-type]
}


The reason for me to stay on v4.20 is that with v5.0-rc8 I don't have ebpf 'raw tracepoint' support any more on my arm64 board. But this issue is not related to your patch though.

Another point which supports the functionality your patch provides is the fact that maintainers don't want to see new TRACE_EVENTs in their code. So here your patch comes handy when using ebpf for tracing in embedded environments.