[PATCH v05 00/72] Userspace compile test and fixes for exported uapi header files

From: Mikko Rapeli
Date: Mon Aug 22 2016 - 14:47:07 EST

Next message: Eric Dumazet: "Re: [PATCH tip/core/rcu 2/2] documentation: Record reason for rcu_head two-byte alignment"
Previous message: Mikko Rapeli: "[PATCH v05 02/72] scripts/headers_compile_test.sh: compile test script for exported headers"
Next in thread: Mikko Rapeli: "[PATCH v05 01/72] headers_install.sh: enhance error handling"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

v5:
two years now from v1 to v5 though some of the patches have been applied
with these changes all uapi headers compile stand alone in users space
added myself to MAINTAINERS for the test script
tried to fix all v04 review findings
due to regression in one of the accepted patches, added a glibc compatibility test
fixed some glibc compatibility issues, which are hopefully on the right path

v4:
https://lkml.org/lkml/2015/10/15/22
tried to fix v03 review findings
tried to fix all hack patches

v3:
https://lkml.org/lkml/2015/5/30/96
tried to fix all v2 review findings
tried to guess how to fix a few more issue
with a few hacks, 0 files fail and 760 files pass the compile test on 32bit x86

v2:
https://lkml.org/lkml/2015/2/16/521
added cross compiler support with CROSS_COMPILE,
detecting libc and GCC headers from compiler,
more header file fixes

v1:
https://lkml.org/lkml/2014/8/21/665

Users of kernel header files would be happier if they did not contain
kernel specific parts and would contain #include statements for all
other header files that they depend on, and in general would compile.

This patch set introduces a compile test for headers exported to userspace
and then fixes many of the userspace header compilation failures.

Changes are also available in headers_test_v05 branch at github:
https://github.com/mcfrisk/linux/tree/headers_test_v05

Here's how to run the userspace compile tests and example output with these
patches applied on top of v4.8-rc2-348-g6040e57:

$ make headers_install && cd usr/include && \
../../scripts/headers_compile_test.sh -l
...
Kernel header compile test statistics:

0 files failed the kernel header compile test.
784 files passed the kernel header compile test.

libc and kernel header compatibility test statistics:
114 files failed the libc compatibility test.
670 files passed the libc compatibility test.
39 files failed libc before kernel include test.
745 files passed libc before kernel include test.
113 files failed kernel before libc include test.
671 files passed kernel before libc include test.

Additionally tested an allyesconfig compilation on i686.

If these changes are merged, it becomes possible to run abi-compliance-checker
tool https://lvc.github.io/abi-compliance-checker/ to detect API and ABI
breakages from the uapi headers, but some scripting around the tool is required.

Once all archs have completely compiling uapi headers, I would add this test
to 'make headers_check' build target to prevent regressions.

If the initial set of glibc compatibility fixes is going in the right
direction, I can try to fix the kernel side of those, and maybe try to
get glibc headers to play nicely with kernel uapi headers.

Some kernel subsystem and their uapi headers adhere to
Documentation/CodingStyle regarding C99 and __u32 etc types but many don't and
have even good reasons for not doing so, e.g. main userspace user of the headers
e.g. libdrm or libfuse require C99 stdint.h types. I'm hoping we could finally
settle this by for example agreeing that C99 stdint.h types are allowed in some
uapi subsystems, e.g. drm and fuse. Help would be appreciated here.

Mikko Rapeli (72):
headers_install.sh: enhance error handling
scripts/headers_compile_test.sh: compile test script for exported
headers
headers_compile_test.sh: add GNU libc compatibility test
dm-log-userspace.h: use __u32, __s32 and __u64 from linux/types.h
hsi_char.h: use __u32 from linux/types.h
include/uapi/linux/hsi/cs-protocol.h: include linux/time.h
cld.h: use __u8, __u16, __s16, __u32 and __s64 from linux/types.h
rds.h: use __u8, __u16, __s16, __u32 and __s64 from linux/types.h
sctp.h: use __u8 and __u32 from linux/types.h
scsi_bsg_fc.h: use __u8, __u32 and __u64 from linux/types.h
scsi_netlink.h: use __u8, __u16 and __u64 from linux/types.h
scsi_netlink_fc.h: use __u16, __u32 and __u64 from linux/types.h
include/uapi/linux/sysctl.h: use __kernel_size_t instead of size_t
include/uapi/asm-generic/ipcbuf.h: include linux/posix_types.h
include/uapi/asm-generic/msgbuf.h: include asm/ipcbuf.h
include/uapi/asm-generic/shmbuf.h: include fixes
asm-generic/sembuf.h: include asm/posix_types.h and asm/ipcbuf.h
include/uapi/asm-generic/signal.h: use __kernel_size_t instead of
size_t
include/uapi/linux/socket.h: include sys/socket.h in userspace
include/uapi/linux/rds.h: include linux/socket.h and linux/types.h
include/uapi/linux/if_pppox.h: include linux/if.h
include/uapi/linux/if_tunnel.h: include linux/if.h, linux/ip.h and
linux/in6.h
include/uapi/linux/ipv6_route.h: include linux/in6.h
include/uapi/linux/ipv6_route.h: include linux/in6.h
include/uapi/linux/if_pppol2tp.h: include linux/in.h and linux/in6.h
include/uapi/linux/if_pppox.h: include linux/in.h and linux/in6.h
linux/if.h linux/hdlc/ioctl.h: move IFNAMSIZ definition to
hdlc/ioctl.h
include/uapi/linux/packet_diag.h: include linux/netdevice.h
include/uapi/linux/llc.h: include linux/if.h
include/uapi/linux/mqueue.h: include linux/types.h
include/uapi/linux/mroute.h: include linux/in.h
include/uapi/linux/dlm_netlink.h: include linux/dlmconstants.h
include/uapi/linux/ip6_tunnel.h: include linux/if.h and linux/in6.h
include/uapi/linux/errqueue.h: include linux/time.h
include/uapi/asm-generic/ucontext.h: include asm/signal.h and
asm/sigcontext.h
arch/x86/include/uapi/asm/signal.h: use __kernel_size_t instead of
size_t
include/uapi/linux/auto_fs.h: include linux/limits.h
include/uapi/linux/openvswitch.h: use __u32 from linux/types.h
include/uapi/linux/target_core_user.h: use __u8, __u16, __u32 and
__u64 from linux/types.h
include/uapi/linux/omapfb.h: use __kernel_size_t instead of size_t
include/uapi/linux/atm_zatm.h: include linux/time.h
include/uapi/linux/scc.h: include linux/sockios.h
include/uapi/linux/btrfs.h: define NULL
include/uapi/linux/kexec.h: use __kernel_size_t instead of size_t
include/uapi/linux/reiserfs_xattr.h: use __kernel_size_t instead of
size_t
include/uapi/linux/patchkey.h: change #error to #warning if file
included directly
include/uapi/linux/coda.h: use __kernel_pid_t and add u_short etc
definitions for userspace
include/uapi/linux/android/binder.h: use __kernel_pid_t and
__kernel_uid_t
arch/x86/include/uapi/asm/sembuf.h: include linux/types.h and
linux/ipc.h
include/uapi/linux/fuse.h: use linux/types.h also in userspace
include/linux/ipmi-fru.h: use linux/types.h instead of stdint.h
include/linux/sdb.h: use linux/types.h types instead of stdint.h
include/linux/xz.h: use linux/types.h types instead of stdint.h
include/uapi/xen/privcmd.h: fix compilation in userspace
include/uapi/xen/gntdev.h: include xen/privcmd.h and define
grant_ref_t
include/uapi/xen/evtchn.h: include xen/privcmd.h
coda_psdev.h: move upc_req definition from uapi to kernel side headers
include/uapi/linux/elfcore.h: remove non-compiling userspace parts
include/uapi/linux/errqueue.h: include time.h in userspace
arch/arm/include/uapi/asm/signal.h: use __kernel_size_t instead of
size_t
include/uapi/rdma/rdma_user_rxe.h: include in.h and in6.h
include/uapi/linux/openvswitch.h: use __u32 from linux/types.h
include/uapi/sound/Kbuild: remove sound/asoc.h from userspace
include/uapi/linux/ipx.h: fix conflicting defitions with glibc
netipx/ipx.h
uapi fcntl.h: glibc fcntl.h header file compatibility fixes
uapi icmp.h: glibc netinet/ip_icmp.h header file compatibility fixes
uapi uio.h: glibc sys/uio.h header file compatibility fixes
uapi ax25.h: glibc netax25/ax25.h header file compatibility fixes
uapi rose.h: glibc netrose/rose.h header file compatibility fixes
uapi time.h: glibc time.h header file compatibility fixes
uapi term*.h: glibc termios.h header file compatibility fixes
uapi route.h: glibc net/route.h compat fix

MAINTAINERS | 5 +
arch/arm/include/asm/xen/interface.h | 2 +-
arch/arm/include/uapi/asm/signal.h | 2 +-
arch/x86/include/uapi/asm/sembuf.h | 3 +
arch/x86/include/uapi/asm/signal.h | 2 +-
include/linux/coda_psdev.h | 11 +
include/linux/ipmi-fru.h | 31 +--
include/linux/sdb.h | 62 ++---
include/linux/xz.h | 11 +-
include/uapi/asm-generic/fcntl.h | 6 +
include/uapi/asm-generic/ipcbuf.h | 2 +
include/uapi/asm-generic/msgbuf.h | 2 +
include/uapi/asm-generic/sembuf.h | 2 +
include/uapi/asm-generic/shmbuf.h | 4 +-
include/uapi/asm-generic/signal.h | 2 +-
include/uapi/asm-generic/termbits.h | 3 +
include/uapi/asm-generic/termios.h | 5 +
include/uapi/asm-generic/ucontext.h | 3 +
include/uapi/linux/android/binder.h | 4 +-
include/uapi/linux/atm_zatm.h | 1 +
include/uapi/linux/auto_fs.h | 1 +
include/uapi/linux/ax25.h | 22 ++
include/uapi/linux/btrfs.h | 6 +
include/uapi/linux/coda.h | 11 +-
include/uapi/linux/coda_psdev.h | 13 -
include/uapi/linux/dlm_netlink.h | 1 +
include/uapi/linux/dm-log-userspace.h | 43 +--
include/uapi/linux/elfcore.h | 21 +-
include/uapi/linux/errqueue.h | 6 +
include/uapi/linux/fcntl.h | 3 +
include/uapi/linux/fuse.h | 452 +++++++++++++++---------------
include/uapi/linux/hdlc/ioctl.h | 5 +
include/uapi/linux/hsi/cs-protocol.h | 1 +
include/uapi/linux/hsi/hsi_char.h | 17 +-
include/uapi/linux/icmp.h | 4 +-
include/uapi/linux/if.h | 5 +-
include/uapi/linux/if_pppol2tp.h | 3 +-
include/uapi/linux/if_pppox.h | 3 +
include/uapi/linux/if_tunnel.h | 3 +
include/uapi/linux/ip6_tunnel.h | 2 +
include/uapi/linux/ipv6_route.h | 1 +
include/uapi/linux/ipx.h | 13 +-
include/uapi/linux/kexec.h | 4 +-
include/uapi/linux/libc-compat.h | 212 ++++++++++++++
include/uapi/linux/llc.h | 1 +
include/uapi/linux/mqueue.h | 2 +
include/uapi/linux/mroute.h | 1 +
include/uapi/linux/mroute6.h | 1 +
include/uapi/linux/nfsd/cld.h | 14 +-
include/uapi/linux/omapfb.h | 2 +-
include/uapi/linux/openvswitch.h | 6 +-
include/uapi/linux/packet_diag.h | 1 +
include/uapi/linux/patchkey.h | 2 +-
include/uapi/linux/rds.h | 103 +++----
include/uapi/linux/reiserfs_xattr.h | 2 +-
include/uapi/linux/rose.h | 15 +
include/uapi/linux/route.h | 4 +-
include/uapi/linux/scc.h | 1 +
include/uapi/linux/sctp.h | 10 +-
include/uapi/linux/socket.h | 4 +
include/uapi/linux/sysctl.h | 4 +-
include/uapi/linux/target_core_user.h | 22 +-
include/uapi/linux/time.h | 18 +-
include/uapi/linux/uio.h | 4 +-
include/uapi/rdma/rdma_user_rxe.h | 2 +
include/uapi/scsi/scsi_bsg_fc.h | 54 ++--
include/uapi/scsi/scsi_netlink.h | 22 +-
include/uapi/scsi/scsi_netlink_fc.h | 16 +-
include/uapi/sound/Kbuild | 1 -
include/uapi/xen/evtchn.h | 2 +
include/uapi/xen/gntdev.h | 6 +
include/uapi/xen/privcmd.h | 12 +-
include/xen/interface/grant_table.h | 6 +-
scripts/headers_compile_test.sh | 507 ++++++++++++++++++++++++++++++++++
scripts/headers_install.sh | 14 +-
75 files changed, 1375 insertions(+), 499 deletions(-)
create mode 100755 scripts/headers_compile_test.sh

--
2.8.1

Next message: Eric Dumazet: "Re: [PATCH tip/core/rcu 2/2] documentation: Record reason for rcu_head two-byte alignment"
Previous message: Mikko Rapeli: "[PATCH v05 02/72] scripts/headers_compile_test.sh: compile test script for exported headers"
Next in thread: Mikko Rapeli: "[PATCH v05 01/72] headers_install.sh: enhance error handling"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]