[GIT PULL] kdbus for 4.1-rc1
From: Greg Kroah-Hartman
Date: Mon Apr 13 2015 - 15:04:11 EST
The following changes since commit 9eccca0843205f87c00404b663188b88eb248051:
Linux 4.0-rc3 (2015-03-08 16:09:09 -0700)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ tags/kdbus-4.1-rc1
for you to fetch changes up to 9fb9cd0f4434a23487b6ef3237e733afae90e336:
kdbus: avoid the use of struct timespec (2015-04-10 14:34:53 +0200)
----------------------------------------------------------------
kdbus for 4.1-rc1
Here's the kdbus pull request for 4.1-rc1.
It's been under development for many years now, and been in linux-next
for many months, and has undergone loads of testing a review and even a few
good arguments. It comes with full documentation and tests.
There has been a few complaints about the code, notably from people who
don't like the use of metadata in the bus messages. That is actually
one of the main features here, as we can get this data in a secure and
reliable way, and it's something that userspace requires today. So
while it does look "odd" to people who are not familiar with dbus, this
is something that finally fixes a number of almost unfixable races in
the current dbus implementations.
The rest of this pull request message comes from the kdbus patch posting
messages as sent to lkml previously:
Reasons kdbus should be in the kernel, instead of userspace as it is
currently done today includes the following:
* Performance: Fewer process context switches, fewer copies, fewer
syscalls, larger memory chunks via memfd. This is really important
for a whole class of userspace programs that are ported from other
operating systems that are run on tiny ARM systems that rely on
hundreds of thousands of messages passed at boot time, and at
"critical" times in their user interaction loops. DBus is not used
for performance sensitive applications because DBus is slow.
We want to make it fast so we can finally use it for low-latency,
high-throughput applications. A simple DBus method-call+reply takes
200us on an up-to-date test machine, with kdbus it takes 8us (with
UDS about 2us). If the packet size is increased from 8k to 128k,
kdbus even beats UDS due to single-copy transfers.
* Security: The peers which communicate do not have to trust each
other, as the only trustworthy component in the game is the kernel
which adds metadata and ensures that all data passed as payload is
either copied or sealed, so that the receiver can parse the data
without having to protect against changing memory while parsing
buffers. Also, all the data transfer is controlled by the kernel,
so that LSMs can track and control what is going on, without
involving userspace. Because of the LSM issue, security people are
much happier with this model than the current scheme of having to
hook into dbus to mediate things.
* More types of metadata can be attached to messages than in userspace
* Semantics for apps with heavy data payloads (media apps, for
instance) with optinal priority message dequeuing, and global
message ordering. Some "crazy" people are playing with using kdbus
for audio data in the system. I'm not saying that this is the best
model for this, but until now, there wasn't any other way to do this
without having to create custom "buses", one for each application
library.
* Being in the kernel closes a lot of races which can't be fixed with
the current userspace solutions. For example, with kdbus, there is a
way a client can disconnect from a bus, but do so only if no further
messages present in its queue, which is crucial for implementing
race-free "exit-on-idle" services
* Eavesdropping on the kernel level, so privileged users can hook into
the message stream without hacking support for that into their
userspace processes
* A number of smaller benefits: for example kdbus learned a way to peek
full messages without dequeing them, which is really useful for
logging metadata when handling bus-activation requests.
* dbus-daemon is not available during early-boot or shutdown.
DBus marshaling is the de-facto standard in all major(!) Linux desktop
systems. It is well established and accepted by many DEs. It also
solves many other problems, including: policy, authentication /
authorization, well-known name registry, efficient broadcasts /
multicasts, peer discovery, bus discovery, metadata transmission, and
more.
It is a shame that we cannot use this well-established protocol for
low-latency applications. We, effectively, have to duplicate all this
code on custom UDS and other transports just because DBus is too slow.
kdbus tries to unify those efforts, so that we don't need multiple
policy implementations, name registries and peer discovery mechanisms.
Furthermore, kdbus implements comprehensive, yet optional, metadata
transmission that allows to identify and authenticate peers in a
race-free manner (which is *not* possible with UDS).
Also, kdbus provides a single transport bus with sequential message
numbering. If you use multiple channels, you cannot give any ordering
guarantees across peers (for instance, regarding parallel name-registry
changes).
Of course, some of the bits above could be implemented in userspace
alone, for example with more sophisticated memory management APIs, but
this is usually done by losing out on the other details. For example,
for many of the memory management APIs, it's hard to not require the
communicating peers to fully trust each other. And we _really_ don't
want peers to have to trust each other.
Another benefit of having this in the kernel, rather than as a userspace
daemon, is that you can now easily use the bus from the initrd, or up to
the very end when the system shuts down. On current userspace D-Bus,
this is not really possible, as this requires passing the bus instance
around between initrd and the "real" system. Such a transition of all
fds also requires keeping full state of what has already been read from
the connection fds. kdbus makes this much simpler, as we can change the
ownership of the bus, just by passing one fd over from one part to the
other.
Given the theoretical advantages above, here are some real-world
examples:
* The Tizen developers have been complaining about the high latency
of DBus for polkit'ish policy queries. That's why their
authentication framework uses custom UDS sockets (called 'Cynara').
If a UI-interaction needs multiple authentication-queries, you don't
want it to take multiple milliseconds, given that you usually want
to render the result in the same frame.
* PulseAudio doesn't use DBus for data transmission. They had to
implement their own marshaling code, transport layer and so on, just
because DBus1-latency is horrible. With kdbus, we can basically drop
this code-duplication and unify the IPC layer. Same is true for
Wayland, btw.
* By moving broadcast-transmission into the kernel, we can use the
time-slices of the sender to perform heavy operations. This is also
true for policy decisions, etc. With a userspace daemon, we cannot
perform operations in a time-slice of the caller. This makes DoS
attacks much harder.
* With priority-inheritance, we can do synchronous calls into trusted
peers and let them optionally use our time-slice to perform the
action. This allows syscall-like/binder-like method-calls into other
processes. Without priority-inheritance, this is not possible in a
secure manner (see 'priority-inheritance').
* Logging-daemons often want to attach metadata to log-messages so
debugging/filtering gets easier. If short-lived programs send
log-messages, the destination peer might not be able to read such
metadata from /proc, as the process might no longer be available at
that time. Same is true for policy-decisions like polkit does. You
cannot send off method-calls and exit. You have to wait for a reply,
even though you might not even care for it. If you don't wait, the
other side might not be able to verify your identity and as such
reject the request.
* Even though the dbus traffic on idle-systems might be low, this
doesn't mean it's not significant at boot-times or under high-load.
If you run a dbus-monitor of your choice, you will see there is an
significant number of messages exchanged during VT-switches, startup,
shutdown, suspend, wakeup, hotplugging and similar situations where
lots of control-messages are exchanged. We don't want to spend
hundreds of ms just to transmit those messages.
Signed-off-by: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx>
----------------------------------------------------------------
Arnd Bergmann (1):
kdbus: avoid the use of struct timespec
Daniel Mack (18):
kdbus: add documentation
kdbus: add uapi header file
kdbus: add driver skeleton, ioctl entry points and utility functions
kdbus: add connection pool implementation
kdbus: add connection, queue handling and message validation code
kdbus: add node and filesystem implementation
kdbus: add code to gather metadata
kdbus: add code for notifications and matches
kdbus: add code for buses, domains and endpoints
kdbus: add name registry implementation
kdbus: add policy database implementation
kdbus: add Makefile, Kconfig and MAINTAINERS entry
kdbus: add walk-through user space example
kdbus: add selftests
Documentation: kdbus: fix location for generated files
kdbus: connection: fix handling of failed fget()
kdbus: Fix CONFIG_KDBUS help text
samples: kdbus: build kdbus-workers conditionally
David Herrmann (5):
kdbus: samples/kdbus: add -lrt
samples/kdbus: drop wrong include
Documentation/kdbus: fix out-of-tree builds
Documentation/kdbus: support quiet builds
selftests/kdbus: fix gitignore
Lucas De Marchi (1):
kdbus: fix header guard name
Lukasz Skalski (1):
Documentation/kdbus: replace 'reply_cookie' with 'cookie_reply'
Nicolas Iooss (1):
kdbus: fix minor typo in the walk-through example
Sergei Zviagintsev (5):
kdbus: uapi: Fix kernel-doc for enum kdbus_send_flags
Documentation: kdbus: Fix list of KDBUS_CMD_ENDPOINT_UPDATE errors
Documentation: kdbus: Update list of ioctls which cause writing to receiver's pool
Documentation: kdbus: Fix description of KDBUS_SEND_SYNC_REPLY flag
Documentation: kdbus: Fix typos
Tyler Baker (1):
selftest/kdbus: enable cross compilation
Documentation/Makefile | 2 +-
Documentation/ioctl/ioctl-number.txt | 1 +
Documentation/kdbus/.gitignore | 2 +
Documentation/kdbus/Makefile | 40 +
Documentation/kdbus/kdbus.bus.xml | 359 ++++
Documentation/kdbus/kdbus.connection.xml | 1250 ++++++++++++
Documentation/kdbus/kdbus.endpoint.xml | 429 ++++
Documentation/kdbus/kdbus.fs.xml | 124 ++
Documentation/kdbus/kdbus.item.xml | 839 ++++++++
Documentation/kdbus/kdbus.match.xml | 555 ++++++
Documentation/kdbus/kdbus.message.xml | 1276 ++++++++++++
Documentation/kdbus/kdbus.name.xml | 711 +++++++
Documentation/kdbus/kdbus.policy.xml | 406 ++++
Documentation/kdbus/kdbus.pool.xml | 326 +++
Documentation/kdbus/kdbus.xml | 1012 ++++++++++
Documentation/kdbus/stylesheet.xsl | 16 +
MAINTAINERS | 13 +
Makefile | 1 +
include/uapi/linux/Kbuild | 1 +
include/uapi/linux/kdbus.h | 979 +++++++++
include/uapi/linux/magic.h | 2 +
init/Kconfig | 13 +
ipc/Makefile | 2 +-
ipc/kdbus/Makefile | 22 +
ipc/kdbus/bus.c | 560 ++++++
ipc/kdbus/bus.h | 101 +
ipc/kdbus/connection.c | 2214 +++++++++++++++++++++
ipc/kdbus/connection.h | 257 +++
ipc/kdbus/domain.c | 296 +++
ipc/kdbus/domain.h | 77 +
ipc/kdbus/endpoint.c | 275 +++
ipc/kdbus/endpoint.h | 67 +
ipc/kdbus/fs.c | 510 +++++
ipc/kdbus/fs.h | 28 +
ipc/kdbus/handle.c | 617 ++++++
ipc/kdbus/handle.h | 85 +
ipc/kdbus/item.c | 339 ++++
ipc/kdbus/item.h | 64 +
ipc/kdbus/limits.h | 64 +
ipc/kdbus/main.c | 125 ++
ipc/kdbus/match.c | 559 ++++++
ipc/kdbus/match.h | 35 +
ipc/kdbus/message.c | 616 ++++++
ipc/kdbus/message.h | 133 ++
ipc/kdbus/metadata.c | 1159 +++++++++++
ipc/kdbus/metadata.h | 57 +
ipc/kdbus/names.c | 772 +++++++
ipc/kdbus/names.h | 74 +
ipc/kdbus/node.c | 910 +++++++++
ipc/kdbus/node.h | 84 +
ipc/kdbus/notify.c | 248 +++
ipc/kdbus/notify.h | 30 +
ipc/kdbus/policy.c | 489 +++++
ipc/kdbus/policy.h | 51 +
ipc/kdbus/pool.c | 728 +++++++
ipc/kdbus/pool.h | 46 +
ipc/kdbus/queue.c | 678 +++++++
ipc/kdbus/queue.h | 92 +
ipc/kdbus/reply.c | 257 +++
ipc/kdbus/reply.h | 68 +
ipc/kdbus/util.c | 201 ++
ipc/kdbus/util.h | 74 +
samples/Kconfig | 7 +
samples/Makefile | 3 +-
samples/kdbus/.gitignore | 1 +
samples/kdbus/Makefile | 9 +
samples/kdbus/kdbus-api.h | 114 ++
samples/kdbus/kdbus-workers.c | 1326 ++++++++++++
tools/testing/selftests/Makefile | 1 +
tools/testing/selftests/kdbus/.gitignore | 1 +
tools/testing/selftests/kdbus/Makefile | 48 +
tools/testing/selftests/kdbus/kdbus-enum.c | 94 +
tools/testing/selftests/kdbus/kdbus-enum.h | 14 +
tools/testing/selftests/kdbus/kdbus-test.c | 923 +++++++++
tools/testing/selftests/kdbus/kdbus-test.h | 85 +
tools/testing/selftests/kdbus/kdbus-util.c | 1615 +++++++++++++++
tools/testing/selftests/kdbus/kdbus-util.h | 222 +++
tools/testing/selftests/kdbus/test-activator.c | 318 +++
tools/testing/selftests/kdbus/test-attach-flags.c | 750 +++++++
tools/testing/selftests/kdbus/test-benchmark.c | 451 +++++
tools/testing/selftests/kdbus/test-bus.c | 175 ++
tools/testing/selftests/kdbus/test-chat.c | 122 ++
tools/testing/selftests/kdbus/test-connection.c | 616 ++++++
tools/testing/selftests/kdbus/test-daemon.c | 65 +
tools/testing/selftests/kdbus/test-endpoint.c | 341 ++++
tools/testing/selftests/kdbus/test-fd.c | 789 ++++++++
tools/testing/selftests/kdbus/test-free.c | 64 +
tools/testing/selftests/kdbus/test-match.c | 441 ++++
tools/testing/selftests/kdbus/test-message.c | 731 +++++++
tools/testing/selftests/kdbus/test-metadata-ns.c | 506 +++++
tools/testing/selftests/kdbus/test-monitor.c | 176 ++
tools/testing/selftests/kdbus/test-names.c | 194 ++
tools/testing/selftests/kdbus/test-policy-ns.c | 632 ++++++
tools/testing/selftests/kdbus/test-policy-priv.c | 1269 ++++++++++++
tools/testing/selftests/kdbus/test-policy.c | 80 +
tools/testing/selftests/kdbus/test-sync.c | 369 ++++
tools/testing/selftests/kdbus/test-timeout.c | 99 +
97 files changed, 34069 insertions(+), 3 deletions(-)
create mode 100644 Documentation/kdbus/.gitignore
create mode 100644 Documentation/kdbus/Makefile
create mode 100644 Documentation/kdbus/kdbus.bus.xml
create mode 100644 Documentation/kdbus/kdbus.connection.xml
create mode 100644 Documentation/kdbus/kdbus.endpoint.xml
create mode 100644 Documentation/kdbus/kdbus.fs.xml
create mode 100644 Documentation/kdbus/kdbus.item.xml
create mode 100644 Documentation/kdbus/kdbus.match.xml
create mode 100644 Documentation/kdbus/kdbus.message.xml
create mode 100644 Documentation/kdbus/kdbus.name.xml
create mode 100644 Documentation/kdbus/kdbus.policy.xml
create mode 100644 Documentation/kdbus/kdbus.pool.xml
create mode 100644 Documentation/kdbus/kdbus.xml
create mode 100644 Documentation/kdbus/stylesheet.xsl
create mode 100644 include/uapi/linux/kdbus.h
create mode 100644 ipc/kdbus/Makefile
create mode 100644 ipc/kdbus/bus.c
create mode 100644 ipc/kdbus/bus.h
create mode 100644 ipc/kdbus/connection.c
create mode 100644 ipc/kdbus/connection.h
create mode 100644 ipc/kdbus/domain.c
create mode 100644 ipc/kdbus/domain.h
create mode 100644 ipc/kdbus/endpoint.c
create mode 100644 ipc/kdbus/endpoint.h
create mode 100644 ipc/kdbus/fs.c
create mode 100644 ipc/kdbus/fs.h
create mode 100644 ipc/kdbus/handle.c
create mode 100644 ipc/kdbus/handle.h
create mode 100644 ipc/kdbus/item.c
create mode 100644 ipc/kdbus/item.h
create mode 100644 ipc/kdbus/limits.h
create mode 100644 ipc/kdbus/main.c
create mode 100644 ipc/kdbus/match.c
create mode 100644 ipc/kdbus/match.h
create mode 100644 ipc/kdbus/message.c
create mode 100644 ipc/kdbus/message.h
create mode 100644 ipc/kdbus/metadata.c
create mode 100644 ipc/kdbus/metadata.h
create mode 100644 ipc/kdbus/names.c
create mode 100644 ipc/kdbus/names.h
create mode 100644 ipc/kdbus/node.c
create mode 100644 ipc/kdbus/node.h
create mode 100644 ipc/kdbus/notify.c
create mode 100644 ipc/kdbus/notify.h
create mode 100644 ipc/kdbus/policy.c
create mode 100644 ipc/kdbus/policy.h
create mode 100644 ipc/kdbus/pool.c
create mode 100644 ipc/kdbus/pool.h
create mode 100644 ipc/kdbus/queue.c
create mode 100644 ipc/kdbus/queue.h
create mode 100644 ipc/kdbus/reply.c
create mode 100644 ipc/kdbus/reply.h
create mode 100644 ipc/kdbus/util.c
create mode 100644 ipc/kdbus/util.h
create mode 100644 samples/kdbus/.gitignore
create mode 100644 samples/kdbus/Makefile
create mode 100644 samples/kdbus/kdbus-api.h
create mode 100644 samples/kdbus/kdbus-workers.c
create mode 100644 tools/testing/selftests/kdbus/.gitignore
create mode 100644 tools/testing/selftests/kdbus/Makefile
create mode 100644 tools/testing/selftests/kdbus/kdbus-enum.c
create mode 100644 tools/testing/selftests/kdbus/kdbus-enum.h
create mode 100644 tools/testing/selftests/kdbus/kdbus-test.c
create mode 100644 tools/testing/selftests/kdbus/kdbus-test.h
create mode 100644 tools/testing/selftests/kdbus/kdbus-util.c
create mode 100644 tools/testing/selftests/kdbus/kdbus-util.h
create mode 100644 tools/testing/selftests/kdbus/test-activator.c
create mode 100644 tools/testing/selftests/kdbus/test-attach-flags.c
create mode 100644 tools/testing/selftests/kdbus/test-benchmark.c
create mode 100644 tools/testing/selftests/kdbus/test-bus.c
create mode 100644 tools/testing/selftests/kdbus/test-chat.c
create mode 100644 tools/testing/selftests/kdbus/test-connection.c
create mode 100644 tools/testing/selftests/kdbus/test-daemon.c
create mode 100644 tools/testing/selftests/kdbus/test-endpoint.c
create mode 100644 tools/testing/selftests/kdbus/test-fd.c
create mode 100644 tools/testing/selftests/kdbus/test-free.c
create mode 100644 tools/testing/selftests/kdbus/test-match.c
create mode 100644 tools/testing/selftests/kdbus/test-message.c
create mode 100644 tools/testing/selftests/kdbus/test-metadata-ns.c
create mode 100644 tools/testing/selftests/kdbus/test-monitor.c
create mode 100644 tools/testing/selftests/kdbus/test-names.c
create mode 100644 tools/testing/selftests/kdbus/test-policy-ns.c
create mode 100644 tools/testing/selftests/kdbus/test-policy-priv.c
create mode 100644 tools/testing/selftests/kdbus/test-policy.c
create mode 100644 tools/testing/selftests/kdbus/test-sync.c
create mode 100644 tools/testing/selftests/kdbus/test-timeout.c
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/