Re: [GIT PULL] kdbus for 4.1-rc1

[Havoc Pennington]

On Tue, Apr 28, 2015 at 7:12 PM, John Stoffel <john@xxxxxxxxxxx> wrote:
> Havoc> Yeah. I don't know how you answer that, because the answer is
> Havoc> probably "it would be good enough for some things and not for
> Havoc> other things." It depends on whether an app is sending enough
> Havoc> data to be too slow, and it depends on the hardware, right.
>
> So what happens if we put kdbus into the kernel and it's still too
> slow?  What then?

What my above paragraph was intended to mean is: I don't understand
what it means to ask about a "too slow" fixed line here. Every time
you make it substantively faster, it works for more apps or on slower
hardware, presumably. You dial the speed, and you include or exclude
certain app ideas accordingly.

I think dbus works for lots of purposes now, despite being slow. Lots
of people are using it. In many uses, super-slow-dbus might be 1% of
the profile of whatever the user-visible functionality is, and nobody
cares how fast dbus is. In other uses, they might.

Some people are saying they would use it in more ways if it were
faster and/or available in early boot and/or whatever else. I'm not
those people, because right now I'm not working on dbus or anything
using dbus. They would have to say what's 'fast enough' for them.

"What happens if unix sockets are too slow? what then?" - it's not a
coherent question. It's always relative to what you're trying to do,
surely.

> I'm not sure I agree with this statement, just putting something into
> the kernel doesn't magically make the work go away

The kdbus guys should really explain this. I have my understanding of
it but theirs will be more accurate.

> Which is also telling you that maybe userspace could be improved more,
> before it needs to even think about going into the kernel?

I imagine people have already improved the part of userspace they are
thinking of keeping (sd-dbus, replacing libdbus) and they don't want
to rewrite dbus-daemon only to immediately discard it. (The part of
the "dbus" overall system which hasn't been rewritten and optimized is
the daemon, which could be dropped completely in kdbus-world.)

It's not especially mysterious what's slow about the existing daemon
implementation, in my opinion; it's been the same for 10 years. The
rough outline of speeding it up would be to replace libdbus with
something sd-dbus-like, and then do a round of profiling and tuning.
The 2012 email I linked to earlier had some other ideas.  But this is
a lot of work, it isn't "just" port to sd-dbus, the daemon is strongly
entangled with libdbus right now. I don't blame people for being
unmotivated on this if they believe it's a dead end.

In that same 2012 email you'll notice I advised doing exactly what
Linus suggests; do the userspace tuning rather than quote "arguing
with kernel developers":
 http://lists.freedesktop.org/archives/dbus/2012-March/015024.html

But I do admire that people felt kdbus was the right answer so have
gone for it anyway, and I do think Linux as a complete OS
(kernel+userspace) deserves a great answer in this problem space.

> LDAP is pretty damn generic, in that you can put pretty large objects
> into it, and pretty large OUs, etc.  So why would it be a candidate
> for going into the kernel?  And why is kdbus so important in the
> kernel as well?  People have talked about it needing to be there for
> bootup, but isn't that why we ripped out RAID detection and such from
> the kernel and built initramfs, so that there's LESS in the kernel,
> and more in an early userspace?  Same idea with dbus in my opinion.
>

I don't have a well-developed philosophy on what should be in the
kernel or not. That is something the kernel maintainers have to
answer. My main concern here is that people understand what dbus is
about historically, so they don't do silly stuff - whether cargo cult
keeping a 'feature' that was always a bad idea, or speeding it up by
breaking intentional and important semantics, or whatever.

When I see people saying they don't understand what dbus is because
they have no idea how a Linux workstation userspace is put together,
that's something I can help with.

When I see people saying maybe it isn't worth the complexity to put
this in the kernel if it's only an N% speedup, I can see that, I'm not
going to say that's wrong or right. It depends to me on what apps are
enabled by the N%, or whether early boot and other factors are
important.

> When Ted is saying it's hard to debug... then maybe it's a bit crappy
> in design or implementation?

Or maybe he just doesn't know how to debug it, honestly. I find the
kernel hard to debug because I know very little about it. I find the
desktop simple to debug, at least as simple as debugging millions of
lines of code can be. The difference is that I have never done kernel
debugging and I'm already familiar with how the desktop works.

dbus has tools that log every message and let you explore and
introspect everything on it, etc. - it works for me.

> So why DOES audio need to go via DBUS?  What about video?  Why
> shouldn't that go via dbus as well?
>
> If one userspace implementation is so crappy, why can't that
> implementation be tossed and a better one done?  Or why can't they
> just optimize/tune it in userspace instead?

In this email I listed what I could remember app developers bringing
up when told to use a sidechannel instead of dbus:
http://article.gmane.org/gmane.linux.kernel/1935002

I can't speak to what makes sense for audio or video, but I'm sure
people who work on those things could.

Re: why can't it be done in userspace, the only thing I'd repeat again
here is that when people mention ways to speed up the bus daemon in
userspace, they often sound like they would abandon one or more of the
semantic guarantees of dbus (usually ordering, sometimes things like
the guaranteed-correct sender information or whatever). And _maybe_
some of those guarantees are worth abandoning, but I'd be very careful
with it.

Havoc
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/