Re: [Gta04-owner] [PATCH 0/4] UART slave device support - version 4

[H. Nikolaus Schaller]

Just a first short answer (can't work 7/24 :).

Am 18.01.2016 um 23:03 schrieb One Thousand Gnomes <gnomes@xxxxxxxxxxxxxxxxxxx>:

>>> Your user space can do it (as most Android does).
>> 
>> How can it do it in automatically in a standardized way without need for daemon support?
> 
> You don't need to - it can be device specific. In Android it usually is.
> I've never understood why low end devices don't also abstract user space
> descriptions of power control into DT nodes as well as kernel properties ?
> 
>> * how can the daemon present another /dev/tty so that applications expecting such
>> an interface can attach to it (maybe through pty - but isn't that an overkill?)
> 
> You don't need to - you can monitor the rx/tx stats via procfs too. Not
> pretty but given the daemon can do both the decoding and the monitoring
> for your device it all seems a strawman. I'd argue given the sheer range
> of ways people PM a GPS that you ought to have those power abstractions
> in the user space daemon. They might use kernel methods, they might not.
> 
>> * Who makes sure that this daemon is installed and running right after boot up on *any* Linux system
>> so that it can always react in case that the chip did start when it should be off?
> 
> Whoever puts the distribution together. The kernel runs init. Beyond that
> we don't care. Not our problem. You can boot into emacs if you want.

Well, it is my big problem which goes contrary to our goal to have the best
supported platform... We would have to provide and maintain such things
so that they are compatible to a plethora of unknown runtime environments.

> 
> 
>> More generally: what is a kernel good for? Why do we need kernel drivers?
> 
> To maintain security, to manage hardware elements that cannot be managed
> in user space, and to provide abstractions where the abstraction is
> necessary and meaningful to a large number of users (not a corner case
> phone device unless the abstraction can be made meaningful and widely
> useful).
> 
>> Just think about waking up the daemon process if a character is received. This is
>> much more costly than calling a notification function in the kernel driver which might
>> have to execute just 4 or 5 assembler instructions to decide what to do.
> 
> The logical continuation of that is of course not to have user space but
> write your entire phone device in ring 0. Point being it is always a
> trade off.
> 
> Your only way currently to do that is to open the tty and set a line
> discipline which does your monitoring then hold it open. We can't stack
> ldiscs either. If you want to monitor the line state with the physical
> uart receiver powered down then this won't work at all.
> 
>> So kernel drivers are sometimes the best solution and more efficient and controllable
>> than a user space daemon.
> 
> Sometimes but not always. And even if a kernel device is the right
> solution to your device that doesn't mean it's general purpose enough to
> justify being stuffed upstream. Android is full of "interesting" device
> specific tweaks the sum of which would sink the kernel project.
> 
>> I understand that. But what is Linux good for? For it's own sake or for
>> users and platforms using it? Isn't it that we take it from the community
>> and contribute improvements back to it?
> 
> Only when the community benefits overall.
> 
>> So to me it appears that such a kernel feature is missing. Therefore we
>> are discussing it.
> 
> I'm glad - because it raises some hard questions and while I don't agree
> with some of your starting points (like needing to "open" a uart without
> user space) I do agree that
> 
> - There isn't a nice way to bind extra non device specific behaviour to
>  open and close (but we have the right places to add one)
> 
> - There isn't a way to monitor rx data (and this is *really* hard to
>  sort especially when the uart is powered down)

Exactly. This is why we already work 3 years on this topic...

The solution is to optionally keep it powered up - as long as the peer
device asks for.

> 
> - Monitoring mctrl via a nice abstraction is going to need the mctrl
>  ops moving from tty to tty_port if you want to use that to trigger tty
>  slave behaviour. Doable but a change.
> 
>> well I think most driver projects that have expressed that they want to have
>> such a solution just want an UART abstraction and not a general tty/serial
>> interface with all bells and whistles.
> 
> There isn't any difference at the proper abstraction layer. If you wanted
> your monitor to power on and off when you open a console that's the same
> problem space.
> 
>>> uart is just a helper library for some types of
>>> port.
>> 
>> Yes, this is the type of port, our peer devices are directly connected to.
> 
> Think about the bigger picture. The high level abstraction is the tty and
> the tty_port. But see below as I think your mental model is perhaps wrong
> and this is a point of confusion ?
> 
>> Another question if we discuss moving the hooks into the tty_port layer:
>> 
>> is it possible from tty layer to activate the UART without a user-space open()
>> and keep it activated after a close()?
> 
> No the entire tty layer from top to bottom assumes that you activate a
> device by opening it from user space. uart, tty, the lot. The only
> corner case weird exception is the serial console and that is a horrible
> hack for output only - so anything that fixes that assumption should also
> be made to fix the console case.
> 
> It's never really mattered because the obscure corner cases where you
> want a tty held open the cost is minimal because you just let systemd or
> similar own the file handle along with everything else it is hanging on
> to.
> 
>>> The ones I am familiar with either have the userspace managing it via
>>> sysfs (which has some latency advantages when doing clever stuff) or
>>> wired the power control to the carrier signal (or that is declared the
>>> gpio that controls it to be the carrier).
>> 
>> How many of these special drivers are in mainline? Our target is to get full support
>> by mainline and not run our own kernel branch forever. Because we are not
>> yet-another-android-thing-that-just-needs-to-work-for-6-months-and-nobody-cares-
>> about-updates-and-GPL.
> 
> Both of those techniques work in mainline without kernel changes (at
> least on devices where the right gpio sysfs nodes exist). Not pretty and
> it would be better to abstract it - although ACPI often abstracts it by
> having the kernel power down the uart which calls into ACPI which pokes a
> GPIO line so presumably devicetree could learn the same tricks if it got
> better at describing power management.
> 
>> In all mainlining attempts I know, the user-space control approach was rejected
>> because it was said that the kernel should take care of and not a /sysfs or some
>> other artificial protocol.
> 
> I'm not sure who said that but they obviously didn't look at the real
> world 8)
> 
>> But anyway, it would not solve the device RX data stream monitoring problem.
>> It just could be a solution to know when to turn the chip on or off.
> 
> This I think is actually the really hard and interesting part of the
> problem. The "tell me about open and close" case is simple and can be
> done via tty_port today with minimal extra hooks. There is a small
> question about how you set those hooks from a DT binding - mostly because
> we want the tty_port method table to be const for security reasons.
> 
> The data one is much harder because the abstractions in the tty_port
> never see data. It goes direct from the hardware interface (possibly via
> a support library liek uart but not always) on to the core of the tty
> code. It's also a very hot path on things like older USB 3G modems that
> use AT commands. Peter has done minor miracles on making it more
> efficient but a USB modem can still really stress a small CPU without any
> further callbacks appearing.
> 
>> Neil's proposal to monitor the RX line was rejected because it was doing
>> nasty tricks with switching pinmux states and setting a temporary interrupt
>> on the UART RX line.
> 
> For some hardware that is the only way I know to do this because the
> power hungry uart receiver is physically powered down. I would have to
> check but I *think* that is true even on a modern x86 PC that supports
> wakeups via serial - although it may be well hidden in ACPI and firmware.
> 
>> Therefore we now want to monitor the RX line "behind" the UART, i.e. on the
>> byte stream coming from the UART shift registers.
>> 
>> This did lead to tty/uart slaves concept and everybody wanted it. Now as we
>> have code proposals it should go back to become a user space daemon...
> 
> I'm not personally opoosed to the tty slave idea providing it ends up
> attached to the tty_port not just uart.
> 
>>> 2. If not then hook tty_port_shutdown() and tty_port_open() because those
>>> are the right abstraction point. Everything in the kernel that is a tty
>>> is a tty_port.
>> 
>> What is in your view the right abstraction point for a peer device driver to get
>> notified about rx characters (even if the tty is currently not open)?
> 
> I don't think you have one. A lot of hardware has the receiver and
> transceiver physically powered off when the tty is closed. You can't even
> touch the registers in some cases (eg a PCI port in D3 state).

This is why I want to keep the UART open as long as the peer driver
tells to do so.

> 
> I certainly have no good ideas for that specific case because apart from
> things buried in stuff like ACPI I don't know of any general purpose way
> to ask "how do I make some other piece of hardware peek at the level on
> that line and/or trigger on rising or falling edges"
> 
>> I think on a level of abstraction of all the different UART drivers. Which is sufficient
>> to access the UART by the peer driver. This is why I call it UART-peer (and
>> not serial or tty peer).
> 
> Not all uarts use the uart layer. It's an optional helper.
> 
>> You want to abstract from UART and add all tty bells and whistles. We need them
>> for the GPS chip's data stream (because our unknown GPS applications might
>> use them) - but other low level UART peer drivers do not even need them.
> 
> Ok I think the model you have may be wrong.
> 
> The lowest level physical interface abstraction in the whole tty stack is
> the tty_port. Every object which can be opened as a tty contains a
> tty_port, and the tty_port lifetime is the lifetime of the "physical"
> interface to which it is attached.
> 
> tty_port requires the device provide a set of methods.
> 
> uart optionally sits on top of tty_port. It provides the tty_port methods
> and provides a differentish set of low level abstractions more suited to
> byte oriended serial port devices. Console doesn't use it, sdio serial
> doesn't use it, some uarts don't use it, usb doesn't use it (including
> on-board HSIC and SSIC devices), most jtag serials don't use it either.
> 
> The tty layer talks to the tty_port layer and also directly to tty
> methods provided by the physical device.

Ok, I see. My picture was more a layer scheme based on the path how
data flows from the hardware specific uart driver to the uart layer and
then to the tty_port things.

> Conveniently for you the
> tty_port abstracts opening and closing the device. Inconveniently for you
> it doesn't really mediate receiving and sending data although it does hold
> the buffers used.

Yes.

> 
>>> Then all you need is the (possibly device specific) small patches to check
>>> the device tree for the bindings on init, and if so set the port->ops
>>> methods according to the binding.
>> 
>> For me it appears to only solves a small part of the whole problem.
>> 
>> 1. it must be possible to start the UART before any user-space open()
> 
> There is no tty layer support at any level to do this, nor any reason
> I've seen advanced to justify all the extra complexity needed.

This is why I want to do it on the uart_port.

> 
>> 2. the UART must be kept running as long as the peer driver wants to
>> monitor rx data
> 
> Correct, otherwise in the general case the uart may be physically powered
> off and the kernel will in fact try aggressively to do so.
> 
>> 3. we need a hook to monitor: that (and which) data is incoming
> 
> That isn't tty_port related - but it's certainly hard.

Not, if we do it on uart_port level... It already works with two not very
big patches.

> 
>> Another aspect is that on a physical RS232 the DTR line is usually
>> used to power on/off a remote device (the DCE). Therefore I prefer
>> to mimic that in software by intercepting the mctrl changes. This
>> additionally allows a client to turn off power of the chip through a "standard"
>> protocol, even while the tty file is kept open.
> 
> The mctrl currently goes via tty->ops->tiocmset()/tiocmget. That IMHO is
> actually an oddity and moving it to the tty_port would make sense anyway
> because it's state and behaviour that is persistent at hardware level.
> 
> I don't know what Peter thinks about that however.
> 
>>> Nobody wants to lose the ability to do so or to move stuff around.
>> 
>> Indeed. But why would you loose this ability at all?
> 
> Because all your hooks would break.

Only if you do big changes. That is why I ask if such big changes are planned
or can be anticipated. They probably don't come unexpectedly around the corner.

And if you reject other changes like ours, there won't be any in the next 20 years :)

> 
>> If significant architectural changes are to be done, you can deprecate this
>> API and ask all users (driver authors) to replace it with something new.
>> And if they aren't adjusted within some time or no new interface is proposed,
>> they get removed. The *I* have the problem back on my table and not you :)
>> 
>> Isn't this a standard way to handle such situations?
>> 
>> Things are gradually changed every now and then. And other parts have to
>> follow. BTW: most of my work to update a production kernel to a new -rc1 is
>> to adjust non-mainline (or not-yet) drivers to such API changes.
> 
> What usually happens is the creator the tangle has despite best
> intentions ended up somewhere else on another project, their users whine
> if we break it, someone screams regression and it all gets dumped on the
> maintainers.

That is of course something we want to avoid. This is why I persistently try
to discuss it with you every detail. Fortunately a good discussion has now started.

And it looks that we either need a kernel solution or a user space solution - but
none makes the responsible party happy... Although that is not a criterion for
a whole system architecture. There the more efficient solution should be done.
Efficient in terms of LOC, speed, maintenance requirements.

BR & GN,
Nikolaus