RE: Intel timed i/o driver in HTE

[Hall, Christopher S]

Hi Linus,

Thanks for your response.

On Saturday, December 03, 2022 1:50 AM Linus Walleij <linus.walleij@xxxxxxxxxx> wrote:
> To: N, Pandith <pandith.n@xxxxxxxxx>; Johan Hovold <johan@xxxxxxxxxx>

> Subject: Re: Intel timed i/o driver in HTE
> 
> On Fri, Dec 2, 2022 at 4:00 AM N, Pandith <pandith.n@xxxxxxxxx> wrote:
> 
> > > 1. Does timed io only meant for GPIO or other signals? if other signals, what
> > > type of signals?
> > This could be time sync signals or periodic/single pulse input for timestamp support
> (...)
> > b. char device creation for timestamping hardware. Something like /dev/hteX
> 
> No way this goes to userspace if the usecase is synchronizing GPS time.

The Timed I/O device is for timestamped input and output only. There are
currently two dedicated single-ended signals on recent Intel platforms.
It is not a GPS device. It does not belong in the GNSS subsystem. We use
that as an example application that you had asked for. We propose any
device that provides a PPS signal out can be used to set the system
clock using Timed I/O.

For PPS input, we plan to use the PPS subsystem.

At a high-level:

The application configures the pin for PPS input
A device in created /dev/ppsX
while 1:
	The timed I/O device captures / timestamps a pulse from an
		external PPS provider
	The timestamp is translated to system time(TN)
	A PPS event is generated using pps_event(TN)

Another application like PHC2SYS or Chrony consumes the timestamps from
the PPS device and disciplines the system clock.

Currently, most - maybe all - PPS clients (drivers/pps/clients) capture
timestamps in software (pps_get_ts()). We can do at least 50x better
timestamping using hardware. The timestamp accuracy is in the range of
a few 10s of nanoseconds. I think this is a good thing.

> It should be in the kernel, where the timebase of the system is.

It is.

> We already created drivers/gnss because power, clock etc management
> need to be close to the hardware, in the kernel. Don't try to push this
> up to userspace, group it with the rest of the kernel GPS handling.


> The GPS vendors have already created enough of a mess by not having
> open documentation for their hardware, if this is an incentive for them to
> be more open if they want proper time sync that is *good*.

Again we provided an example application. Let's not get "hung up" on GPS.
There are many GPS receivers that produce a PPS output. There is some
discussion of the interface here if there is doubt such a thing exists:

https://electronics.stackexchange.com/questions/30750/why-do-gps-receivers-have-a-1-pps-output

If I may, I would like to re-focus the discussion. The question we want
to answer in this thread is whether it makes sense to modify the HTE
subsystem to accommodate our device and whether it "belongs" there.

For some background, I proposed creating a new Timed IO device type at
LPC:

https://lpc.events/event/16/contributions/1387/

The feedback was - to paraphrase: "We cannot support a new device type
for a single vendor. Can you use the HTE subsystem?"

This is a reasonable question. HTE devices implement a similar
function - hardware timestamping - to our Timed IO device. It is the
only similar device time that I am aware of. However, large changes to
the subsystem would be required to support output, for example.

To summarize this and previous threads the Timed IO use case is
importing and exporting system time with about nanosecond precision.
We are able to do this because the clock we use to generate or
timestamp events using TSC which drives the system clock on Intel.
To see how TSC -> System Time conversion works on input:
	kernel/time/timekeeping.c:get_device_system_crosststamp()

More specific use cases are:

- GPS / PPS input to discipline the system clock
- 1 Hz to 1 kHz out signal synchronized with the system clock for
	diagnostic output. This is for safety critical systems where
	unsynchronized clocks are dangerous.

I have also used PPS output to measure synchronization between
devices connected to the platform:

https://drive.google.com/file/d/1OKAOjRzZHlaNoHE7USXpOyIBU0_1EytU/view

This measures PTM performance, but TGPIO can also be used to as a
substitute for PTM to synchronize the system clock with the NIC. The
second (PTM substitute) is similar to the PPS input use.

To support this, we need PPS In / Out functions and up to 1 kHz output.
These all must be aligned with the CLOCK_REALTIME.

To "future proof" our device we also want to use the periodic output
function of the Timed IO device. Using this, we can output up to about
4 MHz but rather than scheduling each output event as in the previous
use cases we start a signal that is aligned to the system clock, but
since the frequency is that of the TSC, not CLOCK_REALTIME, there is
drift and we use a PI controller to maintain zero offset with
CLOCK_REALTIME.

We also want to be able to timestamp / generate single events. 
Existing HTE already do the first.

Are these use cases that you are willing to support in the HTE
subsystem? We seem to be unable to dig into the implementation
without circling back to the use cases which I believe are
clearly defined.

Thanks,
Christopher Hall