Re: [PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Jai Luthra]

Hi Jacopo,

Thanks for the review.

Quoting Jacopo Mondi (2026-02-23 14:41:02)
> Hi Jai
> 
> On Thu, Feb 19, 2026 at 01:20:50PM +0530, Jai Luthra wrote:
> > The previous frame interval formula used analogue crop dimensions. This
> > breaks down for some sensors when binning.
> >
> > For example in imx219 the minimum FLL (frame length in lines) can be
> > lower than the analogue crop height when binning, which would require a
> > negative VBLANK to represent the actual timing. Similarly, imx283 allows
> > a lower minimum HMAX (line length) when doing 2x2 or 3x3 binning than
> > the analogue crop width of the full resolution mode.
> >
> > The CCS specification also describes under section "8.2.6 Line Length
> > and Frame Length" how the horizontal and vertical readout minimums can
> > be different when binning.
> >
> > Replace the formula with the underlying hardware concepts of LLP (line
> > length in pixels) and FLL (frame length in lines). These terms were
> > chosen to match the CCS specification on raw sensors, as it is a cleaner
> > reference compared to a typical sensor vendor datasheet.
> >
> > Finally, define the blanking controls relative to the active pixel
> > readout (post-binning) rather than the analogue crop size. This matches
> > what most sensor drivers already do, and also what applications like
> > libcamera expect. In "Figure 42" of CCS specification too, we see a
> > similar definition:
> >
> >   frame interval = (output width + HBLANK) *
> >                    (output height + VBLANK) / pixel rate
> >
> > Also add a note in the "Writing camera sensor drivers" guide, to ensure
> > this formula is followed by new sensor drivers.
> 
> I agree that using the analogue crop rectangle sizes is not correct,
> however, with this new formulation the LLP and FLL values might be
> smaller than the crop rectangles reported through the selection API,
> which I'm not sure it's great from a consistency point of view ?

While it takes a while to grok, but most sensors I looked at operate in
this way (of reducing the line length or the frame length) when binning,
including CCS (implicitly). A counterexample would really help here.

> 
> Also the below suggested formulation:
> 
>         LLP = active width + V4L2_CID_HBLANK
>         FLL = active height + V4L2_CID_VBLANK
> 
> Assumes the combined effect of [binning + subsampling] always directly
> affect the readout time of pixels on the pixel array. The CCS specs
> and driver seems to suggest that's not always the case ?

Why do you think this model can't work for sensors where the readout time is
unaffected?

Let's say a sensor's pixel array is 1920x1080 with minimum LLP=2000, FLL=1200.

so, HBLANK minimum = 80, VBLANK minimum = 120

It also supports a 2x2 binned mode of 960x540 with the same minimum LLP and FLL
(that is same max framerate, no speed up or change in readout).

so, HBLANK minimum = 2000 - 960 = 1040, VBLANK minimum = 1200 - 540 = 660

If you match this with the CCS Figure 42 it still makes sense.

> 
> Do we need to distinguish between binning modes that affect the
> timings and binning modes that do not do that?
> 
> As we're going to introduce a control for binning to report the
> binning factor in the image dimension domain, should we introduce
> a control to specify the binning factor in the image timing domain ?
> 
>         LLP = (analog_crop_width + HBLANK) / binning_timing_h
>         FLL = (analog_crop_height + VBLANK) / binning_timing_v
>         frame_interval = LLP * FLL / pixel_rate
> 
> I'm not 100% sure this is correct however, as the blankings should be
> expressed on a different clock domain that the pixel sampling rate,
> but I guess this is a reasonable approximation ?

What does these two extra controls really offer us?

All sensors we have seen thus far map their LLP/FLL (or equivalent HTOT/VTOT)
values with respect to the digital readout, and not the analogue pixel array.

If we add these two controls, we will have to support two different models for
frame interval calculation in the application layer too. Which I'm fine with if
it has a practical benefit, that is, it makes it easier to deal with some
particular sensor.

And if exposing the LLP/FLL directly offer the same benefit, then that is
cleaner, as it leaves the HBLANK/VBLANK as-is in the new model.

Thanks,
Jai

[...]