Re: [PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Jacopo Mondi]

Hi Jai

On Tue, Feb 24, 2026 at 10:55:33AM +0530, Jai Luthra wrote:
> Hi Jacopo,
>
> Thanks for the review.
>
> Quoting Jacopo Mondi (2026-02-23 14:41:02)
> > Hi Jai
> >
> > On Thu, Feb 19, 2026 at 01:20:50PM +0530, Jai Luthra wrote:
> > > The previous frame interval formula used analogue crop dimensions. This
> > > breaks down for some sensors when binning.
> > >
> > > For example in imx219 the minimum FLL (frame length in lines) can be
> > > lower than the analogue crop height when binning, which would require a
> > > negative VBLANK to represent the actual timing. Similarly, imx283 allows
> > > a lower minimum HMAX (line length) when doing 2x2 or 3x3 binning than
> > > the analogue crop width of the full resolution mode.
> > >
> > > The CCS specification also describes under section "8.2.6 Line Length
> > > and Frame Length" how the horizontal and vertical readout minimums can
> > > be different when binning.
> > >
> > > Replace the formula with the underlying hardware concepts of LLP (line
> > > length in pixels) and FLL (frame length in lines). These terms were
> > > chosen to match the CCS specification on raw sensors, as it is a cleaner
> > > reference compared to a typical sensor vendor datasheet.
> > >
> > > Finally, define the blanking controls relative to the active pixel
> > > readout (post-binning) rather than the analogue crop size. This matches
> > > what most sensor drivers already do, and also what applications like
> > > libcamera expect. In "Figure 42" of CCS specification too, we see a
> > > similar definition:
> > >
> > >   frame interval = (output width + HBLANK) *
> > >                    (output height + VBLANK) / pixel rate
> > >
> > > Also add a note in the "Writing camera sensor drivers" guide, to ensure
> > > this formula is followed by new sensor drivers.
> >
> > I agree that using the analogue crop rectangle sizes is not correct,
> > however, with this new formulation the LLP and FLL values might be
> > smaller than the crop rectangles reported through the selection API,
> > which I'm not sure it's great from a consistency point of view ?
>
> While it takes a while to grok, but most sensors I looked at operate in
> this way (of reducing the line length or the frame length) when binning,
> including CCS (implicitly). A counterexample would really help here.
>
> >
> > Also the below suggested formulation:
> >
> >         LLP = active width + V4L2_CID_HBLANK
> >         FLL = active height + V4L2_CID_VBLANK
> >
> > Assumes the combined effect of [binning + subsampling] always directly
> > affect the readout time of pixels on the pixel array. The CCS specs
> > and driver seems to suggest that's not always the case ?
>
> Why do you think this model can't work for sensors where the readout time is
> unaffected?
>
> Let's say a sensor's pixel array is 1920x1080 with minimum LLP=2000, FLL=1200.
>
> so, HBLANK minimum = 80, VBLANK minimum = 120
>
> It also supports a 2x2 binned mode of 960x540 with the same minimum LLP and FLL
> (that is same max framerate, no speed up or change in readout).
>
> so, HBLANK minimum = 2000 - 960 = 1040, VBLANK minimum = 1200 - 540 = 660
>
> If you match this with the CCS Figure 42 it still makes sense.
>

Ah well, sure if LLP and FFL remain constant, the usage of the
"active sizes" doesn't matter.

This means that the min hblank and min vblank have to be doubled to
compensate for the halved the active sizes.

Speaking in CCS terms, this means the limits reported in Table 86 are
always populated.

And what I'm struggling with at the moment, is the assumptions that's
always be the case for all sensors. Unfortunately I don't have enough
experience across all vendors to tell if that's the case..


> >
> > Do we need to distinguish between binning modes that affect the
> > timings and binning modes that do not do that?
> >
> > As we're going to introduce a control for binning to report the
> > binning factor in the image dimension domain, should we introduce
> > a control to specify the binning factor in the image timing domain ?
> >
> >         LLP = (analog_crop_width + HBLANK) / binning_timing_h
> >         FLL = (analog_crop_height + VBLANK) / binning_timing_v
> >         frame_interval = LLP * FLL / pixel_rate
> >
> > I'm not 100% sure this is correct however, as the blankings should be
> > expressed on a different clock domain that the pixel sampling rate,
> > but I guess this is a reasonable approximation ?
>
> What does these two extra controls really offer us?
>
> All sensors we have seen thus far map their LLP/FLL (or equivalent HTOT/VTOT)
> values with respect to the digital readout, and not the analogue pixel array.
>
> If we add these two controls, we will have to support two different models for
> frame interval calculation in the application layer too. Which I'm fine with if
> it has a practical benefit, that is, it makes it easier to deal with some
> particular sensor.

Not sure why you would need two modes to calculate timings in
userspace.

>
> And if exposing the LLP/FLL directly offer the same benefit, then that is
> cleaner, as it leaves the HBLANK/VBLANK as-is in the new model.
>

As per above, if the consensus is that having the limits updated when
binning to maintaine the LLP/FFL values will work for all sensors,
then I'm fine with that.

> Thanks,
> Jai
>
> [...]