[PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Jai Luthra]

The previous frame interval formula used analogue crop dimensions. This
breaks down for some sensors when binning.

For example in imx219 the minimum FLL (frame length in lines) can be
lower than the analogue crop height when binning, which would require a
negative VBLANK to represent the actual timing. Similarly, imx283 allows
a lower minimum HMAX (line length) when doing 2x2 or 3x3 binning than
the analogue crop width of the full resolution mode.

The CCS specification also describes under section "8.2.6 Line Length
and Frame Length" how the horizontal and vertical readout minimums can
be different when binning.

Replace the formula with the underlying hardware concepts of LLP (line
length in pixels) and FLL (frame length in lines). These terms were
chosen to match the CCS specification on raw sensors, as it is a cleaner
reference compared to a typical sensor vendor datasheet.

Finally, define the blanking controls relative to the active pixel
readout (post-binning) rather than the analogue crop size. This matches
what most sensor drivers already do, and also what applications like
libcamera expect. In "Figure 42" of CCS specification too, we see a
similar definition:

  frame interval = (output width + HBLANK) *
                   (output height + VBLANK) / pixel rate

Also add a note in the "Writing camera sensor drivers" guide, to ensure
this formula is followed by new sensor drivers.

Signed-off-by: Jai Luthra <jai.luthra@xxxxxxxxxxxxxxxx>
---
 Documentation/driver-api/media/camera-sensor.rst   | 11 ++++
 .../userspace-api/media/drivers/camera-sensor.rst  | 59 +++++++++++++++-------
 2 files changed, 53 insertions(+), 17 deletions(-)

diff --git a/Documentation/driver-api/media/camera-sensor.rst b/Documentation/driver-api/media/camera-sensor.rst
index 94bd1dae82d5c570b2d11c7faee20dd45d2f4be6..8dcac7551f54ac4ffa71173281ae3bbea331c036 100644
--- a/Documentation/driver-api/media/camera-sensor.rst
+++ b/Documentation/driver-api/media/camera-sensor.rst
@@ -120,6 +120,17 @@ The function returns a non-zero value if it succeeded getting the power count or
 runtime PM was disabled, in either of which cases the driver may proceed to
 access the device.
 
+Frame interval
+--------------
+
+If a sensor supports cropping or binning, it is the sensor driver's
+responsibility to ensure that the frame interval formula (see
+:ref:`media_using_camera_sensor_drivers`) remains valid regardless of the
+pipeline configuration. The driver shall adjust the minimum and maximum allowed
+values of ``V4L2_CID_HBLANK`` and ``V4L2_CID_VBLANK`` as needed when the mode
+changes, so that application developers can always rely on the same formula to
+calculate the frame interval.
+
 Rotation, orientation and flipping
 ----------------------------------
 
diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index 75fd9166383fdbb2dabdb6384ed0904c4e78a3c6..30dddea72a12da264fc9c30e37b561c762c09d29 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -49,35 +49,60 @@ depends on the type of the device.
 Raw camera sensors
 ~~~~~~~~~~~~~~~~~~
 
-Instead of a high level parameter such as frame interval, the frame interval is
-a result of the configuration of a number of camera sensor implementation
-specific parameters. Luckily, these parameters tend to be the same for more or
-less all modern raw camera sensors.
+Instead of a high level parameter such as frame interval, the frame interval on
+a raw camera sensor is determined by a number of sensor-specific parameters.
+These parameters tend to be common across most modern raw camera sensors.
 
-The frame interval is calculated using the following equation::
+The pixel array is the full grid of photosensitive elements on the sensor. A
+subregion of it is selected by the analogue crop. The cropped image may then be
+subject to binning (averaging of a NxN block) or subsampling which
+further reduce the image dimensions. The resulting image is then read out by
+the ADC (analogue-to-digital converter) line by line. After ADC readout,
+optional digital crop or scaling may further reduce the image dimensions, see
+:ref:`VIDIOC_SUBDEV_G_SELECTION <VIDIOC_SUBDEV_G_SELECTION>`.
 
-	frame interval = (analogue crop width + horizontal blanking) *
-			 (analogue crop height + vertical blanking) / pixel rate
+The frame size is determined by two timing parameters: line length in pixels
+(LLP) and frame length in lines (FLL). These are fundamental sensor timing
+registers that control how fast the ADC reads out the image. They may go
+by different names for a particular sensor, like HMAX and VMAX, or HTOTAL and
+VTOTAL, or similar.
 
-The formula is bus independent and is applicable for raw timing parameters on
-large variety of devices beyond camera sensors. Devices that have no analogue
-crop, use the full source image size, i.e. pixel array size.
+LLP is the total number of pixel clock cycles per line, including both the
+active readout width and horizontal blanking. FLL is the total number of lines
+per frame, including both the active readout height and vertical blanking.
+
+The frame interval is::
+
+        frame interval = LLP * FLL / pixel rate
 
 Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
 ``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
 is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
-the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
-sub-device. The unit of that control is pixels per second.
+the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the
+same sub-device. The unit of that control is pixels per second.
+
+The blanking is defined relative to the size of the image being sent out to the
+host over the bus (like CSI-2)::
+
+        LLP = active width + V4L2_CID_HBLANK
+        FLL = active height + V4L2_CID_VBLANK
+
+The driver shall set the minimum and maximum values of ``V4L2_CID_HBLANK`` and
+``V4L2_CID_VBLANK`` such that the resulting LLP and FLL values correspond to the
+range permitted by the sensor hardware for the current mode. Sensors that
+support binning often define a lower minimum for LLP or FLL registers, which
+can help achieve higher framerates when binning.
+
+Application developers can calculate the frame interval using the output
+dimensions and the blanking controls::
+
+        frame interval = (output width + horizontal blanking) *
+                         (output height + vertical blanking) / pixel rate
 
 Register list-based drivers need to implement read-only sub-device nodes for the
 purpose. Devices that are not register list based need these to configure the
 device's internal processing pipeline.
 
-The first entity in the linear pipeline is the pixel array. The pixel array may
-be followed by other entities that are there to allow configuring binning,
-skipping, scaling or digital crop, see :ref:`VIDIOC_SUBDEV_G_SELECTION
-<VIDIOC_SUBDEV_G_SELECTION>`.
-
 USB cameras etc. devices
 ~~~~~~~~~~~~~~~~~~~~~~~~
 

---
base-commit: 956b9cbd7f156c8672dac94a00de3c6a0939c692
change-id: 20260219-media-fps-docs-fd1da722cc38

Best regards,
-- 
Jai Luthra <jai.luthra@xxxxxxxxxxxxxxxx>