[PATCH RFC 09/10] media: v4l2-hevc-enc: Re-encode oversized pictures

From: Arun Iyengar via B4 Relay

Date: Tue Nov 10 2026 - 06:14:24 EST


From: Arun Iyengar <aiyengar1990@xxxxxxxxx>

A scene change in a P picture can make it many times larger than its size
target, and the peak bitrate around it can then exceed what a decoder
buffer of the stream takes. v4l2-enc-cbr can ask for a second pass of such
a picture at a higher QP (v4l2_enc_cbr_reencode()).

Provide v4l2_hevc_enc_reencode() in the HEVC encode core for drivers whose
hardware writes the slice header: called after the hardware finished a
picture and before v4l2_hevc_enc_complete(), it returns true with the new
QP in enc->qp when the constant bitrate mode wants the picture encoded
again. Hardware that takes the slice header from the core cannot use it:
that header holds the QP of the first pass.

Signed-off-by: Arun Iyengar <aiyengar1990@xxxxxxxxx>
---
drivers/media/v4l2-core/v4l2-hevc-enc.c | 39 +++++++++++++++++++++++++++++++++
include/media/v4l2-hevc-enc.h | 2 ++
2 files changed, 41 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-hevc-enc.c b/drivers/media/v4l2-core/v4l2-hevc-enc.c
index 2e0ba832b868..d49cee336558 100644
--- a/drivers/media/v4l2-core/v4l2-hevc-enc.c
+++ b/drivers/media/v4l2-core/v4l2-hevc-enc.c
@@ -1369,6 +1369,45 @@ static int ref_complete(struct v4l2_hevc_enc *enc, u64 timestamp)
return 0;
}

+/**
+ * v4l2_hevc_enc_reencode() - decide on encoding the picture a second time
+ * @enc: the core context
+ * @bytesused: size of the encoded picture in bytes, with the headers the core
+ * wrote
+ *
+ * In the constant bitrate mode of the core, a picture far above its size
+ * target can be encoded again with a higher QP (see v4l2_enc_cbr_reencode()).
+ * Drivers that can run the hardware again for the same picture call it after
+ * the hardware finished it and before v4l2_hevc_enc_complete(). If it returns
+ * true, @enc->qp holds the new QP and the driver encodes the picture again,
+ * with the same size target and the same headers, writing the slice data
+ * from the same offset. Only for hardware that writes the slice header
+ * (V4L2_HEVC_ENC_FLAG_HW_SLICE_HEADER): the slice header of the core holds the
+ * QP of the first pass.
+ *
+ * Return: true if the picture is to be encoded again.
+ */
+bool v4l2_hevc_enc_reencode(struct v4l2_hevc_enc *enc, unsigned int bytesused)
+{
+ struct v4l2_hevc_enc_state *state = &enc->state_active;
+ bool intra = state->encode.slice_type == V4L2_HEVC_SLICE_TYPE_I;
+ int qp_max = intra ? state->qp_max_i : state->qp_max;
+
+ if (!(enc->flags & V4L2_HEVC_ENC_FLAG_HW_SLICE_HEADER) ||
+ !rc_core_cbr(enc, state))
+ return false;
+
+ if (!v4l2_enc_cbr_reencode(&enc->cbr, (u64)bytesused * 8,
+ &state->timeperframe, enc->qp,
+ min(qp_max, 51)))
+ return false;
+
+ enc->qp = rc_clamp_qp(state, v4l2_enc_cbr_qp(&enc->cbr));
+
+ return true;
+}
+EXPORT_SYMBOL_GPL(v4l2_hevc_enc_reencode);
+
/**
* v4l2_hevc_enc_complete() - complete the encode run of a picture
* @enc: the core context
diff --git a/include/media/v4l2-hevc-enc.h b/include/media/v4l2-hevc-enc.h
index f1d3da3269f1..5be449e5ec03 100644
--- a/include/media/v4l2-hevc-enc.h
+++ b/include/media/v4l2-hevc-enc.h
@@ -285,6 +285,8 @@ int v4l2_hevc_enc_init(struct v4l2_hevc_enc *enc);
void v4l2_hevc_enc_exit(struct v4l2_hevc_enc *enc);
int v4l2_hevc_enc_step(struct v4l2_hevc_enc *enc,
struct vb2_v4l2_buffer *buffer);
+bool v4l2_hevc_enc_reencode(struct v4l2_hevc_enc *enc,
+ unsigned int bytesused);
int v4l2_hevc_enc_complete(struct v4l2_hevc_enc *enc,
struct vb2_v4l2_buffer *buffer,
const struct v4l2_stateless_encode_stats *stats);

--
2.55.0