Re: [PATCH v3 00/26] Enable H.264/H.265 encoder support and fixes in iris driver common code
From: Neil Armstrong
Date: Wed Aug 20 2025 - 11:15:24 EST
Hi,
On 20/08/2025 11:07, Dikshita Agarwal wrote:
Hi All,
This patch series adds support for H.264 and H.265 encoder in iris
driver and includes a few fixes and cleanup in the common code that were
identified during encoder bring-up process.
The changes include:
- Enabling support for H.264 and H.265 encoding.
- Fixes and improvements in shared componenets used by both encoder and
decoder paths.
- Ensuring compatibility and stability with the existing decoder flow.
Changes in v3:
- Fixed the log when destroying the interanl buffers (Jorge)
- Updated commit text with issue details in patch 05/25 (Krzysztof)
- Added a patch to simplify conditional logic in stop handling for hfi gen1 (Bryan)
- Reduced duplicate code while registering video devices (Bryan)
- Added a fix for try fmt handling in decoder (Self)
- Fixed the value of max core mbps for qcs8300 (Vikash)
- Simplied the frame rate handling in driver by using non q16 format and
converted to q16 when setting to firmware (Vikash)
- Fixed the issue with bitstream resolution setting to firmware (Neil)
- Addressed other review comments (Vikash, Bryan)
- Link to v2: https://lore.kernel.org/r/20250813-iris-video-encoder-v2-0-c725ff673078@xxxxxxxxxxx
Changes in v2:
- Fixed sparse/coccinnelle issues.
- Fixed the kernel doc warning.
- Removed unsupported PEAK_BITRATE property from SM8250.
- Dropped patch 04/25 to fix quality issue with encoder.
- Enhanced command handling for encoder to allow start/stop commands.
- Expanded rate control condition checks to include additional rate
control types for HFI Gen2.
- Updated default value to MAX_QP for all caps related to max QP settings.
- Add support for INPUT/OUTPUT_BUF_HOST_MAX_COUNT caps for encoder.
- Link to v1: https://lore.kernel.org/r/20250704-iris-video-encoder-v1-0-b6ce24e273cf@xxxxxxxxxxx
All patches have been tested with v4l2-compliance, v4l2-ctl and
Gstreamer on SM8250 and SM8550 for encoder, at the same time ensured
that the existing decoder functionality remains uneffected.
Commands used for V4l2-ctl validation:
v4l2-ctl --verbose --set-fmt-video-out=width=1280,height=720,pixelformat=NV12
--set-selection-output target=crop,top=0,left=0,width=1280,height=720
--set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap
--stream-from=/media/cyclists_1280x720_92frames.yuv
--stream-to=/tmp/cyclists_1280x720_92frames.h264 -d /dev/video1
v4l2-ctl --verbose --set-fmt-video-out=width=1280,height=720,pixelformat=NV12
--set-selection-output target=crop,top=0,left=0,width=1280,height=720
--set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap
--stream-from=/media/cyclists_1280x720_92frames.yuv
--stream-to=/tmp/cyclists_1280x720_92frames.hevc -d /dev/video1
- I've tested this serie on SM8650, first I failed to get HEVC working with the following command:
v4l2-ctl --verbose --set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 --set-selection-output target=crop,top=0,left=0,width=1920,height=1080 --set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap --stream-from=Big_Buck_Bunny_1080_10s.yuv --stream-to=Big_Buck_Bunny_1080_10s.hevc -d /dev/video1
I got:
qcom-iris aa00000.video-codec: session error received 0x1000005: unknown
I checked the diff with v2, and reverting this make HEVC encoding work again:
===========================><=================================================
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
@@ -194,9 +194,8 @@ static int iris_hfi_gen2_set_bitstream_resolution(struct iris_inst *inst, u32 pl
inst_hfi_gen2->src_subcr_params.bitstream_resolution = resolution;
payload_type = HFI_PAYLOAD_U32;
} else {
- codec_align = inst->codec == V4L2_PIX_FMT_HEVC ? 32 : 16;
- resolution = ALIGN(inst->fmt_dst->fmt.pix_mp.width, codec_align) << 16 |
- ALIGN(inst->fmt_dst->fmt.pix_mp.height, codec_align);
+ resolution = inst->fmt_dst->fmt.pix_mp.width << 16 |
+ inst->fmt_dst->fmt.pix_mp.height;
inst_hfi_gen2->dst_subcr_params.bitstream_resolution = resolution;
payload_type = HFI_PAYLOAD_32_PACKED;
}
===========================><=================================================
Did I pass some wrong parameters ? I don't expect the fw to crash.
- On the 1280x720 output bug, I tried the same commands as v2:
v4l2-ctl --verbose --set-fmt-video-out=width=1920,height=1080,pixelformat=NV12 --set-selection-output target=crop,top=0,left=0,width=1920,height=1080 --set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap --stream-from=Big_Buck_Bunny_1080_10s.yuv --stream-to=Big_Buck_Bunny_1080_10s.h264 -d /dev/video1
And I still get a 1280x720 h264 video, did I miss something ?
- Finally I tested the firmware you sent today (https://gitlab.com/kernel-firmware/linux-firmware/-/merge_requests/654),
first thanks a lot for that !
But then I was unable to encode with this firmware (v2 or v3 patchset), I got around all the firmwares
I got and here's the results:
| QC_IMAGE_VERSION_STRING | BUILD_DATE | Release | decoding | encoding v2 | encoding v3 |
|-------------------------------------------------------------|-------------|---------------------------------|----------|-------------|-------------|
| video-firmware.3.4-245082a0a3cc5e740f6340c295000ab4bcfc367d | Aug 24 2023 | VIDEO.LA.4.0.r2-02400-lanai.0-1 | OK | OK | KO HEVC |
| video-firmware.3.4-457429862ac40592d143de942b04d80fd9987e56 | Jan 29 2024 | VIDEO.LA.4.0.r2-03800-lanai.0-1 | OK | OK | KO HEVC |
| video-firmware.3.4-0d8a914b010672616a0d0613e22866c0b639a807 | Aug 14 2024 | VIDEO.LA.4.0.r2-05900-lanai.0-1 | OK | KO | KO |
| video-firmware.3.4-75ad4eb9657cf50ee663b05f78c01d6ceccd5632 | Jan 29 2025 | VIDEO.LA.4.0.r2-05900-lanai.0-1 | OK | KO | KO |
| video-firmware.3.4-e299f99ffcd086b43a2ccc7c3279ce5df404d693 | Aug 14 2025 | VIDEO.VPU.3.4-0054 | OK | KO | KO |
------------------------------------------------------------------------------------------------------------------------------------------------------
The change occurs around Aug 14 2024, So I checked the downstream driver and I found that fixes the encoding:
===========================><=================================================
--- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
+++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c
@@ -863,9 +863,18 @@ static inline
u32 size_vpss_line_buf(u32 num_vpp_pipes_enc, u32 frame_height_coded,
u32 frame_width_coded)
{
- return ALIGN(((((((8192) >> 2) << 5) * (num_vpp_pipes_enc)) + 64) +
- (((((max_t(u32, (frame_width_coded),
- (frame_height_coded)) + 3) >> 2) << 5) + 256) * 16)), 256);
+ u32 vpss_4tap_top = 0, vpss_4tap_left = 0, vpss_div2_top = 0, vpss_div2_left = 0, vpss_top_lb = 0, vpss_left_lb = 0, size_left = 0, size_top = 0;
+
+ vpss_4tap_top = ((((max_t(u32, frame_width_coded, frame_height_coded) * 2) + 3) >> 2) << 4) + 256;
+ vpss_4tap_left = (((8192 + 3) >> 2) << 5) + 64;
+ vpss_div2_top = (((max_t(u32,frame_width_coded, frame_height_coded) + 3) >> 2) << 4) + 256;
+ vpss_div2_left = ((((max_t(u32, frame_width_coded, frame_height_coded)* 2) + 3) >> 2) << 5) + 64;
+ vpss_top_lb = (frame_width_coded + 1) << 3;
+ vpss_left_lb = (frame_height_coded << 3) * num_vpp_pipes_enc;
+ size_left = (vpss_4tap_left + vpss_div2_left) * 2 * num_vpp_pipes_enc;
+ size_top = (vpss_4tap_top + vpss_div2_top) * 2;
+
+ return ALIGN(size_left + size_top + vpss_top_lb + vpss_left_lb, DMA_ALIGNMENT);
}
static inline
===========================><=================================================
And I checked and encoding still works with the "old" firmwares.
With both changes, I can get H264 & HEVC encoding working on any firmware.
Neil
Commands used for GST validation:
gst-launch-1.0 -v filesrc location=/media/cyclists_1280x720_92frames.yuv !
rawvideoparse format=nv12 width=1280 height=720 framerate=30/1 ! v4l2h264enc
capture-io-mode=4 output-io-mode=4 ! filesink sync=true
location=/tmp/gst_cyclists_1280x720_92frames.h264
gst-launch-1.0 -v filesrc location=/media/cyclists_1280x720_92frames.yuv !
rawvideoparse format=nv12 width=1280 height=720 framerate=30/1 ! v4l2h265enc
capture-io-mode=4 output-io-mode=4 ! filesink sync=true
location=/tmp/gst_cyclists_1280x720_92frames.hevc
The result of v4l2-compliance on SM8550:
v4l2-compliance 1.29.0-5270, 64 bits, 64-bit time_t
v4l2-compliance SHA: dc947661089e 2024-11-11 10:25:38
Compliance test for iris_driver device /dev/video1:
Driver Info:
Driver name : iris_driver
Card type : Iris Encoder
Bus info : platform:aa00000.video-codec
Driver version : 6.16.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Detected Stateful Encoder
Required ioctls:
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video1 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 37 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
test blocking wait: OK
Test input 0:
Streaming ioctls:
test read/write: OK (Not Supported)
Video Capture Multiplanar: Captured 61 buffers
test MMAP (select, REQBUFS): OK
Video Capture Multiplanar: Captured 61 buffers
test MMAP (epoll, REQBUFS): OK
Video Capture Multiplanar: Captured 61 buffers
test MMAP (select, CREATE_BUFS): OK
Video Capture Multiplanar: Captured 61 buffers
test MMAP (epoll, CREATE_BUFS): OK
test USERPTR (select): OK (Not Supported)
test DMABUF: Cannot test, specify --expbuf-device
Total for iris_driver device /dev/video1: 54, Succeeded: 54, Failed: 0, Warnings: 0
The result of v4l2-compliance on SM8250:
v4l2-compliance 1.29.0-5270, 64 bits, 64-bit time_t
v4l2-compliance SHA: dc947661089e 2024-11-11 10:25:38
Compliance test for iris_driver device /dev/video1:
Driver Info:
Driver name : iris_driver
Card type : Iris Encoder
Bus info : platform:aa00000.video-codec
Driver version : 6.16.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Detected Stateful Encoder
Required ioctls:
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video1 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 19 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
test blocking wait: OK
Test input 0:
Streaming ioctls:
test read/write: OK (Not Supported)
Video Capture Multiplanar: Captured 61 buffers
test MMAP (select, REQBUFS): OK
Video Capture Multiplanar: Captured 61 buffers
test MMAP (epoll, REQBUFS): OK
Video Capture Multiplanar: Captured 61 buffers
test MMAP (select, CREATE_BUFS): OK
Video Capture Multiplanar: Captured 61 buffers
test MMAP (epoll, CREATE_BUFS): OK
test USERPTR (select): OK (Not Supported)
test DMABUF: Cannot test, specify --expbuf-device
Total for iris_driver device /dev/video1: 54, Succeeded: 54, Failed: 0, Warnings: 0
Looking forward to your review and feedback.
Thanks,
Dikshita
Signed-off-by: Dikshita Agarwal <quic_dikshita@xxxxxxxxxxx>
---
Dikshita Agarwal (26):
media: iris: Fix buffer count reporting in internal buffer check
media: iris: Report unreleased PERSIST buffers on session close
media: iris: Fix memory leak by freeing untracked persist buffer
media: iris: Fix port streaming handling
media: iris: Allow substate transition to load resources during output streaming
media: iris: Always destroy internal buffers on firmware release response
media: iris: Update vbuf flags before v4l2_m2m_buf_done
media: iris: Simplify session stop logic by relying on vb2 checks
media: iris: Allow stop on firmware only if start was issued.
media: iris: Send dummy buffer address for all codecs during drain
media: iris: Fix missing LAST flag handling during drain
media: iris: Fix format check for CAPTURE plane in try_fmt
media: iris: Add support for video encoder device
media: iris: Initialize and deinitialize encoder instance structure
media: iris: Add support for ENUM_FMT, S/G/TRY_FMT encoder
media: iris: Add support for ENUM_FRAMESIZES/FRAMEINTERVALS for encoder
media: iris: Add support for VIDIOC_QUERYCAP for encoder video device
media: iris: Add encoder support for V4L2 event subscription
media: iris: Add support for G/S_SELECTION for encoder video device
media: iris: Add support for G/S_PARM for encoder video device
media: iris: Add platform-specific capabilities for encoder video device
media: iris: Add V4L2 streaming support for encoder video device
media: iris: Set platform capabilities to firmware for encoder video device
media: iris: Allocate and queue internal buffers for encoder video device
media: iris: Add support for buffer management ioctls for encoder device
media: iris: Add support for drain sequence in encoder video device
drivers/media/platform/qcom/iris/Makefile | 5 +-
drivers/media/platform/qcom/iris/iris_buffer.c | 220 ++++--
drivers/media/platform/qcom/iris/iris_buffer.h | 7 +-
drivers/media/platform/qcom/iris/iris_common.c | 232 ++++++
drivers/media/platform/qcom/iris/iris_common.h | 18 +
drivers/media/platform/qcom/iris/iris_core.h | 20 +-
drivers/media/platform/qcom/iris/iris_ctrls.c | 675 +++++++++++++++-
drivers/media/platform/qcom/iris/iris_ctrls.h | 15 +
drivers/media/platform/qcom/iris/iris_hfi_common.h | 2 +-
.../platform/qcom/iris/iris_hfi_gen1_command.c | 480 +++++++++---
.../platform/qcom/iris/iris_hfi_gen1_defines.h | 112 ++-
.../platform/qcom/iris/iris_hfi_gen1_response.c | 60 +-
.../platform/qcom/iris/iris_hfi_gen2_command.c | 359 ++++++---
.../platform/qcom/iris/iris_hfi_gen2_defines.h | 44 +-
.../platform/qcom/iris/iris_hfi_gen2_response.c | 46 +-
drivers/media/platform/qcom/iris/iris_instance.h | 24 +
.../platform/qcom/iris/iris_platform_common.h | 74 +-
.../media/platform/qcom/iris/iris_platform_gen2.c | 522 ++++++++++++-
.../platform/qcom/iris/iris_platform_qcs8300.h | 352 ++++++++-
.../platform/qcom/iris/iris_platform_sm8250.c | 234 +++++-
drivers/media/platform/qcom/iris/iris_probe.c | 33 +-
drivers/media/platform/qcom/iris/iris_state.c | 9 +-
drivers/media/platform/qcom/iris/iris_state.h | 1 +
drivers/media/platform/qcom/iris/iris_utils.c | 36 +
drivers/media/platform/qcom/iris/iris_utils.h | 2 +
drivers/media/platform/qcom/iris/iris_vb2.c | 58 +-
drivers/media/platform/qcom/iris/iris_vdec.c | 251 +-----
drivers/media/platform/qcom/iris/iris_vdec.h | 13 +-
drivers/media/platform/qcom/iris/iris_venc.c | 579 ++++++++++++++
drivers/media/platform/qcom/iris/iris_venc.h | 27 +
drivers/media/platform/qcom/iris/iris_vidc.c | 299 +++++++-
drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 847 ++++++++++++++++++++-
drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 20 +
33 files changed, 4964 insertions(+), 712 deletions(-)
---
base-commit: a75b8d198c55e9eb5feb6f6e155496305caba2dc
change-id: 20250704-iris-video-encoder-b193350b487a
Best regards,