[PATCH] venus: vdec: update output buffer size during vdec_s_fmt()

From: Mansur Alisha Shaik
Date: Wed Aug 25 2021 - 04:45:03 EST

Video driver maintains an internal context for the output buffer size.
During S_fmt() on capture plane, the output buffer size is not updated
in driver context. As a result, during buf_prepare(), the size of the
vb2_plane and internal size of the buffer, as maintained by the driver,
does not match. This leads to buf_prepare() failure.
Update the instance context for the output buffer size during s_fmt().

Signed-off-by: Mansur Alisha Shaik <mansur@xxxxxxxxxxxxxx>
drivers/media/platform/qcom/venus/vdec.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 198e47eb63f4..c129b061a325 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -332,8 +332,11 @@ static int vdec_s_fmt(struct file *file, void *fh, struct v4l2_format *f)

inst->fmt_out = fmt;
- else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
inst->fmt_cap = fmt;
+ inst->output2_buf_size =
+ venus_helper_get_framesz(pixfmt_cap, orig_pixmp.width, orig_pixmp.height);
+ }

return 0;
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation