[PATCH v4 5/8] drm/rockchip: vop2: Only accept AFBC YTR for 8 bit RGB on RK3588

From: Arun Iyengar via B4 Relay

Date: Thu Nov 19 2026 - 00:22:04 EST


From: Arun Iyengar <aiyengar1990@xxxxxxxxx>

According to the TRM, the AFBC decoder of the RK3588 Cluster windows
supports the lossless colour transform (YTR) only for the 8 bit RGB
formats. An RGB565 buffer with YTR gives a black picture and
POST_BUF_EMPTY, while the same buffer without YTR displays fine.

Don't accept YTR for any other format on RK3588.

Fixes: 5a028e8f062f ("drm/rockchip: vop2: Add support for rk3588")
Signed-off-by: Arun Iyengar <aiyengar1990@xxxxxxxxx>
---
drivers/gpu/drm/rockchip/rockchip_drm_vop2.c | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_vop2.c b/drivers/gpu/drm/rockchip/rockchip_drm_vop2.c
index e8ef83d6924b..1408b539c311 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_vop2.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_vop2.c
@@ -396,6 +396,15 @@ static bool rockchip_vop2_mod_supported(struct drm_plane *plane, u32 format,
return false;
}

+ /* AFBC YTR on 3588 is for 8bpc RGB only */
+ if (vop2->version == VOP_VERSION_RK3588 && drm_is_afbc(modifier) &&
+ (modifier & AFBC_FORMAT_MOD_YTR) &&
+ vop2_convert_afbc_format(format) != VOP2_AFBC_FMT_ARGB8888 &&
+ vop2_convert_afbc_format(format) != VOP2_AFBC_FMT_RGB888) {
+ drm_dbg_kms(vop2->drm, "Only support YTR with 8bpc RGB formats\n");
+ return false;
+ }
+
/* Linear is otherwise supported everywhere */
if (modifier == DRM_FORMAT_MOD_LINEAR)
return true;

--
2.55.0