Re: [PATCH v4 1/4] gpu: ipu-v3: ipu-ic: Rename yuv2rgb encoding matrices

[Steve Longerbeam]

On 2/13/19 2:35 AM, Philipp Zabel wrote:
> On Tue, 2019-02-12 at 09:42 -0800, Steve Longerbeam wrote:
> [...]
> > > > But what about this "SAT_MODE" field in the IC task parameter memory?
> > > That just controls the saturation. The result after the matrix
> > > multiplication is either saturated to [0..255] or to [16..235]/[16..240]
> > > when converting from the internal representation to the 8 bit output.
> > By saturation I think you mean clipped to those ranges?
> Yes, thanks. I didn't realize it sounds weird to use saturated this way.
> See:https://en.wikipedia.org/wiki/Saturation_arithmetic

Ok, saturation can mean the same thing as clipping/clamping. Thanks for 
the article.

I tested a RGB->YUV pipeline with the .sat bit set in the BT.601 rgb2yuv 
table, with the following pipeline on the SabreSD:

'ov5640 1-003c':0
ÂÂÂ ÂÂÂ [fmt:RGB565_2X8_LE/1024x768@1/30 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]

'imx6-mipi-csi2':0
ÂÂÂ ÂÂÂ [fmt:RGB565_2X8_LE/1024x768 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]

'imx6-mipi-csi2':2
ÂÂÂ ÂÂÂ [fmt:RGB565_2X8_LE/1024x768 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]

'ipu1_csi1':0
ÂÂÂ ÂÂÂ [fmt:RGB565_2X8_LE/1024x768@1/30 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range
ÂÂÂ ÂÂÂ Âcrop.bounds:(0,0)/1024x768
ÂÂÂ ÂÂÂ Âcrop:(0,0)/1024x768
ÂÂÂ ÂÂÂ Âcompose.bounds:(0,0)/1024x768
ÂÂÂ ÂÂÂ Âcompose:(0,0)/1024x768]

'ipu1_csi1':1
ÂÂÂ ÂÂÂ [fmt:ARGB8888_1X32/1024x768@1/30 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]

'ipu1_ic_prp':0
ÂÂÂ ÂÂÂ [fmt:ARGB8888_1X32/1024x768@1/30 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]

'ipu1_ic_prp':1
ÂÂÂ ÂÂÂ [fmt:ARGB8888_1X32/1024x768@1/30 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]

'ipu1_ic_prpenc':0
ÂÂÂ ÂÂÂ [fmt:ARGB8888_1X32/1024x768@1/30 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:full-range]

'ipu1_ic_prpenc':1
ÂÂÂ ÂÂÂ [fmt:AYUV8_1X32/800x600@1/30 field:none colorspace:srgb 
xfer:srgb ycbcr:601 quantization:lim-range]

/dev/video0:0
Format Video Capture:
ÂÂÂ Width/HeightÂÂÂÂÂ : 800/600
ÂÂÂ Pixel FormatÂÂÂÂÂ : 'YV12'
ÂÂÂ FieldÂÂÂÂÂÂÂÂÂÂÂÂ : None
ÂÂÂ Bytes per LineÂÂÂ : 800
ÂÂÂ Size ImageÂÂÂÂÂÂÂ : 720000
ÂÂÂ ColorspaceÂÂÂÂÂÂÂ : sRGB
ÂÂÂ Transfer Function : sRGB
ÂÂÂ YCbCr/HSV Encoding: ITU-R 601
ÂÂÂ QuantizationÂÂÂÂÂ : Limited Range
ÂÂÂ FlagsÂÂÂÂÂÂÂÂÂÂÂÂ :


The result being that the captured image colors are all off (there's a 
bright pink shade to the images). But I discovered the init_csc() 
function was not setting the saturation bit at the correct bit offset 
within the TPMEM. The saturation bit is bit 42, or bit 10 of the second 
32-bit word. But the code was writing to bit 9 of the second word. After 
correcting this, saturation is working fine. I have added another patch 
that fixes this for v5 series.


> > > SAT_MODE should be set for conversions to YUV limited range so that the
> > > coefficients can be rounded to the closest value.
> > Well, we have already rounded the coefficients to the nearest int in the
> > tables. Do you mean the final result (coeff * color component + offset)
> > is rounded?
> The manual says so: "The final calculation result is limited according
> to the SAT_MODE parameter and rounded to 8 bits", but that's not what I
> meant. Still, I might have been mistaken.
>
> I think due to the fact that the coefficients are multiplied by up to
> 255 (max pixel value) and then effectively divided by 256 when
> converting to 8 bit, the only way to overflow limited range is if two
> coefficients are rounded away from zero in the calculation of a single
> component. This doesn't seem to happen in practice.
>
> A constructed example, conversion to YUV limited range with carefully
> chosen coefficients.
>
>    Y = R *Â.1817Â+ G *Â.6153 + B *Â.0618 + 16;
>
> Note that .1817 + .6153 + .0618 < 219/255.
> With rounded coefficients though:
>
>    Y = (R * 47 + G * 158 + B * 16 + (64 << 6)) / 256 = 236.136

Yes, for a rec.709 conversion and max/worst-case RGB signal = (255,255,255).

But the rec.709 coefficients for Y are actually

Y = (R * 47 + G * 157 + B * 16 + (16 << 8)) / 256


which for RGB = (255,255,255), Y = 235.14, which doesn't overflow 
limited range.

Steve