[PATCH 3.2 014/115] [media] bttv: Width must be a multiple of 16 when capturing planar formats

From: Ben Hutchings
Date: Tue Apr 26 2016 - 20:49:00 EST


3.2.80-rc1 review patch. If anyone has any objections, please let me know.

------------------

From: Hans de Goede <hdegoede@xxxxxxxxxx>

commit 5c915c68763889f0183a1cc61c84bb228b60124a upstream.

On my bttv card "Hauppauge WinTV [card=10]" capturing in YV12 fmt at max
size results in a solid green rectangle being captured (all colors 0 in
YUV).

This turns out to be caused by max-width (924) not being a multiple of 16.

We've likely never hit this problem before since normally xawtv / tvtime,
etc. will prefer packed pixel formats. But when using a video card which
is using xf86-video-modesetting + glamor, only planar XVideo fmts are
available, and xawtv will chose a matching capture format to avoid needing
to do conversion, triggering the solid green window problem.

Signed-off-by: Hans de Goede <hdegoede@xxxxxxxxxx>
Acked-by: Hans Verkuil <hans.verkuil@xxxxxxxxx>
Signed-off-by: Mauro Carvalho Chehab <mchehab@xxxxxxxxxxxxxxx>
[bwh: Backported to 3.2: adjust filename, context]
Signed-off-by: Ben Hutchings <ben@xxxxxxxxxxxxxxx>
---
drivers/media/video/bt8xx/bttv-driver.c | 26 ++++++++++++++++++++------
1 file changed, 20 insertions(+), 6 deletions(-)

--- a/drivers/media/video/bt8xx/bttv-driver.c
+++ b/drivers/media/video/bt8xx/bttv-driver.c
@@ -2480,6 +2480,19 @@ static int bttv_g_fmt_vid_overlay(struct
return 0;
}

+static void bttv_get_width_mask_vid_cap(const struct bttv_format *fmt,
+ unsigned int *width_mask,
+ unsigned int *width_bias)
+{
+ if (fmt->flags & FORMAT_FLAGS_PLANAR) {
+ *width_mask = ~15; /* width must be a multiple of 16 pixels */
+ *width_bias = 8; /* nearest */
+ } else {
+ *width_mask = ~3; /* width must be a multiple of 4 pixels */
+ *width_bias = 2; /* nearest */
+ }
+}
+
static int bttv_try_fmt_vid_cap(struct file *file, void *priv,
struct v4l2_format *f)
{
@@ -2488,6 +2501,7 @@ static int bttv_try_fmt_vid_cap(struct f
struct bttv *btv = fh->btv;
enum v4l2_field field;
__s32 width, height;
+ unsigned int width_mask, width_bias;
int rc;

fmt = format_by_fourcc(f->fmt.pix.pixelformat);
@@ -2525,9 +2539,9 @@ static int bttv_try_fmt_vid_cap(struct f
width = f->fmt.pix.width;
height = f->fmt.pix.height;

+ bttv_get_width_mask_vid_cap(fmt, &width_mask, &width_bias);
rc = limit_scaled_size_lock(fh, &width, &height, field,
- /* width_mask: 4 pixels */ ~3,
- /* width_bias: nearest */ 2,
+ width_mask, width_bias,
/* adjust_size */ 1,
/* adjust_crop */ 0);
if (0 != rc)
@@ -2558,6 +2572,7 @@ static int bttv_s_fmt_vid_cap(struct fil
struct bttv_fh *fh = priv;
struct bttv *btv = fh->btv;
__s32 width, height;
+ unsigned int width_mask, width_bias;
enum v4l2_field field;

retval = bttv_switch_type(fh, f->type);
@@ -2572,9 +2587,10 @@ static int bttv_s_fmt_vid_cap(struct fil
height = f->fmt.pix.height;
field = f->fmt.pix.field;

+ fmt = format_by_fourcc(f->fmt.pix.pixelformat);
+ bttv_get_width_mask_vid_cap(fmt, &width_mask, &width_bias);
retval = limit_scaled_size_lock(fh, &width, &height, f->fmt.pix.field,
- /* width_mask: 4 pixels */ ~3,
- /* width_bias: nearest */ 2,
+ width_mask, width_bias,
/* adjust_size */ 1,
/* adjust_crop */ 1);
if (0 != retval)
@@ -2582,8 +2598,6 @@ static int bttv_s_fmt_vid_cap(struct fil

f->fmt.pix.field = field;

- fmt = format_by_fourcc(f->fmt.pix.pixelformat);
-
/* update our state informations */
fh->fmt = fmt;
fh->cap.field = f->fmt.pix.field;