cmap problems with aty128fb.c

From: Paul Mackerras (paulus@samba.org)
Date: Wed May 08 2002 - 03:40:24 EST


I have been having difficulties with aty128fb.c in 2.5.14. The
version in Linus' tree crashes on boot for me with a null pointer
dereference in memcpy, called from fb_copy_cmap from gen_set_cmap,
from fbcon_show_logo via the fbops->fb_set_cmap pointer.

At the lowest level the null pointer dereference is caused by a
signed/unsigned bug in fb_copy_cmap. We are ending up with size > 0
even when to->start and to->len were both zero. The patch below fixes
that.

At the next level up, the problem seems to be that info->cmap is never
getting initialized. How and where is it supposed to be initialized?

Paul.

diff -urN linux-2.5/drivers/video/fbcmap.c pmac-2.5/drivers/video/fbcmap.c
--- linux-2.5/drivers/video/fbcmap.c Mon Apr 29 16:25:24 2002
+++ pmac-2.5/drivers/video/fbcmap.c Wed May 8 16:29:04 2002
@@ -150,9 +150,9 @@
     else
         tooff = from->start-to->start;
     size = to->len-tooff;
- if (size > from->len-fromoff)
+ if (size > (int)(from->len-fromoff))
         size = from->len-fromoff;
- if (size < 0)
+ if (size <= 0)
         return;
     size *= sizeof(u16);
     
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Tue May 14 2002 - 12:00:08 EST