(2.0 bug?) ELF truncated core dump solved - binfmt_elf.c patch

Peter Desnoyers (pjd@midnight.com)
Tue, 11 Jun 1996 17:26:32 -0400

The following code from binfmt_elf.c/maydump causes 1.3.85 to generate
core files under some circumstances (i.e. the product I'm trying to
develop :-() that GDB 4.14 cannot read, giving a truncated core file
error:

#if 1
if (vma->vm_flags & (VM_WRITE|VM_GROWSUP|VM_GROWSDOWN))
return 1;
if (vma->vm_flags & (VM_READ|VM_EXEC|VM_EXECUTABLE|VM_SHARED))
return 0;
#endif

If I change the '#if 1' to '#if 0' I get a core file almost 3 times as
big, and GDB is perfectly happy.

The bug only appears on a large (300k text, 100k bss, data),
statically linked (libc 5.2.18) executable that I am developing. A
tiny test program (dynamic libc 5.0.9) didn't cause the same problem.

I have only tested this fix on 1.3.85; however the particular code in
binfmt_elf.c is unchanged in 2.0, so I assume the bug persists there.

I've enclosed debug kernel printouts of coredumps before and after the
change below for the terminally curious.

-- 


...............................................................................
  Peter Desnoyers  : Midnight Networks Inc. 200 Fifth Avenue Waltham MA 02154 
  pjd@midnight.com : Ph. 617/890-1001 Fax -0028  The Best in Network Software


This is with the #if'ed code left in:


Jun 11 16:37:49 aout kernel: elf_core_dump: 3 segs taking 212992 bytes
Jun 11 16:37:50 aout kernel: Registers: Passed in regs
Jun 11 16:37:50 aout kernel:    0  ebx  =080aabf4 134917108
Jun 11 16:37:50 aout kernel:    1  ecx  =080ac6b4 134923956
Jun 11 16:37:50 aout kernel:    2  edx  =08049010 134516752
Jun 11 16:37:50 aout kernel:    3  esi  =bffff708 3221223176
Jun 11 16:37:50 aout kernel:    4  edi  =08049010 134516752
Jun 11 16:37:50 aout kernel:    5  ebp  =bffff68c 3221223052
Jun 11 16:37:50 aout kernel:    6  eax  =00000000 0
Jun 11 16:37:50 aout kernel:    7  ds   =0000002b 43
Jun 11 16:37:50 aout kernel:    8  es   =0000002b 43
Jun 11 16:37:50 aout kernel:    9  fs   =0000002b 43
Jun 11 16:37:50 aout kernel:    10 gs   =0000002b 43
Jun 11 16:37:50 aout kernel:    11 orig_eax=ffffffff 4294967295
Jun 11 16:37:50 aout kernel:    12 eip  =0804903a 134516794
Jun 11 16:37:50 aout kernel:    13 cs   =00000023 35
Jun 11 16:37:50 aout kernel:    14 efl  =00010282 66178
Jun 11 16:37:50 aout kernel:    15 uesp =bffff24c 3221221964
Jun 11 16:37:50 aout kernel:    16 ss   =0000002b 43
Jun 11 16:37:50 aout kernel: Registers: prstatus regs
Jun 11 16:37:50 aout kernel:    0  ebx  =080aabf4 134917108
Jun 11 16:37:50 aout kernel:    1  ecx  =080ac6b4 134923956
Jun 11 16:37:50 aout kernel:    2  edx  =08049010 134516752
Jun 11 16:37:50 aout kernel:    3  esi  =bffff708 3221223176
Jun 11 16:37:50 aout kernel:    4  edi  =08049010 134516752
Jun 11 16:37:50 aout kernel:    5  ebp  =bffff68c 3221223052
Jun 11 16:37:50 aout kernel:    6  eax  =00000000 0
Jun 11 16:37:50 aout kernel:    7  ds   =0000002b 43
Jun 11 16:37:50 aout kernel:    8  es   =0000002b 43
Jun 11 16:37:50 aout kernel:    9  fs   =0000002b 43
Jun 11 16:37:50 aout kernel:    10 gs   =0000002b 43
Jun 11 16:37:50 aout kernel:    11 orig_eax=ffffffff 4294967295
Jun 11 16:37:50 aout kernel:    12 eip  =0804903a 134516794
Jun 11 16:37:50 aout kernel:    13 cs   =00000023 35
Jun 11 16:37:50 aout kernel:    14 efl  =00010282 66178
Jun 11 16:37:50 aout kernel:    15 uesp =bffff24c 3221221964
Jun 11 16:37:50 aout kernel:    16 ss   =0000002b 43
Jun 11 16:37:50 aout kernel: elf_core_dump: writing 080aa000 3000
Jun 11 16:37:50 aout kernel: elf_core_dump: writing 080ad000 2f000
Jun 11 16:37:52 aout kernel: elf_core_dump: writing bfffe000 2000


and this is after I '#if 0'ed it:


Jun 11 17:03:22 aout kernel: elf_core_dump: 4 segs taking 614400 bytes
Jun 11 17:03:22 aout kernel: Registers: Passed in regs
Jun 11 17:03:22 aout kernel:    0  ebx  =080aabf4 134917108
Jun 11 17:03:22 aout kernel:    1  ecx  =080ac6b4 134923956
Jun 11 17:03:22 aout kernel:    2  edx  =08049010 134516752
Jun 11 17:03:22 aout kernel:    3  esi  =bffff708 3221223176
Jun 11 17:03:22 aout kernel:    4  edi  =08049010 134516752
Jun 11 17:03:22 aout kernel:    5  ebp  =bffff68c 3221223052
Jun 11 17:03:22 aout kernel:    6  eax  =00000000 0
Jun 11 17:03:22 aout kernel:    7  ds   =0000002b 43
Jun 11 17:03:22 aout kernel:    8  es   =0000002b 43
Jun 11 17:03:22 aout kernel:    9  fs   =0000002b 43
Jun 11 17:03:22 aout kernel:    10 gs   =0000002b 43
Jun 11 17:03:22 aout kernel:    11 orig_eax=ffffffff 4294967295
Jun 11 17:03:22 aout kernel:    12 eip  =0804903a 134516794
Jun 11 17:03:22 aout kernel:    13 cs   =00000023 35
Jun 11 17:03:22 aout kernel:    14 efl  =00010282 66178
Jun 11 17:03:22 aout kernel:    15 uesp =bffff24c 3221221964
Jun 11 17:03:22 aout kernel:    16 ss   =0000002b 43
Jun 11 17:03:22 aout kernel: Registers: prstatus regs
Jun 11 17:03:22 aout kernel:    0  ebx  =080aabf4 134917108
Jun 11 17:03:22 aout kernel:    1  ecx  =080ac6b4 134923956
Jun 11 17:03:22 aout kernel:    2  edx  =08049010 134516752
Jun 11 17:03:22 aout kernel:    3  esi  =bffff708 3221223176
Jun 11 17:03:22 aout kernel:    4  edi  =08049010 134516752
Jun 11 17:03:22 aout kernel:    5  ebp  =bffff68c 3221223052
Jun 11 17:03:22 aout kernel:    6  eax  =00000000 0
Jun 11 17:03:22 aout kernel:    7  ds   =0000002b 43
Jun 11 17:03:22 aout kernel:    8  es   =0000002b 43
Jun 11 17:03:22 aout kernel:    9  fs   =0000002b 43
Jun 11 17:03:22 aout kernel:    10 gs   =0000002b 43
Jun 11 17:03:22 aout kernel:    11 orig_eax=ffffffff 4294967295
Jun 11 17:03:22 aout kernel:    12 eip  =0804903a 134516794
Jun 11 17:03:22 aout kernel:    13 cs   =00000023 35
Jun 11 17:03:22 aout kernel:    14 efl  =00010282 66178
Jun 11 17:03:22 aout kernel:    15 uesp =bffff24c 3221221964
Jun 11 17:03:22 aout kernel:    16 ss   =0000002b 43
Jun 11 17:03:24 aout kernel: elf_core_dump: writing 08048000 62000
Jun 11 17:03:34 aout kernel: elf_core_dump: writing 080aa000 3000
Jun 11 17:03:34 aout kernel: elf_core_dump: writing 080ad000 2f000
Jun 11 17:03:37 aout kernel: elf_core_dump: writing bfffe000 2000