Re: BUG ext4_inode_cache: Padding overwritten. 0xffff88000008fe00-0xffff88000008fe32

[Justin P. Mattock]

On 09/07/2010 07:54 AM, Eric Sandeen wrote:
> Justin Mattock wrote:
> > Im not sure if theres already assigned to this or not...  but noticed
> > this in dmesg..
> > only thing I remember was building glibc, and looking into an issue
> > with it, suspended a few times and such..
>
> hm so ext4's inode cache got overwritten, indicative of memory corruption
> but hard to know how it happened; it's not necessarily (maybe even likely)
> an ext4 bug... not quite sure how to track it down though.
>
>
> Also the M taint indicates:
>
>    5: 'M' if any processor has reported a Machine Check Exception,
>       ' ' if no Machine Check Exceptions have occurred.
>
> so something may have gone wrong before this.
>
> -Eric

I was doing a bisect on glibc to find a certain commit, maybe the 
constant installation of those libraries, and the combination of putting 
the machine a sleep while I was doing other things caused this to fire 
off.(after doing all of this, and letting the machine idol and run, 
nothing like this showed up.)

> > [13299.426905] =============================================================================
> > [13299.426913] BUG ext4_inode_cache: Padding overwritten.
> > 0xffff88000008fe00-0xffff88000008fe32
> > [13299.426919] -----------------------------------------------------------------------------
> > [13299.426922]
> > [13299.426928] INFO: Slab 0xffffea0000001ea0 objects=13 used=11
> > fp=0xffff88000008d798 flags=0x4081
> > [13299.426936] Pid: 17044, comm: rm Tainted: G   M
> > 2.6.36-rc3-00001-gb0425ef #2
> > [13299.426942] Call Trace:
> > [13299.426955]  [<ffffffff810e3a5c>] slab_err+0x89/0x95
> > [13299.426966]  [<ffffffff8109c5fe>] ? call_rcu_sched+0x10/0x12
> > [13299.426974]  [<ffffffff8109c609>] ? call_rcu+0x9/0xb
> > [13299.426982]  [<ffffffff810e48b9>] slab_pad_check.part.46+0xc0/0xf7
> > [13299.426991]  [<ffffffff8114c1b9>] ? ext4_destroy_inode+0x96/0x9d
> > [13299.426999]  [<ffffffff810e4984>] check_slab+0x94/0x9a
> > [13299.427008]  [<ffffffff813e5c38>] __slab_free+0x15e/0x342
> > [13299.427008]  [<ffffffff810e4f34>] kmem_cache_free+0x96/0xed
> > [13299.427008]  [<ffffffff8114c1b9>] ext4_destroy_inode+0x96/0x9d
> > [13299.427008]  [<ffffffff81114fd9>] ? __fsnotify_inode_delete+0x9/0xb
> > [13299.427008]  [<ffffffff810fd046>] ? __destroy_inode+0x27/0x76
> > [13299.427008]  [<ffffffff810fd4fb>] destroy_inode+0x2a/0x3f
> > [13299.427008]  [<ffffffff810fd948>] iput+0x223/0x228
> > [13299.427008]  [<ffffffff810f4bcb>] do_unlinkat+0x118/0x16b
> > [13299.427008]  [<ffffffff810ee3ad>] ? sys_newfstatat+0x25/0x2e
> > [13299.427008]  [<ffffffff81094841>] ? audit_syscall_entry+0x1d1/0x1fd
> > [13299.427008]  [<ffffffff810f641e>] sys_unlinkat+0x24/0x26
> > [13299.427008]  [<ffffffff81027602>] system_call_fastpath+0x16/0x1b
> > [13299.427008]  Padding 0xffff88000008fb8b:  00 ad 4e ad de ff ff ff
> > ff 00 00 00 00 ff ff ff .­N­Þÿÿÿÿ....ÿÿÿ
> > [13299.427008]  Padding 0xffff88000008fb9b:  ff ff ff ff ff a0 fb 08
> > 00 00 88 ff ff a0 fb 08 ÿÿÿÿÿ.û....ÿÿ.û.
> > [13299.427008]  Padding 0xffff88000008fbab:  00 00 88 ff ff 00 00 00
> > 00 00 00 00 00 b8 fb 08 ...ÿÿ........¸û.
> > [13299.427008]  Padding 0xffff88000008fbbb:  00 00 88 ff ff b8 fb 08
> > 00 00 88 ff ff 00 00 00 ...ÿÿ¸û....ÿÿ...
> > [13299.427008]  Padding 0xffff88000008fbcb:  00 00 00 00 00 25 53 d9
> > 92 00 00 00 00 00 00 00 .....%SÙ........
> > [13299.427008]  Padding 0xffff88000008fbdb:  00 00 00 00 00 00 00 00
> > 00 00 00 00 00 d9 59 bb .............ÙY»
> > [13299.427008]  Padding 0xffff88000008fbeb:  00 01 00 00 00 00 00 00
> > 00 00 00 00 00 10 1a b7 ...............·
> > [13299.427008]  Padding 0xffff88000008fbfb:  2e 00 88 ff ff ff ff ff
> > ff ff ff ff ff ff ff ff ...ÿÿÿÿÿÿÿÿÿÿÿÿÿ
> > [13299.427008]  Padding 0xffff88000008fc0b:  ff ff ff ff ff 00 00 00
> > 00 00 00 00 00 00 00 00 ÿÿÿÿÿ...........
> > [13299.427008]  Padding 0xffff88000008fc1b:  00 00 00 00 00 00 00 00
> > 00 00 00 00 00 00 01 10 ................
> > [13299.427008]  Padding 0xffff88000008fc2b:  00 00 00 ad de 00 02 20
> > 00 00 00 ad de 80 f9 08 ...­Þ......­Þ.ù.
> > [13299.427008]  Padding 0xffff88000008fc3b:  00 00 88 ff ff 00 00 00
> > 00 5a 5a 5a 5a b8 b6 28 ...ÿÿ....ZZZZ¸¶(
> > [13299.427008]  Padding 0xffff88000008fc4b:  00 00 00 00 00 00 00 00
> > 00 01 00 00 00 02 00 00 ................
> > [13299.427008]  Padding 0xffff88000008fc5b:  00 00 00 00 00 a2 99 81
> > 4c 00 00 00 00 13 41 21 .....¢..L.....A!
> > [13299.427008]  Padding 0xffff88000008fc6b:  37 00 00 00 00 70 fc 08
> > 00 00 88 ff ff 70 fc 08 7....pü....ÿÿpü.
> > [13299.427008]  Padding 0xffff88000008fc7b:  00 00 88 ff ff 01 01 00
> > 00 ad 4e ad de ff ff ff ...ÿÿ....­N­Þÿÿÿ
> > [13299.427008]  Padding 0xffff88000008fc8b:  ff 5a 5a 5a 5a ff ff ff
> > ff ff ff ff ff ff ff ff ÿZZZZÿÿÿÿÿÿÿÿÿÿÿ
> > [13299.427008]  Padding 0xffff88000008fc9b:  ff 00 00 00 00 00 00 00
> > 00 00 00 00 00 00 00 5a ÿ..............Z
> > [13299.427008]  Padding 0xffff88000008fcab:  5a 5a 5a 5a 5a 00 00 00
> > 00 00 00 00 00 00 00 00 ZZZZZ...........
> > [13299.427008]  Padding 0xffff88000008fcbb:  00 1c 00 5a 5a 0e 0e 00
> > 00 ad 4e ad de ff ff ff ...ZZ....­N­Þÿÿÿ
> > [13299.427008]  Padding 0xffff88000008fccb:  ff 5a 5a 5a 5a ff ff ff
> > ff ff ff ff ff d8 fc 08 ÿZZZZÿÿÿÿÿÿÿÿØü.
> > [13299.427008]  Padding 0xffff88000008fcdb:  00 00 88 ff ff d8 fc 08
> > 00 00 88 ff ff 00 00 00 ...ÿÿØü....ÿÿ...
> > [13299.427008]  Padding 0xffff88000008fceb:  00 ad 4e ad de ff ff ff
> > ff 5a 5a 5a 5a ff ff ff .­N­ÞÿÿÿÿZZZZÿÿÿ
> > [13299.427008]  Padding 0xffff88000008fcfb:  ff ff ff ff ff 00 00 00
> > 00 00 00 00 00 bb 59 03 ÿÿÿÿÿ........»Y.
> > [13299.427008]  Padding 0xffff88000008fd0b:  00 bb 59 03 00 cc cc cc
> > cc cc cc cc cc e8 f3 08 .»Y..ÌÌÌÌÌÌÌÌèó.
> > [13299.427008]  Padding 0xffff88000008fd1b:  00 00 88 ff ff 9f 60 14
> > 81 ff ff ff ff 01 00 00 ...ÿÿ.`..ÿÿÿÿ...
> > [13299.427008]  Padding 0xffff88000008fd2b:  00 c0 28 00 00 a5 58 bb
> > 00 01 00 00 00 b9 c1 14 .À(..¥X».....¹Á.
> > [13299.427008]  Padding 0xffff88000008fd3b:  81 ff ff ff ff 01 00 00
> > 00 0f 1c 00 00 b0 7a ba .ÿÿÿÿ........°zº
> > [13299.427008]  Padding 0xffff88000008fd4b:  00 01 00 00 00 5a 5a 5a
> > 5a 5a 5a 5a 5a 5a 5a 5a .....ZZZZZZZZZZZ
> > [13299.427008]  Padding 0xffff88000008fd5b:  5a 5a 5a 5a 5a 5a 5a 5a
> > 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> > [13299.427008]  Padding 0xffff88000008fd6b:  5a 5a 5a 5a 5a 5a 5a 5a
> > 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> > [13299.427008]  Padding 0xffff88000008fd7b:  5a 5a 5a 5a 5a 5a 5a 5a
> > 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> > [13299.427008]  Padding 0xffff88000008fd8b:  5a 5a 5a 5a 5a 5a 5a 5a
> > 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> > [13299.427008]  Padding 0xffff88000008fd9b:  5a 5a 5a 5a 5a 5a 5a 5a
> > 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> > [13299.427008]  Padding 0xffff88000008fdab:  5a 5a 5a 5a 5a 5a 5a 5a
> > 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> > [13299.427008]  Padding 0xffff88000008fdbb:  5a 5a 5a 5a 5a 5a 5a 5a
> > 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> > [13299.427008]  Padding 0xffff88000008fdcb:  5a 5a 5a 5a 5a 5a 5a 5a
> > 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> > [13299.427008]  Padding 0xffff88000008fddb:  5a 5a 5a 5a 5a 5a 5a 5a
> > 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> > [13299.427008]  Padding 0xffff88000008fdeb:  5a 5a 5a 5a 5a 5a 5a 5a
> > 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> > [13299.427008]  Padding 0xffff88000008fdfb:  5a 5a 5a 5a 5a 00 5a 5a
> > 5a 10 20 0f 3f 00 80 00 ZZZZZ.ZZZ...?...
> > [13299.427008]  Padding 0xffff88000008fe0b:  00 c7 60 ed 3f 3f 00 10
> > df ed 3f ff 07 10 c0 ed .Ç`í??..ßí?ÿ..Àí
> > [13299.427008]  Padding 0xffff88000008fe1b:  3f fa 33 c9 33 d2 b8 3c
> > fe 08 00 0f 01 c8 b8 31 ?ú3É3Ò¸<þ....ȸ1
> > [13299.427008]  Padding 0xffff88000008fe2b:  00 00 00 0f 01 c9 eb e9
> >                        .....Éëé
> > [13299.427008] FIX ext4_inode_cache: Restoring
> > 0xffff88000008fb8b-0xffff88000008fe32=0x5a
> > [13299.427008]
> > [13369.168442] general protection fault: 0000 [#1] SMP
> > [13369.168448] last sysfs file: /sys/devices/platform/applesmc.768/light
> > [13369.168451] CPU 0
> > [13369.168453] Modules linked in: xfrm4_mode_transport xcbc rmd160
> > sha512_generic sco bnep xt_tcpudp ipt_LOG iptable_nat nf_nat xt_state
> > nf_conntrack_ftp nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4
> > iptable_filter ip_tables x_tables snd_hda_codec_idt ath9k
> > firewire_ohci firewire_core ath9k_common evdev ath9k_hw ohci1394
> > joydev button thermal battery sky2 ac ath snd_hda_intel snd_hda_codec
> > snd_hwdep snd_pcm snd_timer snd soundcore snd_page_alloc i2c_i801
> > video aes_x86_64 lzo lzo_compress tun kvm_intel ipcomp xfrm_ipcomp
> > crypto_null sha256_generic cbc des_generic cast5 blowfish serpent
> > camellia twofish_generic twofish_x86_64 twofish_common ctr ah4 esp4
> > authenc raw1394 ieee1394 uhci_hcd ehci_hcd hci_uart rfcomm btusb hidp
> > l2cap bluetooth coretemp acpi_cpufreq processor mperf appletouch
> > applesmc
> > [13369.168522]
> > [13369.168526] Pid: 21876, comm: objdump Tainted: G   M
> > 2.6.36-rc3-00001-gb0425ef #2 Mac-F42187C8/MacBookPro2,2
> > [13369.168529] RIP: 0010:[<ffffffff811a19ad>]  [<ffffffff811a19ad>]
> > selinux_dentry_open+0x27/0x91
> > [13369.168540] RSP: 0018:ffff88000dcb7c98  EFLAGS: 00010292
> > [13369.168543] RAX: 5a5a5a5a5a5a5a5a RBX: ffff88000008f980 RCX: ffff880001800000
> > [13369.168545] RDX: ffff880001800000 RSI: ffff88000ddf6800 RDI: ffff880030d6f500
> > [13369.168548] RBP: ffff88000dcb7cb8 R08: 0000000000000000 R09: ffff88001aa70360
> > [13369.168551] R10: ffff880030f25c6c R11: ffff88000191c000 R12: ffff880030d6f500
> > [13369.168553] R13: ffff88001e6b79a0 R14: ffff88000ddf6800 R15: ffff88000008f980
> > [13369.168557] FS:  00002ab84b7ebb20(0000) GS:ffff880001800000(0000)
> > knlGS:0000000000000000
> > [13369.168560] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [13369.168562] CR2: 00002ab84b52b300 CR3: 00000000276ed000 CR4: 00000000000006f0
> > [13369.168565] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > [13369.168568] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> > [13369.168571] Process objdump (pid: 21876, threadinfo
> > ffff88000dcb6000, task ffff88001e799ec0)
> > [13369.168573] Stack:
> > [13369.168575]  ffff880030d6f500 ffff88003ecf08c0 ffff88001aa70360
> > 0000000000000000
> > [13369.168579]<0>  ffff88000dcb7cd8 ffffffff8119afc9 ffff880030d6f500
> > ffff880030d6f500
> > [13369.168584]<0>  ffff88000dcb7d48 ffffffff810e8ebc ffff88000dcb7d18
> > ffffffff810f2f78
> > [13369.168589] Call Trace:
> > [13369.168595]  [<ffffffff8119afc9>] security_dentry_open+0x19/0x4c
> > [13369.168601]  [<ffffffff810e8ebc>] __dentry_open+0x159/0x2be
> > [13369.168607]  [<ffffffff810f2f78>] ? generic_permission+0x17/0xa4
> > [13369.168611]  [<ffffffff810e9e96>] nameidata_to_filp+0x3a/0x4b
> > [13369.168616]  [<ffffffff810f58cb>] do_last+0x504/0x691
> > [13369.168620]  [<ffffffff810f5c48>] do_filp_open+0x1f0/0x621
> > [13369.168625]  [<ffffffff810ff29f>] ? alloc_fd+0x102/0x114
> > [13369.168629]  [<ffffffff810e9f02>] do_sys_open+0x5b/0xf7
> > [13369.168633]  [<ffffffff810e9fb9>] sys_open+0x1b/0x1d
> > [13369.168639]  [<ffffffff81027602>] system_call_fastpath+0x16/0x1b
> > [13369.168642] Code: ff ff c9 c3 55 48 89 e5 41 56 49 89 f6 41 55 41
> > 54 49 89 fc 53 48 8b 47 18 4c 8b af c0 00 00 00 48 8b 58 28 48 8b 83
> > 78 02 00 00<8b>  40 1c 41 89 45 08 e8 a1 af ff ff 41 89 45 0c 41 8b 44
> > 24 54
> > [13369.168681] RIP  [<ffffffff811a19ad>] selinux_dentry_open+0x27/0x91
> > [13369.168685]  RSP<ffff88000dcb7c98>
> > [13369.168688] ---[ end trace a79a28a58d538d97 ]---
> >
> > as for performance issues,would of never know about this if I hadnt
> > looked in dmesg.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/