Re: file corruption in loop device in 3.4.0

[Ming Lei]

On Wed, Jun 6, 2012 at 10:24 PM, Bob Sullivan <bobsullivan60@xxxxxxxxx> wrote:
> We have a EXT3 filesystem contained in a file
> (a disk image mapped to a file using loop device).
> Using diff, observed random differences between
> files on the image and a backup copies of those files.
> This problem is reproducible,
> tried booting several times with 3.3.8 and 3.4.0.
> No problems on 3.3.8.

A bit different with me, no problems on 3.4, but have problems on 3.5-rc1.

I can easily reproduce the problem, just kernel boot and mount
rootfs(ext4), then
reboot again and found there are many rootfs errors as below, see attachment
for fsck check result.

[   22.451385] EXT4-fs error (device mmcblk0p2):
ext4_mb_generate_buddy:741: group 2, 0 clusters in bitmap, 1 in gd
[   23.263092] JBD2: Spotted dirty metadata buffer (dev = mmcblk0p2,
blocknr = 0). There's a risk of filesystem corruption in case of
system crash.


BTW: I use mmc card to store rootfs and test it on OMAP3 based beagle-xm board.

Thanks,
-- 
Ming Lei
[tom@~]$sudo fsck.ext4 -v -n  /dev/sdb2
e2fsck 1.42 (29-Nov-2011)
rootfs contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes

Running additional passes to resolve blocks claimed by more than one inode...
Pass 1B: Rescanning for multiply-claimed blocks
Multiply-claimed block(s) in inode 1231: 589988 589989 589990 589991
Multiply-claimed block(s) in inode 131359: 589988 589989 589990 589991
Multiply-claimed block(s) in inode 131423: 1082148
Multiply-claimed block(s) in inode 131424: 1056825
Multiply-claimed block(s) in inode 131552: 591167 591168 591169 591170 591171
591172 591173 591174 591175 591176 591177 591178 591179 591180 591181 591182
591183 591184 591185 591186 591187 591188 591189 591190 591191 591192 591193
591194 591195 591196 591197 591198 591199 591200 591201 591202 591203 591204
591205 591206 591207 591208 591209 591210 591211
Multiply-claimed block(s) in inode 134672: 697812 697813 697814 697815 697816
697817 697818 697819 697820 697821
Multiply-claimed block(s) in inode 262703: 1082148
Multiply-claimed block(s) in inode 262704: 1056825
Multiply-claimed block(s) in inode 262751: 1056835
Multiply-claimed block(s) in inode 262752: 1082186
Multiply-claimed block(s) in inode 263040: 591167 591168 591169 591170 591171
591172 591173 591174 591175 591176 591177 591178 591179 591180 591181 591182
591183 591184 591185 591186 591187 591188 591189 591190 591191 591192 591193
591194 591195 591196 591197 591198 591199 591200 591201 591202 591203 591204
591205 591206 591207 591208 591209 591210 591211
Multiply-claimed block(s) in inode 263103: 1056835
Multiply-claimed block(s) in inode 263104: 1082186
Multiply-claimed block(s) in inode 396943: 1615002 1615003
Multiply-claimed block(s) in inode 396944: 1615004 1615005 1615006 1615007
1615008 1615009
Multiply-claimed block(s) in inode 397023: 1615002 1615003
Multiply-claimed block(s) in inode 397024: 1615004 1615005 1615006 1615007
1615008 1615009
Multiply-claimed block(s) in inode 397312: 697812 697813 697814 697815 697816
697817 697818 697819 697820 697821
Pass 1C: Scanning directories for inodes with multiply-claimed blocks
Pass 1D: Reconciling multiply-claimed blocks
(There are 18 inodes containing multiply-claimed blocks.)

File /home/root/bin/test/perf_event_tests-0.24/papi/papi_fp_ops.c (inode
#1231, mod time Wed Sep  8 18:04:15 2010) 
  has 4 multiply-claimed block(s), shared with 1 file(s):
	/bin/kill.procps (inode #131359, mod time Wed Sep  8 18:04:15 2010)
Clone multiply-claimed blocks? no

Delete file? no

File /bin/kill.procps (inode #131359, mod time Wed Sep  8 18:04:15 2010) 
  has 4 multiply-claimed block(s), shared with 1 file(s):
	/home/root/bin/test/perf_event_tests-0.24/papi/papi_fp_ops.c (inode
#1231, mod time Wed Sep  8 18:04:15 2010)
Clone multiply-claimed blocks? no

Delete file? no

File /bin/true (inode #131423, mod time Thu Oct 28 19:33:39 2010) 
  has 1 multiply-claimed block(s), shared with 1 file(s):
	/etc/network/options (inode #262703, mod time Thu Oct 28 19:33:39
2010)
Clone multiply-claimed blocks? no

Delete file? no

File /bin/ln.coreutils (inode #131424, mod time Sat Jan  1 08:04:46 2000) 
  has 1 multiply-claimed block(s), shared with 1 file(s):
	/etc/network/run (inode #262704, mod time Sat Jan  1 08:05:46 2000)
Clone multiply-claimed blocks? no

Delete file? no

File /lib/libcidn-2.9.so (inode #131552, mod time Sat Oct  2 08:20:57 2010) 
  has 45 multiply-claimed block(s), shared with 1 file(s):
	/etc/init.d/reboot (inode #263040, mod time Sat Oct  2 08:20:57 2010)
Clone multiply-claimed blocks? no

Delete file? no

File /lib/libudev.so.0.6.1 (inode #134672, mod time Sat Oct 30 17:21:36 2010) 
  has 10 multiply-claimed block(s), shared with 1 file(s):
	/usr/lib/libxcb-screensaver.so.0 (inode #397312, mod time Sat Oct 30
17:21:36 2010)
Clone multiply-claimed blocks? no

Delete file? no

File /etc/network/options (inode #262703, mod time Thu Oct 28 19:33:39 2010) 
  has 1 multiply-claimed block(s), shared with 1 file(s):
	/bin/true (inode #131423, mod time Thu Oct 28 19:33:39 2010)
Clone multiply-claimed blocks? no

Delete file? no

File /etc/network/run (inode #262704, mod time Sat Jan  1 08:05:46 2000) 
  has 1 multiply-claimed block(s), shared with 1 file(s):
	/bin/ln.coreutils (inode #131424, mod time Sat Jan  1 08:04:46 2000)
Clone multiply-claimed blocks? no

Delete file? no

File /etc/iproute2 (inode #262751, mod time Tue Nov  2 19:17:20 2010) 
  has 1 multiply-claimed block(s), shared with 1 file(s):
	/lib/modules/3.4.0-rc5+/kernel/drivers/net/usb/usbnet.ko (inode
#263103, mod time Tue Nov  2 19:17:20 2010)
Clone multiply-claimed blocks? no

Delete file? no

File /etc/iproute2/rt_dsfield (inode #262752, mod time Thu Sep  9 19:16:08
2010) 
  has 1 multiply-claimed block(s), shared with 1 file(s):
	/lib/modules/3.4.0-rc5+/kernel/drivers/net/usb/net1080.ko (inode
#263104, mod time Thu Sep  9 19:16:08 2010)
Clone multiply-claimed blocks? no

Delete file? no

File /etc/init.d/reboot (inode #263040, mod time Sat Oct  2 08:20:57 2010) 
  has 45 multiply-claimed block(s), shared with 1 file(s):
	/lib/libcidn-2.9.so (inode #131552, mod time Sat Oct  2 08:20:57 2010)
Clone multiply-claimed blocks? no

Delete file? no

File /lib/modules/3.4.0-rc5+/kernel/drivers/net/usb/usbnet.ko (inode #263103,
mod time Tue Nov  2 19:17:20 2010) 
  has 1 multiply-claimed block(s), shared with 1 file(s):
	/etc/iproute2 (inode #262751, mod time Tue Nov  2 19:17:20 2010)
Clone multiply-claimed blocks? no

Delete file? no

File /lib/modules/3.4.0-rc5+/kernel/drivers/net/usb/net1080.ko (inode #263104,
mod time Thu Sep  9 19:16:08 2010) 
  has 1 multiply-claimed block(s), shared with 1 file(s):
	/etc/iproute2/rt_dsfield (inode #262752, mod time Thu Sep  9 19:16:08
2010)
Clone multiply-claimed blocks? no

Delete file? no

File /usr/bin/setkeycodes (inode #396943, mod time Thu Sep  9 19:18:44 2010) 
  has 2 multiply-claimed block(s), shared with 1 file(s):
	/usr/bin/sensible-pager (inode #397023, mod time Thu Sep  9 19:18:44
2010)
Clone multiply-claimed blocks? no

Delete file? no

File /usr/bin/dirname.coreutils (inode #396944, mod time Wed Dec 15 05:19:56
2010) 
  has 6 multiply-claimed block(s), shared with 1 file(s):
	/usr/bin/git-receive-pack (inode #397024, mod time Wed Dec 15 05:19:56
2010)
Clone multiply-claimed blocks? no

Delete file? no

File /usr/bin/sensible-pager (inode #397023, mod time Thu Sep  9 19:18:44
2010) 
  has 2 multiply-claimed block(s), shared with 1 file(s):
	/usr/bin/setkeycodes (inode #396943, mod time Thu Sep  9 19:18:44
2010)
Clone multiply-claimed blocks? no

Delete file? no

File /usr/bin/git-receive-pack (inode #397024, mod time Wed Dec 15 05:19:56
2010) 
  has 6 multiply-claimed block(s), shared with 1 file(s):
	/usr/bin/dirname.coreutils (inode #396944, mod time Wed Dec 15
05:19:56 2010)
Clone multiply-claimed blocks? no

Delete file? no

File /usr/lib/libxcb-screensaver.so.0 (inode #397312, mod time Sat Oct 30
17:21:36 2010) 
  has 10 multiply-claimed block(s), shared with 1 file(s):
	/lib/libudev.so.0.6.1 (inode #134672, mod time Sat Oct 30 17:21:36
2010)
Clone multiply-claimed blocks? no

Delete file? no

Pass 2: Checking directory structure
Entry 'papi_l1_tch.c' in /home/root/bin/test/perf_event_tests-0.24/papi (1223)
has an incorrect filetype (was 1, should be 7).
Fix? no

Entry 'true' in /bin (131329) has an incorrect filetype (was 7, should be 1).
Fix? no

Entry 'ln.coreutils' in /bin (131329) has an incorrect filetype (was 1, should
be 2).
Fix? no

Invalid inode number for '.' in directory inode 131424.
Fix? no

Invalid inode number for '.' in directory inode 263103.
Fix? no

Entry 'rcS' in /etc/init.d (263027) has an incorrect filetype (was 1, should
be 7).
Fix? no

Entry 'usbnet.ko' in /lib/modules/3.4.0-rc5+/kernel/drivers/net/usb (263100)
has an incorrect filetype (was 1, should be 2).
Fix? no

Entry 'git-receive-pack' in /usr/bin (396673) has an incorrect filetype (was
7, should be 1).
Fix? no

Entry 'sensible-pager' in /usr/bin (396673) has an incorrect filetype (was 7,
should be 1).
Fix? no

Entry 'libxcb-screensaver.so.0' in /usr/lib (397295) has an incorrect filetype
(was 7, should be 1).
Fix? no

Pass 3: Checking directory connectivity
'..' in /bin/ln.coreutils (131424) is /etc/network (262694), should be /bin
(131329).
Fix? no

'..' in /lib/modules/3.4.0-rc5+/kernel/drivers/net/usb/usbnet.ko (263103) is
/etc (262662), should be /lib/modules/3.4.0-rc5+/kernel/drivers/net/usb
(263100).
Fix? no

Pass 4: Checking reference counts
Inode 131424 ref count is 2, should be 1.  Fix? no

Inode 131567 ref count is 36, should be 37.  Fix? no

Inode 262662 ref count is 33, should be 34.  Fix? no

Inode 262694 ref count is 7, should be 8.  Fix? no

Inode 262704 ref count is 2, should be 3.  Fix? no

Inode 262705 ref count is 1, should be 2.  Fix? no

Inode 262751 ref count is 2, should be 3.  Fix? no

Inode 262752 ref count is 1, should be 2.  Fix? no

Inode 262753 ref count is 1, should be 2.  Fix? no

Inode 262754 ref count is 1, should be 2.  Fix? no

Inode 262755 ref count is 1, should be 2.  Fix? no

Inode 262756 ref count is 1, should be 2.  Fix? no

Inode 262757 ref count is 1, should be 2.  Fix? no

Inode 263103 ref count is 2, should be 1.  Fix? no

Pass 5: Checking group summary information
Block bitmap differences:  -(96426--96427) -96948 -(590388--590399)
-(1082428--1082429) -(1082567--1082679)
Fix? no

Free blocks count wrong for group #2 (1, counted=0).
Fix? no

Free blocks count wrong (1468255, counted=1468254).
Fix? no

Directories count wrong for group #16 (863, counted=864).
Fix? no

Directories count wrong for group #32 (1111, counted=1112).
Fix? no


rootfs: ********** WARNING: Filesystem still has errors **********


   31403 inodes used (6.48%)
      14 non-contiguous files (0.0%)
       6 non-contiguous directories (0.0%)
         # of inodes with ind/dind/tind blocks: 0/0/0
         Extent depth histogram: 30306/1
  465057 blocks used (24.05%)
       0 bad blocks
       1 large file

   27009 regular files
    3131 directories
      33 character device files
      36 block device files
       2 fifos
       7 links
    1183 symbolic links (1017 fast symbolic links)
       0 sockets
--------
   31401 files