Re: zero out blocks of freed user data for operation a virtual machine environment

From: Bill Davidsen
Date: Mon May 25 2009 - 17:19:41 EST

Thomas Glanzmann wrote:
Hello Ted,
I would like to know if there is already a mount option or feature in
ext3/ext4 that automatically overwrites freed blocks with zeros? If this
is not the case I would like to know if you would consider a patch for
upstream? I'm asking this because I currently do some research work on
data deduplication in virtual machine environments and corresponding
backups. It would be a huge space saver if there is such a feature
because todays and tomorrows backup tools for virtual machine
environments work on the block layer (VMware Consolidated Backup, VMware
Data Recovery, and NetApp Snapshots). This is not only true for backup
tools but also for running Virtual machines. The case that this future
addresses is the following: A huge file is downloaded and later delted.
The backup and datadeduplication that is operating on the block level
can't identify the block as unused. This results in backing up the
amount of the data that was previously allocated by the file and as such
introduces an performance overhead. If you're interested in real live
data, I'm able to provide them.

If you don't intend to have such an optional feature in ext3/ext4 I
would like to know if you know a tool that makes it possible to zero out
unused blocks?

Treating blocks as unused due to content seems a bad idea, if you want them to be unused look for references to TRIM, if you want this for security look at shred. And if you are interested in backing sparse files I believe that the tar "-S" option will do what you want or provide code you can use to start writing what you want.

I don't think this is a good solution to the problem that unused space is not accounted as you wish it would be. Most filesystems have a bitmap to track this already, a handle on that would be more generally useful.

Deleting files is slow enough, identifying unused storage by content is 1950s thinking, and also ignores the fact that new drives often don't come zeroed, and would behave badly unless you manually zeroed the unused portions.

I doubt this is the optimal solution, since you would have to read the zeros to see if they were present, making backup smaller but no faster than just doing a copy.

Bill Davidsen <davidsen@xxxxxxx>
"We have more to fear from the bungling of the incompetent than from
the machinations of the wicked." - from Slashdot
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at