Re: Q: sparse file creation in existing data?

From: Ph. Marek (marek@bmlv.gv.at)
Date: Fri Jun 29 2001 - 07:55:29 EST


>For your specific problem I'd suggest the following approach:
>
>write a new filter prog, kind of a 'destructive cat' command.
>
>Open the file for read-modify (non destructive)
>Let it read some blocks (number controllable on commandline) from the
>beginning and pipe them to stdout. Then.. read in same amount (well;
>block aligned) from the end of file and copy it over the beginning,
>truncate the file accordingly. After some time reading and truncating
>will meet in the middle of the file, then you read the blocks back in
>on reverse (and truncate them after reading)
>
>It shouldn't be too difficult to write and result in a multipurpose
>commandline utility.
>
>It does some disk i/o but I don't think it will too bad due to disk
>buffers of the kernel and read-ahead. Theoretically, the kernel, buffers
>etc. could detect you are just moving file blocks to different places and
>does this in a 'zero-copy' fashion by just moving some inode entries
>around, but I doubt that it is soo smart.
>
>Drawback: If you have choosen the read block size too large (hmm.. you
>might want to read some data from beginning, some from the end, copy over
>beginning, then truncate, and only then pipe to stdout: this should
>eliminate this 'buffer/diskspace overrun') of 'destructive cat' too big,
>or the archive is compressed and doesn't fit uncompressed, the backup was
>destructed. This is by far not as safe as any backup tool should be.
>
>A major advantage, however, is that this tool will run w/o a kernel patch,
>on any Unix on any filesystem, even those w/o holes (I think truncate
>works in general, maybe not samba?)
I didn't even get this idea - it sound's very io-intensive.
Of course, if you look at it from the right angle - you'd
- read the first halve,
- read the 2nd halve,
- and write the 2nd halve to the 1st,
- and then read the 1st again.

That's two times the amount of io.
If I assume using tar it's because of writing the data only one third more
io than necessary.

Hmmm, on second thought ... But I'd like it better to have a fcntl for
hole-making :-)
Maybe I'll implement this myself.

Thanks,

Phil

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Sat Jun 30 2001 - 21:00:22 EST