using mmap in creating a new 1GB file

James Manning (jmm@raleigh.ibm.com)
Thu, 2 Dec 1999 14:40:11 -0500


I've been trying to figure out the best way to write out a file (known to
be 1GB in size once finished and non-existant to start) using mmap's... as
it stands, I mmap() an anon 1GB area, write to that and then spend a
large chunk of time in a single write() call at the end of the program.

What I'm hoping for is something that would let me allocate a 1GB file
on the ext2 filesystem (without any I/O, I don't even care about really
checking for enough free space), mmap it with PROT_WRITE,MAP_PRIVATE,
then do the sequential writing to the mmap'd area madvise'd such that
once I've finished writing to a page the page will get written out to
the actual file without blocking the process that is writing. (so I'm
just trying to get a lot of that drive I/O that is now serially after
the processing to occur in parallel).

As has been pointed out (many thanks to Chuck Lever), unless memory
gets low I risk waiting on all the pages written to be flush out at
munmap/close time.... one of his ideas included explicitly unmapping
each page once the app done writing to it (100% sequential writes),
although something that could generate this behavior implicitly
would be fantastic.

The writing process will take long enough such that there's more than
enough time for all the drive I/O's to occur to write out the data,
and I'm hoping to minimize or eliminate any blocking of the process
due to I/O.

Thanks,

James

-- 
Miscellaneous Engineer --- IBM Netfinity Performance Development

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/