Re: Large memory application exhuasts buffers during write.
From: Christoph Hellwig
Date: Fri Mar 26 2004 - 05:59:36 EST
On Thu, Mar 25, 2004 at 07:20:56PM -0600, Robin Holt wrote:
> This is a 2.4 based kernel with many of the redhat patches applied.
> Before the application is started, there is approx 350GB of memory
> free according to top. When the app starts, it mallocs a 300GB
> buffer, initializes it, does computations into it, and then starts
> to write it to a disk file.
> What we see happen is the first approx 30GBs gets written and then
> swap starts getting utilized. Once swap has been heavily utilized,
> the OOM killer kicks in and kills the job.
Buffered writes or O_DIRECT? I guess you're doing the former and
actually want the latter. Try preloading a tiny library stub that
adds O_DIRECT to open for the interesting fds.
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/