On Sat, 11 Feb 2006, Nick Piggin wrote:
When MS_ASYNC is specified, msync() shall return immediately once all
the write operations are initiated or queued for servicing;
It is talking about write operations, not dirtying. Actually the only
difference with MS_SYNC is that it waits for said write operations (of the
type queued up by MS_ASYNC) to complete.
Right. And it's what we do. We queue them by moving the pages to the dirty lists (yeah, it's just a tag on the page index thing, whatever).
And yes, you argue that we should move the queue closer to the actual disk, but I have used at least one app that really hated the "start IO now" approach. I can't talk about that app in any detail, but I can say that it was an in-memory checkpoint thing with the checkpoints easily being in the hundred-meg range.
And moving a hundred megs to the IO layer is insane. It also makes the system pretty unusable.
So we may have different expectations, because we've seen different patterns. Me, I've seen the "events are huge, and you stagger them", so that the previous event has time to flow out to disk while you generate the next one. There, MS_ASYNC starting IO is _wrong_, because the scale of the event is just huge, so trying to push it through the IO subsystem asap just makes everything suck.
In contrast, you seem to be coming at it from a standpoint of "only one event ever outstanding at any particular time, and it's either small or it's the only thing the whole system is doing". In which case pushing it out to IO buffers is probably the right thing to do.