Re: OFFTOPIC: write atomicity/jumbled logs

Rogier Wolff (R.E.Wolff@BitWizard.nl)
Thu, 30 Apr 1998 09:10:05 +0200 (MET DST)

Adam D. Bradley wrote:
>
>
> > > writing them all the time, and in the past four years nobody has
> > > complained of jumbled log messages. But Single Unix doesn't mention this,
> >
> > i don't know what's really involved, but jumbled logs happen constantly.
> > turn on ipfw logging then flood your server with network traffic.
>
> I think that's just printk's non-atomicity.
>
> As best I recall, the value write() returns always represents a
> contiguous stream of bytes written to the file descriptor (so that
> chunk will be atomic, but it may be less than the whole buffer you
> asked to be written), and for things like pipes/sockets and local
> files opened with O_APPEND, write()'s are guaranteed to be
> non-overlapping.

I don't think that that's the case.

There is an 8k buffer that holds the kernel output until "klogd" comes
and gets it. If you print twenty lines, klogd will send them to syslog.
This causes abbout 40 IOs which take almost a second to complete. If
during that second you overflow the 8k buffer, you'll get a "cut".

A method to significantly reduce this effect is to make syslog not
sync every message to disk. You could for instance make klogd simply
dump its output into a file. (cat /proc/kmsg > /var/log/kernel.log
would do fine if you're too lazy to RTFM, like me)

But take care: syslog now has a bandwidth of 30 messages per second.
Dropping the syncs will increase the bandwidth to 5Mb per second. You
can fill up your disk with logs much quicker than before.

Roger. 

-- 
If it's there and you can see it, it's REAL      |___R.E.Wolff@BitWizard.nl  |
If it's there and you can't see it, it's TRANSPARENT |  Tel: +31-15-2137555  |
If it's not there and you can see it, it's VIRTUAL   |__FAX:_+31-15-2138217  |
If it's not there and you can't see it, it's GONE! -- Roy Wilks, 1983  |_____|


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu