Re: inotify limits - thousands (tens of thousands?) of watches

[Martin Steigerwald]

Am Mittwoch, 20. Mai 2009 schrieb Marcin Krol:
> Martin Steigerwald wrote:
> > Hmmm, I think you could just run a rsync periodically. It might even be
> > faster detecting changed files.
>
> I beg to differ on this: rsync does quite intensive (in terms of disk
> activity and CPU activity) comparisons at the beginning of
> synchronization. It's pretty light later, true, but running rsync every
> few minutes on entire /home is IMO out of question.

It depends on the amount of directories being watched and the amount directory 
entries in there. Rsync version 3 starts of synchronizing before building up 
the entire trees on both sides. It uses an incremental approach and this 
appears to make quite a difference.

For syncing /home I wonder whether inotify or fsnotify/fanotify is the right 
approach. A cluster filesystem of some sort comes to my mind instead. But 
question is, which one really works good enough. Didn't do any testing in 
that area yet.

> > I wrote a ruby script using libinotify-ruby which does just that. I only
> > syncs on demand tough. I.e. when someplace places a special sync file in
> > a watched directory.
> >
> > That script is running productively for well over a year now.
>
> Good to know the idea is not totally off the wall.. Thanks.

Main difference is that the sync is only triggered on demand. But the watching 
part appears to work quite nicely. Its not used for /home, but for the static 
html/php/image content for an apache2.2 which handles a lot of domains.

> Anyway, I'll try using fsnotify / fanotify.
>
> My main gripe with it, though, is that it is not in the mainline kernel,
> and thus in all probability it is not tested as widely as inotify.

Well then it will get more testing now I think ;-).

> Are there any chances for its inclusion in the near future?

Indeed would be nice to have.

-- 
Martin Steigerwald - team(ix) GmbH - http://www.teamix.de
gpg: 19E3 8D42 896F D004 08AC A0CA 1E10 C593 0399 AE90

Attachment: signature.asc
Description: This is a digitally signed message part.