Re: Things I wish I'd known about Inotify

From: Eric W. Biederman
Date: Thu Apr 03 2014 - 11:39:31 EST


"Michael Kerrisk (man-pages)" <mtk.manpages@xxxxxxxxx> writes:

> (To: == [the set of people I believe know a lot about inotify])
>
> Hello all,
>
> Lately, I've been studying the inotify API fairly thoroughly and
> realized that there's a very big gap between knowing what the system
> calls do versus using them to reliably and efficiently monitor the
> state of a set of filesystem objects.
>
> With that in mind, I've drafted some substantial additions to the
> inotify(7) man page. I would be very happy if folk on the "To:" list
> could comment on the text below, since I believe you all have a lot of
> practical experience with Inotify. (Of course, I also welcome comments
> from anyone else.) In particular, I would like comments on the
> accuracy of the various technical points (especially those relating to
> matching up related IN_MOVED_FROM and IN_MOVED_TO events), as well as
> pointers on any other pitfalls that the programmers should be wary of
> that should be added to the page.


Other pitfalls.

Inotify only report events that a user space program triggers through
the filesystem API. Which means inotify is limited for remote
filesystems, and filesystems like proc and sys have no monitorable
events.

Eric


> Thanks,
>
> Michael
>
> Limitations and caveats
> The inotify API provides no information about the user or process
> that triggered the inotify event. In particular, there is no
> easy way for a process that is monitoring events via inotify to
> distinguish events that it triggers itself from those that are
> triggered by other processes.
>
> The inotify API identifies affected files by filename. However,
> by the time an application processes an inotify event, the fileâ
> name may already have been deleted or renamed.
>
> The inotify API identifies events via watch descriptors. It is
> the application's responsibility to cache a mapping (if one is
> needed) between watch descriptors and pathnames. Be aware that
> directory renamings may affect multiple cached pathnames.
>
> Inotify monitoring of directories is not recursive: to monitor
> subdirectories under a directory, additional watches must be creâ
> ated. This can take a significant amount time for large direcâ
> tory trees.
>
> If monitoring an entire directory subtree, and a new subdirectory
> is created in that tree or an existing directory is renamed into
> that tree, be aware that by the time you create a watch for the
> new subdirectory, new files (and subdirectories) may already
> exist inside the subdirectory. Therefore, you may want to scan
> the contents of the subdirectory immediately after adding the
> watch (and, if desired, recursively add watches for any subdirecâ
> tories that it contains).
>
> Note that the event queue can overflow. In this case, events are
> lost. Robust applications should handle the possibility of lost
> events gracefully. For example, it may be necessary to rebuild
> part or all of the application cache. (One simple, but possibly
> expensive, approach is to close the inotify file descriptor,
> empty the cache, create a new inotify file descriptor, and then
> re-create watches and cache entries for the objects to be moniâ
> tored.)
>
> Dealing with rename() events
> The IN_MOVED_FROM and IN_MOVED_TO events that are generated by
> rename(2) are usually available as consecutive events when readâ
> ing from the inotify file descriptor. However, this is not guarâ
> anteed. If multiple processes are triggering events for moniâ
> tored objects, then (on rare occasions) an arbitrary number of
> other events may appear between the IN_MOVED_FROM and IN_MOVED_TO
> events.
>
> Matching up the IN_MOVED_FROM and IN_MOVED_TO event pair generâ
> ated by rename(2) is thus inherently racy. (Don't forget that if
> an object is renamed outside of a monitored directory, there may
> not even be an IN_MOVED_TO event.) Heuristic approaches (e.g.,
> assume the events are always consecutive) can be used to ensure a
> match in most cases, but will inevitably miss some cases, causing
> the application to perceive the IN_MOVED_FROM and IN_MOVED_TO
> events as being unrelated. If watch descriptors are destroyed
> and re-created as a result, then those watch descriptors will be
> inconsistent with the watch descriptors in any pending events.
> (Re-creating the inotify file descriptor and rebuilding the cache
> may be useful to deal with this scenario.)
>
> Applications should also allow for the possibility that the
> IN_MOVED_FROM event was the last event that could fit in the bufâ
> fer returned by the current call to read(2), and the accompanying
> IN_MOVED_TO event might be fetched only on the next read(2).
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/