Re: [RFC 1/2] fanotify: new event FAN_MODIFY_DIR

[Amir Goldstein]

On Tue, Mar 14, 2017 at 3:46 PM, Filip ÅtÄdronskà <r.lkml@xxxxxxxxxx> wrote:
> Hi,
>
> On Tue, Mar 14, 2017 at 12:40:56PM +0200, Amir Goldstein wrote:
>> An I am very happy that you used filehandles to keep track of objects
>> in your example, because it fits my proposal really well.
>
> you can read more about the rationales in the WIP draft of my thesis
> (especially the chapter Identifying Inodes and its surroundings):
>
>     http://regnarg.cz/tmp/thesis.pdf
>     https://github.com/regnarg/bc
>
> Either way, one can never use pathnames as identifiers because in
> a world with parallel renames on multiple levels of the hierarchy
> (between which no ordering is guaranteed unless they are cross-dir),
> pathnames are not even a well-defined concept.
>
> NB: I used a simple script for testing that does precisely this
> (a lot of parallel within-directory renames at many levels):
>     https://github.com/regnarg/bc/blob/master/experiments/fanotify/parallel_renamist.sh
> It seems like a good stress test for any application that tries to
> do reliable filesystem monitoring.

I agree. not only to filesystem monitoring.
filesystems could use a test like this in general.
It would be good to stress test overlayfs dir rename like this.

>
>> See, if you used my proposed API, you would have had
>>
>>       fan_fd = CHK(fanotify_init(FAN_UNLIMITED_QUEUE| \
>>                              FAN_EVENT_INFO_PARENT | FAN_EVENT_INFO_FH,
>>                              O_RDONLY));
>
> Sure, I'll definitely try to implement your interface as an
> optional backend and mention it in my thesis and definitely
> give it some heavy testing.
>
>> Furthermore, my proposal records the filehandle at the time of the event
>> and therefore, does not have to keep an elevated refcount of the object
>> in the events queue.
>
> That sounds good but from what I've understood,  not all
> filesystems support handles. Is this a concern? (Maybe the
> right answer is to extend handle support...)
>

Very interesting question.
Extending fhandle support is not always easy.

The majority of local file system do support fhandles,
because they are required for exporting a file system over NFS.
see: git grep -lw s_export_op fs/

The reason that some file systems "don't support" fhandles
is usually because they either do not have an efficient way to implement
open_by_handle() or because they do not have a way to encode a unique
and persistent (over reboot) identifer for name_to_handle().

An example of a commonly used file system that currently does not support
file handles is overlayfs.

The interesting thing to note wrt fanotify and overlayfs is that while it is
technically hard to implement open_by_handle() for overlayfs, it is quite
trivial to implement name_to_handle() for the special common case of all
layers on the same underlying fs (e.g. docker).
The trivial implementation (at least for directories) is to return fhandle
of lower most dir in case of lower or merge dir and fhandle of upper dir
otherwise.

I don't think there is any filesystem that implements  name_to_handle()
without implementing open_by_handle(), but technically it is possible.
And for fanotify FAN_EVENT_INFO_FH only the former is required.

But in any case, FAN_EVENT_INFO_FH is optional.
If your userspace implementation can settle with parent fd information
then filesystems that do not support fhandles can use this mode.

Amir.