udevd is killing file write performance.

[Robin Holt]

I have a customer application which sees a significant performance
degradation when run with udevd running.  This appears to be due to:


void inotify_dentry_parent_queue_event(struct dentry *dentry, u32 mask,
                                       u32 cookie, const char *name)
{
...
        if (!atomic_read (&inotify_watches))
                return;

        spin_lock(&dentry->d_lock);

The particular application is a 512 thread application.  When run with
udevd turned off the application finishes in about 6 seconds.  With it
turned on, the appliction takes minutes (I think we timed it to around
22 minutes, but we have not been patient enough to wait it through to
the end in some time).  This is being run on a 512cpu system, but there
is a noticable performance hit on smaller systems as well.

As far as I can tell, the application has multiple threads writing
different portions of the same file, but the customer is still working
on providing a simplified test case.

I know _VERY_ little about filesystems.  udevd appears to be looking
at /etc/udev/rules.d.  This bumps inotify_watches to 1.  The file
being written is on an xfs filesystem mounted at a different mountpoint.
Could the inotify flag be moved from a global to a sb (or something
finer) point and therefore avoid taking the dentry->d_lock when there
is no possibility of a watch event being queued.

Thanks,
Robin Holt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/