Re: [GIT PULL] orangefs: kernel side caching and executable bugfix

[Martin Brandenburg]

Linus,

Here is another copy of the previous request to fix a few problems. I have
rebased my changes on top of the point where Al Viro's recent change to
OrangeFS, among other things, was merged.

(i.e. f7b32e4c021fd788f13f6785e17efbc3eb05b351)

I have also fixed a whitespace error in my own code.

I have also wrapped my tag message at column 72 so it can be indented by
Git without overflowing.

Sorry, I am still learning! I hope I've done everything right now.

The following changes since commit f7b32e4c021fd788f13f6785e17efbc3eb05b351:

  Merge branch 'for-linus' of
git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux (2016-08-02
12:41:13 -0400)

are available in the git repository at:


  https://github.com/martinbrandenburg/linux.git tags/for-linus-v4.8

for you to fetch changes up to 302f0493f0bfaabd6f77ce7bfaa12620abf74948:

  Orangefs: update orangefs.txt (2016-08-02 15:39:14 -0400)

----------------------------------------------------------------
orangefs: kernel side caching and executable bugfix

This allows OrangeFS to utilize the dcache and adds an in kernel
attribute cache. We previously used the user side client for this
purpose.

We see a modest performance increase on small file operations. For
example, without the cache, compiling coreutils takes about 17 minutes.
With the patch and a 50 millisecond timeout for dcache_timeout_msecs and
getattr_timeout_msecs (the default), compiling coreutils takes about
6 minutes 20 seconds. On the same hardware, compiling coreutils on an
xfs filesystem takes 90 seconds. We see similar improvements with mdtest
and a test involving writing, reading, and deleting a large number of
small files.

Interested parties can review more data at the following URL.

https://docs.google.com/spreadsheets/d/1v4aUeppKexIbRMz_Yn9k4eaM3uy2KCaPoe_93YKWOtA/pubhtml

The eventual goal of this is to allow getdents to turn into a
readdirplus to the OrangeFS server. The cache will be filled then, which
should provide a performance benefit to the common case of readdir
followed by getattr on each entry (i.e. ls -l).

This also fixes a bug. When orangefs_inode_permission was added, it did
not collect i_size from the OrangeFS server, since this presses an
unnecessary load on the OrangeFS server. However, it left a case where
i_size is never initialized. Then running an executable could fail.

With this patch, size is always collected to be inserted into the cache.
Thus the bug disappears. If this patch is not accepted during this merge
window, we will send a one-line band-aid for this bug instead.

----------------------------------------------------------------
Martin Brandenburg (5):
      orangefs: Use d_time to avoid excessive lookups
      orangefs: Cache getattr results.
      orangefs: Allow dcache and getattr cache time to be configured.
      orangefs: Change default dcache and getattr timeout to 50 msec.
      orangefs: Account for jiffies wraparound.

Mike Marshall (1):
      Orangefs: update orangefs.txt

 Documentation/filesystems/orangefs.txt | 50 +++++++++++++++++++++++++++++++---
 fs/orangefs/dcache.c                   |  4 +++
 fs/orangefs/inode.c                    |  6 ++--
 fs/orangefs/namei.c                    | 12 ++++++++
 fs/orangefs/orangefs-kernel.h          |  6 +++-
 fs/orangefs/orangefs-mod.c             |  2 ++
 fs/orangefs/orangefs-sysfs.c           | 43 ++++++++++++++++++++++++++++-
 fs/orangefs/orangefs-utils.c           | 38 ++++++++++++++------------
 fs/orangefs/protocol.h                 |  8 ------
 9 files changed, 135 insertions(+), 34 deletions(-)

On Mon, Aug 1, 2016 at 4:12 PM, Martin Brandenburg <martin@xxxxxxxxxxxx> wrote:
> Linus,
>
> The following changes since commit 2dcd0af568b0cf583645c8a317dd12e344b1c72a:
>
>   Linux 4.6 (2016-05-15 15:43:13 -0700)
>
> are available in the git repository at:
>
>   https://github.com/martinbrandenburg/linux.git tags/for-linus
>
> for you to fetch changes up to 858b62694d893218abbf293a3eac16d910cb6a90:
>
>   Orangefs: update orangefs.txt (2016-08-01 14:59:53 -0400)
>
> ----------------------------------------------------------------
> orangefs: kernel side caching and executable bugfix
>
> This allows OrangeFS to utilize the dcache and adds an in kernel attribute
> cache. We previously used the user side client for this purpose.
>
> We see a modest performance increase on small file operations. For example,
> without the cache, compiling coreutils takes about 17 minutes. With the patch
> and a 50 millisecond timeout for dcache_timeout_msecs and getattr_timeout_msecs
> (the default), compiling coreutils takes about 6 minutes 20 seconds. On the
> same hardware, compiling coreutils on an xfs filesystem takes 90 seconds. We
> see similar improvements with mdtest and a test involving writing, reading, and
> deleting a large number of small files.
>
> Interested parties can review more data at the following URL.
>
> https://docs.google.com/spreadsheets/d/1v4aUeppKexIbRMz_Yn9k4eaM3uy2KCaPoe_93YKWOtA/pubhtml
>
> The eventual goal of this is to allow getdents to turn into a readdirplus to
> the OrangeFS server. The cache will be filled then, which should provide a
> performance benefit to the common case of readdir followed by getattr on each
> entry (i.e. ls -l).
>
> This also fixes a bug. When orangefs_inode_permission was added, it did not
> collect i_size from the OrangeFS server, since this presses an unnecessary load
> on the OrangeFS server. However, it left a case where i_size is never
> initialized. Then running an executable could fail.
>
> With this patch, size is always collected to be inserted into the cache. Thus
> the bug disappears. If this patch is not accepted during this merge window, we
> will send a one-line band-aid for this bug instead.
>
> ----------------------------------------------------------------
> Martin Brandenburg (5):
>       orangefs: Use d_time to avoid excessive lookups
>       orangefs: Cache getattr results.
>       orangefs: Allow dcache and getattr cache time to be configured.
>       orangefs: Change default dcache and getattr timeout to 50 msec.
>       orangefs: Account for jiffies wraparound.
>
> Mike Marshall (1):
>       Orangefs: update orangefs.txt
>
>  Documentation/filesystems/orangefs.txt | 50 +++++++++++++++++++++++++++++++---
>  fs/orangefs/dcache.c                   |  4 +++
>  fs/orangefs/inode.c                    |  6 ++--
>  fs/orangefs/namei.c                    | 12 ++++++++
>  fs/orangefs/orangefs-kernel.h          |  6 +++-
>  fs/orangefs/orangefs-mod.c             |  2 ++
>  fs/orangefs/orangefs-sysfs.c           | 43 ++++++++++++++++++++++++++++-
>  fs/orangefs/orangefs-utils.c           | 38 ++++++++++++++------------
>  fs/orangefs/protocol.h                 |  8 ------
>  9 files changed, 135 insertions(+), 34 deletions(-)