Re: [REGRESSION] tmpfs: flistxattr(fd) small-list slowdown after simple_xattrs rhashtable conversion

[Jan Kara]

Hi!

On Mon 06-07-26 16:53:28, Chengfeng Lin wrote:
> I found a small-list flistxattr(fd) slowdown on tmpfs while looking at FS
> performance regressions.  This is a performance regression report.
> 
> The scope is intentionally narrow:
> 
>   tmpfs + user xattrs + flistxattr(fd) + small xattr lists
> 
> The cleanest reproducer creates a tmpfs file, sets one or more `user.*`
> xattrs, and repeatedly measures:
> 
>   flistxattr(fd, list, list_size)
> 
> The single-xattr scenario checks that the returned list is non-empty; the
> count-gradient scenario also checks the exact returned list length.
> 
> The bare-metal test machine is an x86-64 Intel Core i7-14700 system with 28
> logical CPUs and 30 GiB RAM.  The benchmark was pinned to CPU 2, and the test
> filesystem was tmpfs.
> 
> On bare metal, the parent/child A/B around:
> 
>   52b364fed6e1 shmem: adapt to rhashtable-based simple_xattrs with
> lazy allocation
> 
> shows a clear jump.  The parent has the rhashtable infrastructure but tmpfs
> still uses the old rbtree path.  The child switches tmpfs to the lazy
> rhashtable-based simple_xattrs path.  I understand that the lazy allocation
> part is meant to avoid per-inode rhashtable overhead for tmpfs inodes that do
> not use xattrs; this report focuses on the small-list listxattr cost for tmpfs
> inodes that already have user xattrs.
> 
> With one xattr, skipping the first round:
> 
>   kernel / state                         flistxattr_ns_per_op
>   parent A, tmpfs old rbtree path         135.754
>   child,  tmpfs lazy rhashtable path      229.663
>   parent B, tmpfs old rbtree path         135.335
> 
> That is about +69.4% for the child vs the average of the two parent runs.
> The two parent runs agree closely, and the skip-first-round ranges were:
> 
>   parent A: 134.451..140.153 ns/op
>   child:   217.758..239.498 ns/op
>   parent B: 134.273..138.098 ns/op

Thanks for report! Couple of questions:

1) Can you still observe a difference in your benchmark between 7.0 kernel
and kernel after commit 1e7cd8a53b72 ("simpe_xattr: use per-sb cache").
Because that was exactly addressing performance regression for small number
of xattrs.

2) If yes, we need to better understand where the time is spent but we need
to analyse profiles after commit 1e7cd8a53b72 because that significanly
changed how things behave...

								Honza
-- 
Jan Kara <jack@xxxxxxxx>
SUSE Labs, CR