Re: [PATCH] vfs: Delete the associated dentry when deleting a file
From: Oliver Sang
Date: Wed May 22 2024 - 04:51:37 EST
hi, Linus, hi, Yafang Shao,
On Wed, May 15, 2024 at 09:05:24AM -0700, Linus Torvalds wrote:
> Oliver,
> is there any chance you could run this through the test robot
> performance suite? The original full patch at
>
> https://lore.kernel.org/all/20240515091727.22034-1-laoar.shao@xxxxxxxxx/
>
> and it would be interesting if the test robot could see if the patch
> makes any difference on any other loads?
>
we just reported a stress-ng performance improvement by this patch [1]
test robot applied this patch upon
3c999d1ae3 ("Merge tag 'wq-for-6.10' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq")
filesystem is not our team's major domain, so we just made some limited review
of the results, and decided to send out the report FYI.
at first stage, we decided to check below catagories of tests as priority:
stress-ng filesystem
filebench mailserver
reaim fileserver
we also pick sysbench-fileio, blogbench into coverage.
here is a summary.
for stress-ng, besided [1] which was reported, we got below data that are
about this patch comparing to 3c999d1ae3.
either there is no significant performance change, or the change is smaller
than the noise which will make test robot's bisect fail, so these information
is just FYI. and if you have any doubt about any subtests, could you let us know
then we could check further?
(also included some net test results)
12.87 ± 6% -0.6% 12.79 stress-ng.xattr.ops_per_sec
6721 ± 5% +7.5% 7224 ± 27% stress-ng.rawdev.ops_per_sec
9002 ± 7% -8.7% 8217 stress-ng.dirmany.ops_per_sec
8594743 ± 4% -3.0% 8337417 stress-ng.rawsock.ops_per_sec
2056 ± 3% +2.9% 2116 stress-ng.dirdeep.ops_per_sec
4307 ± 21% -6.9% 4009 stress-ng.dir.ops_per_sec
137946 ± 18% +5.8% 145942 stress-ng.fiemap.ops_per_sec
22413006 ± 2% +2.5% 22982512 ± 2% stress-ng.sockdiag.ops_per_sec
286714 ± 2% -3.8% 275876 ± 5% stress-ng.udp-flood.ops_per_sec
82904 ± 46% -31.6% 56716 stress-ng.sctp.ops_per_sec
9853408 -0.3% 9826387 stress-ng.ping-sock.ops_per_sec
84667 ± 12% -26.7% 62050 ± 17% stress-ng.dccp.ops_per_sec
61750 ± 25% -24.2% 46821 ± 38% stress-ng.open.ops_per_sec
583443 ± 3% -3.4% 563822 stress-ng.file-ioctl.ops_per_sec
11919 ± 28% -34.3% 7833 stress-ng.dentry.ops_per_sec
18.59 ± 12% -23.9% 14.15 ± 27% stress-ng.swap.ops_per_sec
246.37 ± 2% +15.9% 285.58 ± 12% stress-ng.aiol.ops_per_sec
7.45 -4.8% 7.10 ± 7% stress-ng.fallocate.ops_per_sec
207.97 ± 7% +5.2% 218.70 stress-ng.copy-file.ops_per_sec
69.87 ± 7% +5.8% 73.93 ± 5% stress-ng.fpunch.ops_per_sec
0.25 ± 21% +24.0% 0.31 stress-ng.inode-flags.ops_per_sec
849.35 ± 6% +1.4% 861.51 stress-ng.mknod.ops_per_sec
926144 ± 4% -5.2% 877558 stress-ng.lease.ops_per_sec
82924 -2.1% 81220 stress-ng.fcntl.ops_per_sec
6.19 ±124% -50.7% 3.05 stress-ng.chattr.ops_per_sec
676.90 ± 4% -1.9% 663.94 ± 5% stress-ng.iomix.ops_per_sec
0.93 ± 6% +5.6% 0.98 ± 7% stress-ng.symlink.ops_per_sec
1703608 -3.8% 1639057 ± 3% stress-ng.eventfd.ops_per_sec
1735861 -0.6% 1726072 stress-ng.sockpair.ops_per_sec
85440 -2.0% 83705 stress-ng.rawudp.ops_per_sec
6198 +0.6% 6236 stress-ng.sockabuse.ops_per_sec
39226 +0.0% 39234 stress-ng.sock.ops_per_sec
1358 +0.3% 1363 stress-ng.tun.ops_per_sec
9794021 -1.7% 9623340 stress-ng.icmp-flood.ops_per_sec
1324728 +0.3% 1328244 stress-ng.epoll.ops_per_sec
146150 -2.0% 143231 stress-ng.rawpkt.ops_per_sec
6381112 -0.4% 6352696 stress-ng.udp.ops_per_sec
1234258 +0.2% 1236738 stress-ng.sockfd.ops_per_sec
23954 -0.1% 23932 stress-ng.sockmany.ops_per_sec
257030 -0.1% 256860 stress-ng.netdev.ops_per_sec
6337097 +0.1% 6341130 stress-ng.flock.ops_per_sec
173212 -0.3% 172728 stress-ng.rename.ops_per_sec
199.69 +0.6% 200.82 stress-ng.sync-file.ops_per_sec
606.57 +0.8% 611.53 stress-ng.chown.ops_per_sec
183549 -0.9% 181975 stress-ng.handle.ops_per_sec
1299 +0.0% 1299 stress-ng.hdd.ops_per_sec
98371066 +0.2% 98571113 stress-ng.lockofd.ops_per_sec
25.49 -4.3% 24.39 stress-ng.ioprio.ops_per_sec
96745191 -1.5% 95333632 stress-ng.locka.ops_per_sec
582.35 +0.1% 582.86 stress-ng.chmod.ops_per_sec
2075897 -2.2% 2029552 stress-ng.getdent.ops_per_sec
60.47 -1.9% 59.34 stress-ng.metamix.ops_per_sec
14161 -0.3% 14123 stress-ng.io.ops_per_sec
23.98 -1.5% 23.61 stress-ng.link.ops_per_sec
27514 +0.0% 27528 stress-ng.filename.ops_per_sec
44955 +1.6% 45678 stress-ng.dnotify.ops_per_sec
160.94 +0.4% 161.51 stress-ng.inotify.ops_per_sec
2452224 +4.0% 2549607 stress-ng.lockf.ops_per_sec
6761 +0.3% 6779 stress-ng.fsize.ops_per_sec
775083 -1.5% 763487 stress-ng.fanotify.ops_per_sec
309124 -4.2% 296285 stress-ng.utime.ops_per_sec
25567 -0.1% 25530 stress-ng.dup.ops_per_sec
1858 +0.9% 1876 stress-ng.procfs.ops_per_sec
105804 -3.9% 101658 stress-ng.access.ops_per_sec
1.04 -1.9% 1.02 stress-ng.chdir.ops_per_sec
82753 -0.3% 82480 stress-ng.fstat.ops_per_sec
681128 +3.7% 706375 stress-ng.acl.ops_per_sec
11892 -0.1% 11875 stress-ng.bind-mount.ops_per_sec
for filebench, similar results, but data is less unstable than stress-ng, which
means for most of them, we regarded them as that they should not be impacted by
this patch.
for reaim/sysbench-fileio/blogbench, the data are quite stable, and we didn't
notice any significant performance changes. we even doubt whether they go
through the code path changed by this patch.
so for these, we don't list full results here.
BTW, besides filesystem tests, this patch is also piped into other performance
test categories such like net, scheduler, mm and others, and since it also goes
into our so-called hourly kernels, it could run by full other performance test
suites which test robot supports. so in following 2-3 weeks, it's still possible
for us to report other results including regression.
thanks
[1] https://lore.kernel.org/all/202405221518.ecea2810-oliver.sang@xxxxxxxxx/
> Thanks,
> Linus
>
> On Wed, 15 May 2024 at 02:17, Yafang Shao <laoar.shao@xxxxxxxxx> wrote:
> >
> > Our applications, built on Elasticsearch[0], frequently create and delete
> > files. These applications operate within containers, some with a memory
> > limit exceeding 100GB. Over prolonged periods, the accumulation of negative
> > dentries within these containers can amount to tens of gigabytes.
> >
> > Upon container exit, directories are deleted. However, due to the numerous
> > associated dentries, this process can be time-consuming. Our users have
> > expressed frustration with this prolonged exit duration, which constitutes
> > our first issue.