[GIT PULL] erofs updates for 5.13-rc1

From: Gao Xiang
Date: Mon Apr 26 2021 - 06:58:14 EST


Hi Linus,

Could you consider this pull request for 5.13-rc1?

In this cycle, we would like to introduce a new feature called big
pcluster so EROFS can compress file data into more than 1 fs block
and different pcluster size can be selected for each (sub-)files by
design. The current EROFS test results on my laptop are [1]:

Testscript: erofs-openbenchmark
Testdata: enwik9 (1000000000 bytes)
________________________________________________________________
| file system | size | seq read | rand read | rand9m read |
|_______________|___________|_ MiB/s __|__ MiB/s __|___ MiB/s ___|
|___erofs_4k____|_556879872_|_ 781.4 __|__ 55.3 ___|___ 25.3 ___|
|___erofs_16k___|_452509696_|_ 864.8 __|_ 123.2 ___|___ 20.8 ___|
|___erofs_32k___|_415223808_|_ 899.8 __|_ 105.8 _*_|___ 16.8 ____|
|___erofs_64k___|_393814016_|_ 906.6 __|__ 66.6 _*_|___ 11.8 ____|
|__squashfs_8k__|_556191744_|_ 64.9 __|__ 19.3 ___|____ 9.1 ____|
|__squashfs_16k_|_502661120_|_ 98.9 __|__ 38.0 ___|____ 9.8 ____|
|__squashfs_32k_|_458784768_|_ 115.4 __|__ 71.6 _*_|___ 10.0 ____|
|_squashfs_128k_|_398204928_|_ 257.2 __|_ 253.8 _*_|___ 10.9 ____|
|____ext4_4k____|____()_____|_ 786.6 __|__ 28.6 ___|___ 27.8 ____|

, which has been verified but I'd like warn it as experimental for
a while. This matchs erofs-utils dev branch and I'll also release
a new userspace version for this later.

Apart from that, several improvements are also included: e.g. complete
a missing case for inplace I/O, optimize endio decompression logic for
non-atomic contexts and support adjustable sliding window size, ...
In addition to those, there are some cleanups as always.

All commits have been tested and have been in linux-next for about 2
weeks and I've just confirmed with 5.12 release. This merges cleanly
with master.

[1] https://lore.kernel.org/r/20210329053654.GA3281654@xxxxxxxxxxxxxxxxxx

Thanks,
Gao Xiang

The following changes since commit a5e13c6df0e41702d2b2c77c8ad41677ebb065b3:

Linux 5.12-rc5 (2021-03-28 15:48:16 -0700)

are available in the Git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs.git tags/erofs-for-5.13-rc1

for you to fetch changes up to 8e6c8fa9f2e95c88a642521a5da19a8e31748846:

erofs: enable big pcluster feature (2021-04-10 03:20:19 +0800)

----------------------------------------------------------------
Changes since last update:

- avoid memory failure when applying rolling decompression;

- optimize endio decompression logic for non-atomic contexts;

- complete a missing case which can be safely selected for inplace
I/O and thus decreasing more memory footprint;

- check unsupported on-disk inode i_format strictly;

- support adjustable lz4 sliding window size to decrease runtime
memory footprint;

- support on-disk compression configurations;

- support big pcluster decompression;

- several code cleanups / spelling correction.

----------------------------------------------------------------
Gao Xiang (15):
erofs: complete a missing case for inplace I/O
erofs: add unsupported inode i_format check
erofs: introduce erofs_sb_has_xxx() helpers
erofs: introduce on-disk lz4 fs configurations
erofs: add on-disk compression configurations
erofs: reserve physical_clusterbits[]
erofs: introduce multipage per-CPU buffers
erofs: introduce physical cluster slab pools
erofs: fix up inplace I/O pointer for big pcluster
erofs: add big physical cluster definition
erofs: adjust per-CPU buffers according to max_pclusterblks
erofs: support parsing big pcluster compress indexes
erofs: support parsing big pcluster compact indexes
erofs: support decompress big pcluster for lz4 backend
erofs: enable big pcluster feature

Huang Jianan (4):
erofs: avoid memory allocation failure during rolling decompression
erofs: use workqueue decompression for atomic contexts only
erofs: use sync decompression for atomic contexts only
erofs: support adjust lz4 history window size

Ruiqi Gong (1):
erofs: Clean up spelling mistakes found in fs/erofs

Yue Hu (1):
erofs: don't use erofs_map_blocks() any more

fs/erofs/Kconfig | 14 ---
fs/erofs/Makefile | 2 +-
fs/erofs/data.c | 19 +---
fs/erofs/decompressor.c | 272 ++++++++++++++++++++++++++++++------------------
fs/erofs/erofs_fs.h | 54 ++++++++--
fs/erofs/inode.c | 7 ++
fs/erofs/internal.h | 86 ++++++++++-----
fs/erofs/pcpubuf.c | 148 ++++++++++++++++++++++++++
fs/erofs/super.c | 148 +++++++++++++++++++++++++-
fs/erofs/utils.c | 12 ---
fs/erofs/zdata.c | 254 +++++++++++++++++++++++++++++---------------
fs/erofs/zdata.h | 14 +--
fs/erofs/zmap.c | 164 +++++++++++++++++++++++------
13 files changed, 887 insertions(+), 307 deletions(-)
create mode 100644 fs/erofs/pcpubuf.c