Re: [PATCH 00/10] erofs: 48-bit layout support

[Gao Xiang]

On 2025/3/13 20:06, Chao Yu wrote:
> On 2025/3/10 17:54, Gao Xiang wrote:
> > Hi folks,
> >
> > The current 32-bit block addressing limits EROFS to a 16TiB maximum
> > volume size with 4KiB blocks.  However, several new use cases now
> > require larger capacity support:
> >   - Massive datasets for model training to boost random sampling
> >     performance for each epoch;
> >   - Object storage clients using EROFS direct passthrough.
> >
> > This extends core on-disk structures to support 48-bit block addressing,
> > such as inodes, device slots, and inode chunks.
> >
> > In addition, it introduces an mtime field to 32-byte compact inodes for
> > basic timestamp support, as well as expands the superblock root NID to
> > an 8-byte rootnid_8b for out-of-place update incremental builds.
> >
> > In order to support larger images beyond 32-bit block addressing and
> > efficient indexing of large compression units for compressed data, and
> > to better support popular compression algorithms (mainly Zstd) that
> > still lack native fixed-sized output compression support, introduce
> > byte-oriented encoded extents, so that these compressors are allowed
> > to retain their current methods.
> >
> > Therefore, it speeds up Zstd image building a lot by using:
> > Processor: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz * 96
> > Dataset: enwik9
> > Build time Size Type Command Line
> > 3m52.339s 266653696 FO -C524288 -zzstd,22
> > 3m48.549s 266174464 FO -E48bit -C524288 -zzstd,22
> > 0m12.821s 272134144 FI -E48bit -C1048576 --max-extent-bytes=1048576 -zzstd,22
> >
> > It has been stress-tested on my local setup for a while without any
> > strange happens.
>
> Cool, good work! For this serial,
>
> Acked-by: Chao Yu <chao@xxxxxxxxxx>
>
> Hoping to grab continuous free slots to check the details, then we can
> change it to rvb status before merge window. :)

Thanks, I don't find any issue so far and users would like to
get 48-bit addressing and quick Zstd compression. It's not a
complex update anyway.

Thanks,
Gao Xiang