Re: [PATCH] lib/xz: Decode LZMA with local range decoder and dictionary state

From: Lasse Collin

Date: Fri Nov 20 2026 - 14:20:04 EST


On 2026-11-13 Paul Menzel wrote:
> lzma_main() and the functions it calls access the range decoder and
> the dictionary through s->rc and s->dict. Every decoded bit stores to
> a uint16_t probability, and every decoded byte stores to dict.buf, and
> the kernel builds with -fno-strict-aliasing. So the compiler has to
> assume that these stores modify *s, and reloads and stores rc.range,
> rc.code, rc.in_pos, dict.pos, dict.full, … around each of them.
>
> Copy both structures to local variables at the start of lzma_main()
> and write them back at the end, and force inlining of the helpers that
> take their address, so that the compiler can keep them in registers.

Something like this seems to be a good idea.

While -fno-strict-aliasing hurts speed a little, it's a minor thing.
Even with strict aliasing, writes to dict->buf[...] are a problem
because uint8_t usually is unsigned char which can alias anything.
Simply adding "restrict" to the pointer arguments of dict_repeat() and
dict_put() reduces decompression time by several percent on some
machines.

I separated the changes into multiple commits so that the effect of
local rc, local dict (or *restrict dict), and various __always_inline
additions are easier to test. I created three temporary branches in the
upstream repository[1]:

optim_paul Your version
optim1 Local rc and "restrict" for dict
optim2 Local rc and local dict (no "restrict")

optim2 is similar to your version. optim1 and optim2 contain a few
other changes too, the first one being "Split state updates".

The results vary between systems:

- On Ryzen 7 9700X with GCC 16.2, the the local rc clearly helps, but
the dict changes don't make much difference in speed.

- On ARM64, the dict changes in both optim1 and optim2 are significant
improvements (tested on NVIDIA GB10 Spark on cfarm.net).

- On x86-64 and Clang 23 with -O2, the rc and dict changes make little
difference.

Could you test and compare optim1/2? I mean primarily the commits
before "Split state updates", but thoughts about the later commits are
welcome too, of course.

I suggest creating a test file with xz --threads=1 --check=none so that
one is measuring only the decompression speed without CRC32. The CRC32
implementation in XZ Embedded is slow.

There is a benchmark program in the repo:

cd userspace
make CFLAGS='-O2 -fno-strict-aliasing' # Or use kernel's flags.
./bench /path/to/testfile.xz m 10 # Multi-call mode (XZ_DYNALLOC)
./bench /path/to/testfile.xz s 10 # Single-call mode (XZ_SINGLE)

The source files aren't completely identical to the versions in the
kernel. If you want to test optim1/optim2 in kernel context, copy
xz_private.h, xz_lzma2.h, and xz_dec_lzma2.c to the kernel tree and
remove XZ_EXTERN:

sed -i 's/XZ_EXTERN //' xz_private.h xz_dec_lzma2.c

It might take several days for me to reply, so please be patient.

Thanks!

[1] https://github.com/tukaani-project/xz-embedded

--
Lasse Collin