Re: [PATCH] lib/lz4: add KUnit tests for compression and decompression
From: Michal Wilczynski
Date: Thu Nov 19 2026 - 14:43:57 EST
On 11/18/26 19:23, S. Biplab Raut wrote:
> lib/lz4 has no in-tree tests, and this series replaces both the
> compressor and the decompressor. Add KUnit suites for the behaviour
> callers rely on.
>
> lz4_decompress:
> - round-trips through LZ4_decompress_safe() and
> LZ4_decompress_safe_partial() with exact, oversized and partial
> capacities;
> - truncated and corrupted blocks never write past the destination,
> through either entry point. The input sits at the end of its
> allocation, so KASAN also reports over-reads;
> - an offset-0 match decodes the same whatever the buffer held before;
> - literal and match lengths longer than the output are rejected. On
> 32-bit, lengths whose extension bytes sum to SIZE_MAX are rejected
> rather than wrapped;
> - in-place decoding within LZ4_DECOMPRESS_INPLACE_MARGIN() leaves the
> output intact. A fixed block that corrupts with the old +32 margin
> covers the vendored decoder's 32-byte wild copies, and a port of
> upstream's worst-case test covers the 64-byte copies its decoder
> makes since v1.11.0.
>
> lz4_compress:
> - LZ4_compress_default() fits LZ4_COMPRESSBOUND(), and with a smaller
> buffer fails rather than overflowing;
> - LZ4_compress_fast() across documented acceleration boundaries;
> - LZ4_compress_destSize() output decodes to exactly the consumed
> prefix;
> - the dictionary paths of zram's lz4 and lz4hc backends;
> - LZ4_compress_HC() across supported levels, including the kernel's
> clamp boundary, with full and limited output.
>
> Work memory is allocated at exactly LZ4_MEM_COMPRESS, LZ4HC_MEM_COMPRESS
> and LZ4_MEM_DECOMPRESS bytes, so KASAN reports an implementation that
> outgrows them.
>
> Without this series, the 32-bit length case fails on i386 ("long match
> accepted: ret=31"): the old decoder wraps the match length to 18 bytes
> and accepts the block.
>
> Signed-off-by: S. Biplab Raut <Biplab.Raut@xxxxxxx>
> ---
>
> Michal, this goes on top of your v2: it uses LZ4_MEM_DECOMPRESS and the
> opaque stream types, so please fold it into v3 if you find it useful.
>
> The tests are deterministic regression tests. Every randomized case
> uses a fixed seed, so each run checks the same inputs and any failure
> reproduces exactly. The in-place suite has two fixed cases:
>
> - a block that corrupts with the old (compressedSize >> 8) + 32 margin
> under a 32-byte fast loop, the case patch 12 fixes;
>
> - a port of upstream's decompress_safe_inPlace_worstCase test [1]:
> fixed noise with a 65-byte match at offset 65, for every source size
> from 300 to 4096.
>
> On patch 12 and the +65 margin: since v1.11.0, upstream's decoder makes
> 64-byte wild copies on x86-64, aarch64 and ppc64, and upstream has raised
> its margin to (compressedSize >> 8) + 65 to match [1]. That fix is on
> the dev branch for v1.11.2; the released v1.11.0 and v1.11.1 headers
> still say +32. Your v1.10.0 import uses the 32-byte loop, so +36 is
> right for this series. A later re-sync to v1.11.0 or newer needs the
> kernel's margin raised to +65 in the same patch, whatever the upstream
> header at that tag says.
>
> I checked that the worst-case test catches this, using a userspace
> build of the same test. Against upstream dev's decoder, all 3797 sizes
> fail at +36 and none fail at +65. Against your vendored v1.10.0
> decoder, none fail at +36.
>
> The HC cases are built only when LZ4HC_COMPRESS is reachable.
>
> With your v2 applied on v7.3-rc4, the suite passes on UML (17 passed,
> one 32-bit-only skip), on i386 (18 passed), and on x86_64 with KASAN and
> KASAN_VMALLOC (17 passed, the same skip). A UML build without
> LZ4HC_COMPRESS passes 15 tests with the same skip.
>
> [1] https://protect2.fireeye.com/v1/url?k=f02093a2-91ab868d-f02118ed-74fe485cbfe7-df7f89af9c73957c&q=1&e=79e95ae8-ccd9-4aaa-b235-c5f62451957b&u=https%3A%2F%2Fgithub.com%2Flz4%2Flz4%2Fpull%2F2104
>
> lib/Kconfig | 15 +
> lib/lz4/.kunitconfig | 5 +
> lib/lz4/Makefile | 1 +
> lib/lz4/tests/Makefile | 2 +
> lib/lz4/tests/lz4_kunit.c | 911 ++++++++++++++++++++++++++++++++++++++
> 5 files changed, 934 insertions(+)
> create mode 100644 lib/lz4/.kunitconfig
> create mode 100644 lib/lz4/tests/Makefile
> create mode 100644 lib/lz4/tests/lz4_kunit.c
>
Hi Biplab,
Thanks for the tests and for testing v2
v3 is coming in a couple of days. It vendors v1.11.1 instead of v1.10.0,
so it already has the 64-byte copies, and the in-place margin is
(compressedSize >> 8) + 68.
I ran your suite on v3. Everything passes except
lz4hc_compress_test_levels. That's because v3 no longer clamps HC levels
to 9: levels 10-12 need a bigger state, sized with the new
lz4hc_mem_compress(level). Allocating wrkmem per level with it fixes the
test.
If that's OK with you, I'll carry your patch in v3 with that small
change keeping you as the author. Since patches 10, 11 and 13 changed
a lot I'll drop your Tested-by - a re-test on v3 would be great.
Best regards,
--
Michal Wilczynski <m.wilczynski@xxxxxxxxxxx>