Re: [PATCH v2] zram: Optimize LZ4 dictionary compression performance

[Sergey Senozhatsky]

On (26/03/13 02:41), gao xu wrote:
> Calling `LZ4_loadDict()` repeatedly in Zram causes significant overhead
> due to its internal dictionary pre-processing. This commit introduces a
> template stream mechanism to pre-process the dictionary only once when
> the dictionary is initially set or modified. It then efficiently copies
> this state for subsequent compressions.
> 
> Verification Test Items:
> Test Platform: android16-6.12
> 1. Collect Anonymous Page Dataset
> 1) Apply the following patch:
> static bool zram_meta_alloc(struct zram *zram, u64 disksize)
> 	if (!huge_class_size)
> -		huge_class_size = zs_huge_class_size(zram->mem_pool);
> +		huge_class_size = 0;
> 2）Install multiple apps and monkey testing until SwapFree is close to 0.
> 3）Execute the following command to export data:
> dd if=/dev/block/zram0 of=/data/samples/zram_dump.img bs=4K
> 
> 2. Train Dictionary
> Since LZ4 does not have a dedicated dictionary training tool, the zstd
> tool can be used for training[1]. The command is as follows:
> zstd --train /data/samples/* --split=4096 --maxdict=64KB -o /vendor/etc/dict_data
> 
> 3. Test Code
> adb shell "dd if=/data/samples/zram_dump.img of=/dev/test_pattern bs=4096 count=131072 conv=fsync"
> adb shell "swapoff /dev/block/zram0"
> adb shell "echo 1 > /sys/block/zram0/reset"
> adb shell "echo lz4 > /sys/block/zram0/comp_algorithm"
> adb shell "echo dict=/vendor/etc/dict_data   >  /sys/block/zram0/algorithm_params"
> adb shell "echo 6G > /sys/block/zram0/disksize"
> echo "Start Compression"
> adb shell "taskset 80 dd if=/dev/test_pattern of=/dev/block/zram0 bs=4096 count=131072 conv=fsync"
> echo.
> echo "Start Decompression"
> adb shell "taskset 80 dd if=/dev/block/zram0 of=/dev/output_result bs=4096 count=131072 conv=fsync"
> echo "mm_stat:"
> adb shell "cat /sys/block/zram0/mm_stat"
> echo.
> Note: To ensure stable test results, it is best to lock the CPU frequency
> before executing the test.
> 
> LZ4 supports dictionaries up to 64KB. Below are the test results for
> compression rates at various dictionary sizes:
> dict_size          base        patch
>   4 KB          156M/s      219M/s
>   8 KB          136M/s      217M/s
>  16KB           98M/s       214M/s
>  32KB           66M/s       225M/s
>  64KB           38M/s       224M/s
> 
> When an LZ4 compression dictionary is enabled, compression speed is
> negatively impacted by the dictionary's size; larger dictionaries result
> in slower compression. This patch eliminates the influence of dictionary
> size on compression speed, ensuring consistent performance regardless of
> dictionary scale.
> 
> [1] https://github.com/lz4/lz4?tab=readme-ov-file
> 
> Signed-off-by: gao xu <gaoxu2@xxxxxxxxx>

Acked-by: Sergey Senozhatsky <senozhatsky@xxxxxxxxxxxx>