Re: zram: per-cpu compression streams

From: Sergey Senozhatsky
Date: Wed Mar 30 2016 - 04:35:12 EST

Hello Minchan,
sorry for long reply.

On (03/28/16 12:21), Minchan Kim wrote:
> group_reporting
> buffer_compress_percentage=50
> filename=/dev/zram0
> loops=10

I used a bit different script. no `buffer_compress_percentage' option,
because it provide "a mix of random data and zeroes"

If this is set, then fio will attempt to provide IO buffer content
(on WRITEs) that compress to the specified level. Fio does this by
providing a mix of random data and zeroes

and I also used scramble_buffers=0. but default scramble_buffers is
true, so

If refill_buffers is too costly and the target is using data
deduplication, then setting this option will slightly modify the IO
buffer contents to defeat normal de-dupe attempts. This is not
enough to defeat more clever block compression attempts, but it will
stop naive dedupe of blocks. Default: true.

hm, but I guess it's not enough; fio probably will have different
data (well, only if we didn't ask it to zero-fill the buffers) for
different tests, causing different zram->zsmalloc behaviour. need
to check it.

> Hmm, Could you retest to who how the benefit is big?

sure. the results are:

- seq-read
- rand-read
- seq-write
- rand-write (READ + WRITE)
- mixed-seq
- mixed-rand (READ + WRITE)

TEST 4 streams 8 streams per-cpu

READ: 2665.4MB/s 2515.2MB/s 2632.4MB/s
READ: 2258.2MB/s 2055.2MB/s 2166.2MB/s
WRITE: 933180KB/s 894260KB/s 898234KB/s
WRITE: 765576KB/s 728154KB/s 746396KB/s
READ: 563169KB/s 541004KB/s 551541KB/s
WRITE: 562660KB/s 540515KB/s 551043KB/s
READ: 493656KB/s 477990KB/s 488041KB/s
WRITE: 493210KB/s 477558KB/s 487600KB/s
READ: 5116.7MB/s 4607.1MB/s 4401.5MB/s
READ: 4401.5MB/s 3993.6MB/s 3831.6MB/s
WRITE: 1539.9MB/s 1425.5MB/s 1600.0MB/s
WRITE: 1311.1MB/s 1228.7MB/s 1380.6MB/s
READ: 1001.8MB/s 960799KB/s 989.63MB/s
WRITE: 998.31MB/s 957540KB/s 986.26MB/s
READ: 921439KB/s 860387KB/s 899720KB/s
WRITE: 918314KB/s 857469KB/s 896668KB/s
READ: 6670.9MB/s 6469.9MB/s 6548.8MB/s
READ: 5743.4MB/s 5507.8MB/s 5608.4MB/s
WRITE: 1923.8MB/s 1885.9MB/s 2191.9MB/s
WRITE: 1622.4MB/s 1605.4MB/s 1842.2MB/s
READ: 1277.3MB/s 1295.8MB/s 1395.2MB/s
WRITE: 1276.9MB/s 1295.4MB/s 1394.7MB/s
READ: 1152.6MB/s 1137.1MB/s 1216.6MB/s
WRITE: 1152.2MB/s 1137.6MB/s 1216.2MB/s
READ: 8720.4MB/s 7301.7MB/s 7896.2MB/s
READ: 7510.3MB/s 6690.1MB/s 6456.2MB/s
WRITE: 2211.6MB/s 1930.8MB/s 2713.9MB/s
WRITE: 2002.2MB/s 1629.8MB/s 2227.7MB/s
READ: 1657.8MB/s 1437.1MB/s 1765.8MB/s
WRITE: 1651.7MB/s 1432.7MB/s 1759.3MB/s
READ: 1467.7MB/s 1201.7MB/s 1523.5MB/s
WRITE: 1462.3MB/s 1197.3MB/s 1517.9MB/s
READ: 7791.9MB/s 6852.7MB/s 7487.9MB/s
READ: 6214.6MB/s 6449.6MB/s 7106.5MB/s
WRITE: 2017.9MB/s 1978.1MB/s 2221.5MB/s
WRITE: 1913.1MB/s 1664.9MB/s 1985.8MB/s
READ: 1417.6MB/s 1447.7MB/s 1558.8MB/s
WRITE: 1419.8MB/s 1449.3MB/s 1561.2MB/s
READ: 1336.9MB/s 1234.1MB/s 1404.7MB/s
WRITE: 1338.2MB/s 1236.9MB/s 1406.8MB/s
READ: 8680.9MB/s 8500.0MB/s 7116.3MB/s
READ: 7329.4MB/s 6580.7MB/s 6476.2MB/s
WRITE: 2121.4MB/s 1918.6MB/s 2472.8MB/s
WRITE: 1936.8MB/s 1826.9MB/s 2106.8MB/s
READ: 1559.9MB/s 1506.3MB/s 1643.6MB/s
WRITE: 1554.7MB/s 1501.2MB/s 1637.2MB/s
READ: 1459.7MB/s 1258.9MB/s 1502.6MB/s
WRITE: 1454.8MB/s 1254.6MB/s 1497.5MB/s
READ: 9170.0MB/s 7905.2MB/s 8043.9MB/s
READ: 6412.7MB/s 6792.7MB/s 6457.8MB/s
WRITE: 2042.4MB/s 1972.5MB/s 2400.6MB/s
WRITE: 1938.8MB/s 1808.7MB/s 2152.6MB/s
READ: 1634.9MB/s 1505.8MB/s 1746.4MB/s
WRITE: 1640.1MB/s 1511.4MB/s 1753.7MB/s
READ: 1407.9MB/s 1239.1MB/s 1480.8MB/s
WRITE: 1413.8MB/s 1245.2MB/s 1486.1MB/s
READ: 8563.4MB/s 8106.7MB/s 7696.3MB/s
READ: 6909.1MB/s 5790.5MB/s 6537.7MB/s
WRITE: 2040.3MB/s 2061.2MB/s 2481.7MB/s
WRITE: 1993.5MB/s 1859.4MB/s 2171.5MB/s
READ: 1691.6MB/s 1585.8MB/s 1749.9MB/s
WRITE: 1686.3MB/s 1580.1MB/s 1744.5MB/s
READ: 1478.3MB/s 1365.4MB/s 1529.2MB/s
WRITE: 1473.5MB/s 1361.2MB/s 1525.3MB/s
READ: 9272.8MB/s 11769MB/s 8052.7MB/s
READ: 6954.6MB/s 6375.0MB/s 7116.3MB/s
WRITE: 2170.3MB/s 1961.6MB/s 2487.9MB/s
WRITE: 1865.9MB/s 1974.2MB/s 2233.6MB/s
READ: 1663.9MB/s 1561.1MB/s 1719.1MB/s
WRITE: 1662.3MB/s 1560.6MB/s 1718.4MB/s
READ: 1515.6MB/s 1379.2MB/s 1546.2MB/s
WRITE: 1514.2MB/s 1377.8MB/s 1544.8MB/s
READ: 8737.3MB/s 7950.4MB/s 7915.5MB/s
READ: 6884.9MB/s 6332.8MB/s 6334.3MB/s
WRITE: 2077.9MB/s 2005.4MB/s 2509.5MB/s
WRITE: 1893.3MB/s 1935.9MB/s 2155.4MB/s
READ: 1519.2MB/s 1548.1MB/s 1622.9MB/s
WRITE: 1519.5MB/s 1549.2MB/s 1623.2MB/s
READ: 1383.4MB/s 1385.2MB/s 1437.6MB/s
WRITE: 1383.6MB/s 1385.4MB/s 1437.8MB/s