Re: zram: per-cpu compression streams

[Sergey Senozhatsky]

Hello Minchan,
sorry for long reply.

On (03/28/16 12:21), Minchan Kim wrote:
[..]
> group_reporting
> buffer_compress_percentage=50
> filename=/dev/zram0
> loops=10

I used a bit different script. no `buffer_compress_percentage' option,
because it provide "a mix of random data and zeroes"

buffer_compress_percentage=int
    If this is set, then fio will attempt to provide IO buffer content
    (on WRITEs) that compress to the specified level. Fio does this by
    providing a mix of random data and zeroes

and I also used scramble_buffers=0. but default scramble_buffers is
true, so

scramble_buffers=bool
    If refill_buffers is too costly and the target is using data
    deduplication, then setting this option will slightly modify the IO
    buffer contents to defeat normal de-dupe attempts. This is not
    enough to defeat more clever block compression attempts, but it will
    stop naive dedupe of blocks. Default: true.

hm, but I guess it's not enough; fio probably will have different
data (well, only if we didn't ask it to zero-fill the buffers) for
different tests, causing different zram->zsmalloc behaviour. need
to check it.


> Hmm, Could you retest to who how the benefit is big?

sure. the results are:

- seq-read
- rand-read
- seq-write
- rand-write  (READ + WRITE)
- mixed-seq
- mixed-rand  (READ + WRITE)

TEST        4 streams     8 streams       per-cpu

#jobs1               	           	           
READ:      2665.4MB/s	 2515.2MB/s	 2632.4MB/s
READ:      2258.2MB/s	 2055.2MB/s	 2166.2MB/s
WRITE:     933180KB/s	 894260KB/s	 898234KB/s
WRITE:     765576KB/s	 728154KB/s	 746396KB/s
READ:      563169KB/s	 541004KB/s	 551541KB/s
WRITE:     562660KB/s	 540515KB/s	 551043KB/s
READ:      493656KB/s	 477990KB/s	 488041KB/s
WRITE:     493210KB/s	 477558KB/s	 487600KB/s
#jobs2               	           	           
READ:      5116.7MB/s	 4607.1MB/s	 4401.5MB/s
READ:      4401.5MB/s	 3993.6MB/s	 3831.6MB/s
WRITE:     1539.9MB/s	 1425.5MB/s	 1600.0MB/s
WRITE:     1311.1MB/s	 1228.7MB/s	 1380.6MB/s
READ:      1001.8MB/s	 960799KB/s	 989.63MB/s
WRITE:     998.31MB/s	 957540KB/s	 986.26MB/s
READ:      921439KB/s	 860387KB/s	 899720KB/s
WRITE:     918314KB/s	 857469KB/s	 896668KB/s
#jobs3               	           	           
READ:      6670.9MB/s	 6469.9MB/s	 6548.8MB/s
READ:      5743.4MB/s	 5507.8MB/s	 5608.4MB/s
WRITE:     1923.8MB/s	 1885.9MB/s	 2191.9MB/s
WRITE:     1622.4MB/s	 1605.4MB/s	 1842.2MB/s
READ:      1277.3MB/s	 1295.8MB/s	 1395.2MB/s
WRITE:     1276.9MB/s	 1295.4MB/s	 1394.7MB/s
READ:      1152.6MB/s	 1137.1MB/s	 1216.6MB/s
WRITE:     1152.2MB/s	 1137.6MB/s	 1216.2MB/s
#jobs4               	           	           
READ:      8720.4MB/s	 7301.7MB/s	 7896.2MB/s
READ:      7510.3MB/s	 6690.1MB/s	 6456.2MB/s
WRITE:     2211.6MB/s	 1930.8MB/s	 2713.9MB/s
WRITE:     2002.2MB/s	 1629.8MB/s	 2227.7MB/s
READ:      1657.8MB/s	 1437.1MB/s	 1765.8MB/s
WRITE:     1651.7MB/s	 1432.7MB/s	 1759.3MB/s
READ:      1467.7MB/s	 1201.7MB/s	 1523.5MB/s
WRITE:     1462.3MB/s	 1197.3MB/s	 1517.9MB/s
#jobs5               	           	           
READ:      7791.9MB/s	 6852.7MB/s	 7487.9MB/s
READ:      6214.6MB/s	 6449.6MB/s	 7106.5MB/s
WRITE:     2017.9MB/s	 1978.1MB/s	 2221.5MB/s
WRITE:     1913.1MB/s	 1664.9MB/s	 1985.8MB/s
READ:      1417.6MB/s	 1447.7MB/s	 1558.8MB/s
WRITE:     1419.8MB/s	 1449.3MB/s	 1561.2MB/s
READ:      1336.9MB/s	 1234.1MB/s	 1404.7MB/s
WRITE:     1338.2MB/s	 1236.9MB/s	 1406.8MB/s
#jobs6               	           	           
READ:      8680.9MB/s	 8500.0MB/s	 7116.3MB/s
READ:      7329.4MB/s	 6580.7MB/s	 6476.2MB/s
WRITE:     2121.4MB/s	 1918.6MB/s	 2472.8MB/s
WRITE:     1936.8MB/s	 1826.9MB/s	 2106.8MB/s
READ:      1559.9MB/s	 1506.3MB/s	 1643.6MB/s
WRITE:     1554.7MB/s	 1501.2MB/s	 1637.2MB/s
READ:      1459.7MB/s	 1258.9MB/s	 1502.6MB/s
WRITE:     1454.8MB/s	 1254.6MB/s	 1497.5MB/s
#jobs7               	           	           
READ:      9170.0MB/s	 7905.2MB/s	 8043.9MB/s
READ:      6412.7MB/s	 6792.7MB/s	 6457.8MB/s
WRITE:     2042.4MB/s	 1972.5MB/s	 2400.6MB/s
WRITE:     1938.8MB/s	 1808.7MB/s	 2152.6MB/s
READ:      1634.9MB/s	 1505.8MB/s	 1746.4MB/s
WRITE:     1640.1MB/s	 1511.4MB/s	 1753.7MB/s
READ:      1407.9MB/s	 1239.1MB/s	 1480.8MB/s
WRITE:     1413.8MB/s	 1245.2MB/s	 1486.1MB/s
#jobs8               	           	           
READ:      8563.4MB/s	 8106.7MB/s	 7696.3MB/s
READ:      6909.1MB/s	 5790.5MB/s	 6537.7MB/s
WRITE:     2040.3MB/s	 2061.2MB/s	 2481.7MB/s
WRITE:     1993.5MB/s	 1859.4MB/s	 2171.5MB/s
READ:      1691.6MB/s	 1585.8MB/s	 1749.9MB/s
WRITE:     1686.3MB/s	 1580.1MB/s	 1744.5MB/s
READ:      1478.3MB/s	 1365.4MB/s	 1529.2MB/s
WRITE:     1473.5MB/s	 1361.2MB/s	 1525.3MB/s
#jobs9               	           	           
READ:      9272.8MB/s	 11769MB/s	 8052.7MB/s
READ:      6954.6MB/s	 6375.0MB/s	 7116.3MB/s
WRITE:     2170.3MB/s	 1961.6MB/s	 2487.9MB/s
WRITE:     1865.9MB/s	 1974.2MB/s	 2233.6MB/s
READ:      1663.9MB/s	 1561.1MB/s	 1719.1MB/s
WRITE:     1662.3MB/s	 1560.6MB/s	 1718.4MB/s
READ:      1515.6MB/s	 1379.2MB/s	 1546.2MB/s
WRITE:     1514.2MB/s	 1377.8MB/s	 1544.8MB/s
#jobs10              	           	           
READ:      8737.3MB/s	 7950.4MB/s	 7915.5MB/s
READ:      6884.9MB/s	 6332.8MB/s	 6334.3MB/s
WRITE:     2077.9MB/s	 2005.4MB/s	 2509.5MB/s
WRITE:     1893.3MB/s	 1935.9MB/s	 2155.4MB/s
READ:      1519.2MB/s	 1548.1MB/s	 1622.9MB/s
WRITE:     1519.5MB/s	 1549.2MB/s	 1623.2MB/s
READ:      1383.4MB/s	 1385.2MB/s	 1437.6MB/s
WRITE:     1383.6MB/s	 1385.4MB/s	 1437.8MB/s


	-ss