Re: DAX can not work on virtual nvdimm device

From: Ross Zwisler
Date: Thu Sep 01 2016 - 22:57:49 EST


On Wed, Aug 31, 2016 at 04:44:47PM +0800, Xiao Guangrong wrote:
> On 08/31/2016 01:09 AM, Dan Williams wrote:
> >
> > Can you post your exact reproduction steps? This test is not failing for me.
> >
>
> Sure.
>
> 1. make the guest kernel based on your tree, the top commit is
> 10d7902fa0e82b (dax: unmap/truncate on device shutdown) and
> the config file can be found in this thread.
>
> 2. add guest kernel command line: memmap=6G!10G
>
> 3: start the guest:
> x86_64-softmmu/qemu-system-x86_64 -machine pc,nvdimm --enable-kvm \
> -smp 16 -m 32G,maxmem=100G,slots=100 /other/VMs/centos6.img -monitor stdio
>
> 4: in guest:
> mkfs.ext4 /dev/pmem0
> mount -o dax /dev/pmem0 /mnt/pmem/
> echo > /mnt/pmem/xxx
> ./mmap /mnt/pmem/xxx
> ./read /mnt/pmem/xxx
>
> The source code of mmap and read has been attached in this mail.
>
> Hopefully, you can detect the error triggered by read test.
>
> Thanks!

Okay, I think I've isolated this issue. Xiao's VM was an old CentOS 6 system,
and for some reason ext4+DAX with the old tools found in that VM fails. I was
able to reproduce this failure with a freshly installed CentOS 6.8 VM.

You can see the failure with his tests, or perhaps more easily with this
series of commands:

# mkfs.ext4 /dev/pmem0
# mount -o dax /dev/pmem0 /mnt/pmem/
# touch /mnt/pmem/x
# md5sum /mnt/pmem/x
md5sum: /mnt/pmem/x: Bad address

This sequence of commands works fine in the old CentOS 6 system if you use XFS
instead of ext4, and it works fine with both ext4 and XFS in CentOS 7 and
with recent versions of Fedora.

I've added the ext4 folks to this mail in case they care, but my guess is that
the tools in CentOS 6 are so old that it's not worth worrying about. For
reference, the kernel in CentOS 6 is based on 2.6.32. :) DAX was introduced
in v4.0.