Re: Why is O_DSYNC on linux so slow / what's wrong with my SSD?

[Stefan Priebe]

Hi Ric,

Am 23.11.2013 20:35, schrieb Ric Wheeler:
> On 11/23/2013 01:27 PM, Stefan Priebe wrote:
> > Hi Ric,
> >
> > Am 22.11.2013 21:37, schrieb Ric Wheeler:
> > > On 11/22/2013 03:01 PM, Stefan Priebe wrote:
> > > > Hi Christoph,
> > > > Am 21.11.2013 11:11, schrieb Christoph Hellwig:
> > > > > > 2. Some drives may implement CMD_FLUSH to return immediately i.e. no
> > > > > > guarantee the data is actually on disk.
> > > > >
> > > > > In which case they aren't spec complicant.  While I've seen countless
> > > > > data integrity bugs on lower end ATA SSDs I've not seen one that
> > > > > simpliy
> > > > > ingnores flush.  If you'd want to cheat that bluntly you'd be better
> > > > > of just claiming to not have a writeback cache.
> > > > >
> > > > > You solve your performance problem by completely disabling any chance
> > > > > of having data integrity guarantees, and do so in a way that is not
> > > > > detectable for applications or users.
> > > > >
> > > > > If you have a workload with lots of small synchronous writes disabling
> > > > > the writeback cache on the disk does indeed often help, especially
> > > > > with
> > > > > the non-queueable FLUSH on all but the most recent ATA devices.
> > > >
> > > > But this isn't correct for drives with capicitors like Crucial m500,
> > > > Intel DC S3500, DC S3700 isn't it? Shouldn't the linux kernel has an
> > > > option to disable this for drives like these?
> > > > /sys/block/sdX/device/ignore_flush
> > >
> > > If you know 100% for sure that your drive has a non-volatile write
> > > cache, you can run the file system without the flushing by mounting "-o
> > > nobarrier".  With most devices, this is not needed since they tend to
> > > simply ignore the flushes if they know they are power failure safe.
> > >
> > > Block level, we did something similar for users who are not running
> > > through a file system for SCSI devices - James added support to echo
> > > "temporary" into the sd's device's cache_type field:
> > >
> > > See:
> > >
> > > https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=2ee3e26c673e75c05ef8b914f54fadee3d7b9c88
> >
> > At least to me this does not work. I get the same awful speed as
> > before - also the I/O waits stay the same. I'm still seeing CMD
> > flushes going to the devices.
> >
> > Is there any way to check whether the temporary got accepted and works?
> >
> > I simply executed:
> > for i in /sys/class/scsi_disk/*/cache_type; do echo $i; echo temporary
> > write back >$i; done
> >
> > Stefan
>
> What kernel are you running?  This is a new addition....

3.10.19

> Also, you can "cat" the same file to see what it says.

Sure:
[cloud2-1338: ~]# for i in /sys/class/scsi_disk/*/cache_type; do echo 
$i; echo temporary write through >$i; done
/sys/class/scsi_disk/1:0:0:0/cache_type
/sys/class/scsi_disk/2:0:0:0/cache_type
/sys/class/scsi_disk/3:0:0:0/cache_type
/sys/class/scsi_disk/4:0:0:0/cache_type
/sys/class/scsi_disk/5:0:0:0/cache_type
/sys/class/scsi_disk/6:0:0:0/cache_type
[cloud2-1338: ~]# for i in /sys/class/scsi_disk/*/cache_type; do echo 
$i; cat $i; done
/sys/class/scsi_disk/1:0:0:0/cache_type
write through
/sys/class/scsi_disk/2:0:0:0/cache_type
write through
/sys/class/scsi_disk/3:0:0:0/cache_type
write through
/sys/class/scsi_disk/4:0:0:0/cache_type
write through
/sys/class/scsi_disk/5:0:0:0/cache_type
write through
/sys/class/scsi_disk/6:0:0:0/cache_type
write through
[cloud2-1338: ~]# for i in /sys/class/scsi_disk/*/cache_type; do echo 
$i; echo temporary write back >$i; done
/sys/class/scsi_disk/1:0:0:0/cache_type
/sys/class/scsi_disk/2:0:0:0/cache_type
/sys/class/scsi_disk/3:0:0:0/cache_type
/sys/class/scsi_disk/4:0:0:0/cache_type
/sys/class/scsi_disk/5:0:0:0/cache_type
/sys/class/scsi_disk/6:0:0:0/cache_type
[cloud2-1338: ~]# for i in /sys/class/scsi_disk/*/cache_type; do echo 
$i; cat $i; done
/sys/class/scsi_disk/1:0:0:0/cache_type
write back
/sys/class/scsi_disk/2:0:0:0/cache_type
write back
/sys/class/scsi_disk/3:0:0:0/cache_type
write back
/sys/class/scsi_disk/4:0:0:0/cache_type
write back
/sys/class/scsi_disk/5:0:0:0/cache_type
write back
/sys/class/scsi_disk/6:0:0:0/cache_type
write back
[cloud2-1338: ~]#

Stefan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/