Re: File system for scratch space (in HPC cluster)

[Paul Menzel]

Dear Boaz, dear Theodore,


Thank you for your replies.

On 2019-10-24 22:34, Theodore Y. Ts'o wrote:
> On Thu, Oct 24, 2019 at 06:01:05PM +0300, Boaz Harrosh wrote:
>>> You could use ext4 in nojournal mode.  If you want to make sure that
>>> fsync() doesn't force a cache flush, you can mount with the nobarrier
>>> mount option.

Yeah, that is the current settings we use.

>> And open the file with O_TMPFILE|O_EXCL so there is no metadata as well.
> 
> O_TMPFILE means that there is no directory entry created.  The
> pathname passed to the open system call is the directory specifying
> the file system where the temporary file will be created.

Interesting.

The main problem is, that we canât control what the users put into the
cluster, so a mount option is needed.

> This may or may not be what the original poster wanted, depending on
> whether by "scratch file" he meant a file which could be opened by
> pathname by another, subsequent process or not.

Yeah, the scientists send often scripts, where they access the files
by subsequent processes.


Kind regards,

Paul

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature