Re: File system for scratch space (in HPC cluster)

From: Paul Menzel
Date: Fri Oct 25 2019 - 04:35:39 EST

Dear Andreas,

On 2019-10-24 19:51, Andreas Dilger wrote:
> On Oct 24, 2019, at 4:43 AM, Paul Menzel <pmenzel@xxxxxxxxxxxxx>
> wrote:

>> In our cluster, we offer scratch space for temporary files. As
>> these files are temporary, we do not need any safety requirements
>> â especially not those when the system crashes or shuts down. So
>> no `sync` is for example needed.
>> Are there file systems catering to this need? I couldnât find any?
>> Maybe I missed some options for existing file systems.
> How big do you need the scratch filesystem to be? Is it local to
> the node or does it need to be shared between nodes?

In this case local.

> If it needs to be large and shared between nodes then Lustre is
> typically used for this. If it is local and relatively small you
> could consider using tmpfs backed by swab on an NVMe flash device
> (M.2 or U.2, Optane if you can afford it) inside the node.
> That way you get RAM-like performance for many files, with a larger
> capacity than RAM when needed (tmpfs can use swap).
> You might consider to mount a new tmpfs filesystem per job (no
> formatting is needed for tmpfs), and then unmount it when the job is
> done, so that the old files are automatically cleaned up.
That is a good idea, but probably not practical for 10 TB. Out of
curiosity, what is the limit for ârelatively smallâ in your

Kind regards,


Attachment: smime.p7s
Description: S/MIME Cryptographic Signature