Re: [RFC PATCH] misc: fgds: enable GPU-NVMe direct I/O via POSIX and io_uring

[Pavel Begunkov]

On 9/10/26 04:57, Li Wang wrote:
> Hi Christian,
>
> > > > > Nor why "fgds" is the name, that's going to be hard to remember, does it
> > > > > stand for something?
> > > > "FGDS" stands for Fast GPUDirect Storage. GPUDirect Storage (GDS) is NVIDIA's
> > > > technology enabling direct I/O between GPU memory and files on NVMe,
> > > > widely used in LLM workloads to bypass CPU overhead.
> > >
> > > That's nvidia's specific solution, but this works on other devices,
> > > right?  Or just for that one platform?
> >
> > That was nvidia's specific and very hacky out of tree solution
> Exactly, I completely agree with your viewpoint. In fact, we previously conducted
> a deep analysis of NVIDIA's GDS implementation code, which was also one of the
> motivations for us to develop fgds.
>
> Please allow me to introduce NVIDIA's GDS implementation briefly:
> NVIDIA introduced two kernel modules: one module called nvidia-fs, and the other module
> is a customized NVMe driver replacing the Linux kernel's default NVMe driver.
> nvidia-fs creates a device file for every NVIDIA GPU on the machine: /dev/nvidia-fs<gpu-id>.

I'm not an nvidia expert, but IIRC they switched to P2PDMA, which
is supported by the upstream nvme driver.

--
Pavel Begunkov