Re: [PATCH 0/6] drm/nouveau: GSP telemetry via RUSD, and fdinfo telemetry exposure

[lyude]

(just for other nouveau maintainers, e.g. danilo, I'm currently
planning on looking at this during this week. Already started reviewing
some of this, slightly underestimated how much I had to review :)

On Wed, 2026-07-15 at 01:14 +0400, Mohamed Ahmed wrote:
> On the GSP firmware, nouveau reports nothing about the GPU. The GSP
> owns
> the sensors, so the legacy nvkm therm/clk/volt hwmon paths never run,
> and
> monitoring tools have no data to read.
> 
> GSP on later firmware does publish all of it through a stable
> interface.
> It maintains a shared-memory region called "RM User Shared Data"
> (RUSD),
> which the firmware fills with all the relevant sensor readings like
> power,
> temperatures, clocks, and many others. Consuming it needs only two
> internal subdevice controls: INIT_USER_SHARED_DATA to register the
> buffer,
> and SET_DATA_POLL to tell the GSP which data groups to poll and how
> often.
> 
> This series makes nouveau a RUSD client and exposes the data through
> two
> main surfaces:
> 
>   - hwmon, for the values that map onto standard channels: GPU and
> memory
>     temperature, GPU/board/VRAM power, power cap. 
> 
>   - sysfs, under /sys/class/drm/cardN/device/rusd/, for everything
> hwmon
>     has no channel type for: clocks, utilisation, pstate, throttle
> reason,
>     ECC counters, PCIe error counters, and everything exposed to
> hwmon as
>     well. One value per file according to sysfs convention.
> 
> Polling is demand-driven. GSP polls a group only while a client is
> reading
> it, and a group idles off after 10s after the last read. Polling
> costs
> power, so nothing is polled unless someone is looking. A section that
> has
> not been polled yet, or that the board does not support, reads -
> ENODATA.
> 
> The sysfs interface is documented in
> Documentation/ABI/testing/sysfs-driver-nouveau.
> 
> While the original scope was RUSD support only, monitoring tools
> needed
> two extra additions to complete the picture:
> 
>   - VRAM size and usage on sysfs (patch 5): since this series adds a
> sysfs
>     interface for nouveau, it's better to expose VRAM metrics there
> rather
>     than force monitoring apps to pull in libdrm for the GETPARAM
> memory
>     ioctls.
> 
>   - fdinfo (patch 6): nouveau implemented none at all, and monitoring
> apps
>     could not show per-process GPU and memory usage. It now reports
>     per-client memory and per-engine busy time, accounted from the
> DRM
>     scheduler that the EXEC/VM_BIND path already uses.
> 
> Limitations, caveats, and additional considerations or questions:
> 
>   - RUSD needs r570 or newer GSP firmware; r535 does not have the
> needed
>     controls. Pre-GSP and r535 behavior is identical to what it was:
> the
>     rusd/ group is not created, and hwmon keeps its current behavior.
> 
>   - This is all read-only telemetry. Clock, voltage, or power control
> is
>     through other interfaces out of scope of this series.
> 
>   - The sysfs interface exposed here should be stable and valid for
> nova
>     as well. RUSD is stable and only grows more fields with newer GSP
>     firmware (e.g., fan telemetry gets added in r580), so it is
> possible
>     to keep the same interface as we use newer firmware or as we move
> to
>     nova and any potential considerations should be ironed out before
> this
>     gets merged and the interface gets frozen.
> 
>   - amdgpu offers a single binary blob with all the telemetry
> embedded in
>     addition to the sysfs sensor files. This series currently only
> exposes
>     sensor files, but it is possible to add a similar binary for a
> v2.
>     This mainly offers a more efficient way to read the telemetry as
>     clients would read only a few files instead of reading all of
> them.
> 
>   - The last two patches don't depend on RUSD and work on every GPU.
> But
>     they are here because they complete the sensor telemetry story. I
> am
>     happy to split them into their own series if that is preferred.
> 
> Userspace WIP work built on this sysfs surface exists in form of
> nouveau
> backends for nvtop, LACT, and a NVK HUD:
> 
>   - nvtop: https://github.com/mohamexiety/nvtop/tree/nvtop-nouvtop
> 
>   - LACT: https://github.com/mohamexiety/LACT/tree/lnouvct
> 
>   - NVK HUD:
>     https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39898
> 
> None of these are merged yet and exist for testing only. Upstreaming
> will
> begin when this series gets merged and the interface is stable.
> 
> Mohamed Ahmed (6):
>   drm/nouveau/gsp: vendor the RUSD header
>   drm/nouveau/gsp: add RUSD telemetry support
>   drm/nouveau: add GSP hwmon support
>   drm/nouveau: expose RUSD telemetry via sysfs
>   drm/nouveau: expose global VRAM size and usage via sysfs
>   drm/nouveau: expose per-client GPU usage via fdinfo
> 
>  .../ABI/testing/sysfs-driver-nouveau          | 167 ++++++
>  MAINTAINERS                                   |   1 +
>  drivers/gpu/drm/nouveau/Kbuild                |   1 +
>  .../gpu/drm/nouveau/include/nvkm/subdev/gsp.h |  33 ++
>  .../drm/nouveau/include/nvkm/subdev/rusd.h    | 114 ++++
>  .../gpu/drm/nouveau/include/nvrm/nvtypes.h    |   3 +
>  drivers/gpu/drm/nouveau/nouveau_abi16.c       |  16 +
>  drivers/gpu/drm/nouveau/nouveau_chan.h        |   2 +
>  drivers/gpu/drm/nouveau/nouveau_drm.c         |  22 +
>  drivers/gpu/drm/nouveau/nouveau_drv.h         |  29 +
>  drivers/gpu/drm/nouveau/nouveau_exec.c        |   3 +
>  drivers/gpu/drm/nouveau/nouveau_gem.c         |  16 +
>  drivers/gpu/drm/nouveau/nouveau_hwmon.c       | 375 ++++++++++---
>  drivers/gpu/drm/nouveau/nouveau_sched.c       |  87 +++
>  drivers/gpu/drm/nouveau/nouveau_sched.h       |  11 +
>  drivers/gpu/drm/nouveau/nouveau_sysfs.c       | 311 ++++++++++
>  drivers/gpu/drm/nouveau/nouveau_sysfs.h       |  10 +
>  .../drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c |  26 +
>  .../drm/nouveau/nvkm/subdev/gsp/rm/r535/rm.c  |   1 +
>  .../nouveau/nvkm/subdev/gsp/rm/r570/Kbuild    |   1 +
>  .../nvkm/subdev/gsp/rm/r570/nvrm/rusd.h       | 303 ++++++++++
>  .../drm/nouveau/nvkm/subdev/gsp/rm/r570/rm.c  |   1 +
>  .../nouveau/nvkm/subdev/gsp/rm/r570/rusd.c    | 531
> ++++++++++++++++++
>  .../gpu/drm/nouveau/nvkm/subdev/gsp/rm/rm.h   |   7 +
>  24 files changed, 1990 insertions(+), 81 deletions(-)
>  create mode 100644 Documentation/ABI/testing/sysfs-driver-nouveau
>  create mode 100644
> drivers/gpu/drm/nouveau/include/nvkm/subdev/rusd.h
>  create mode 100644 drivers/gpu/drm/nouveau/nouveau_sysfs.c
>  create mode 100644 drivers/gpu/drm/nouveau/nouveau_sysfs.h
>  create mode 100644
> drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/rusd.h
>  create mode 100644
> drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/rusd.c