Re: [PATCH] drm/msm/adreno: keep the GPU timestamp monotonic across power collapse

[Connor Abbott]

On Tue, Sep 8, 2026 at 11:49 AM Dmitry Baryshkov
<dmitry.baryshkov@xxxxxxxxxxxxxxxx> wrote:
>
> On Tue, Sep 08, 2026 at 06:26:22PM +0300, Dmitry Baryshkov wrote:
> > On Tue, Sep 08, 2026 at 11:09:17AM -0400, Connor Abbott wrote:
> > > On Tue, Sep 8, 2026 at 10:51 AM Rob Clark <rob.clark@xxxxxxxxxxxxxxxx> wrote:
> > > >
> > > > On Tue, Sep 8, 2026 at 4:41 AM Dmitry Baryshkov
> > > > <dmitry.baryshkov@xxxxxxxxxxxxxxxx> wrote:
> > > > >
> > > > > On Mon, Sep 07, 2026 at 04:20:43PM -0700, Rob Clark wrote:
> > > > > > On Thu, Aug 27, 2026 at 9:07 PM Connor Abbott <cwabbott0@xxxxxxxxx> wrote:
> > > > > > >
> > > > > > > On Thu, Aug 27, 2026 at 7:13 PM Dmitry Baryshkov
> > > > > > > <dmitry.baryshkov@xxxxxxxxxxxxxxxx> wrote:
> > > > > > > >
> > > > > > > > The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> > > > > > > > domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from
> > > > > > > > zero whenever the GPU is powered up again and the timestamp reported to
> > > > > > > > userspace jumps backwards. On an a702 six reads three seconds apart all
> > > > > > > > land in the 500..1200 tick range, stepping backwards twice, and on an
> > > > > > > > a530 the OpenCL device timer conformance test fails because
> > > > > > > > clGetDeviceAndHostTimer() returns an end time below the start time.
> > > > > > > >
> > > > > > > > Save the counter in the suspend path of the affected generations, while
> > > > > > > > the GPU is still powered, and add the accumulated ticks to the value
> > > > > > > > reported to userspace.
> > > > > > >
> > > > > > > This is useless because the entire point of clGetDeviceAndHostTimer()
> > > > > > > (and the similar thing in Vulkan) is to match what the GPU itself
> > > > > > > returns, and now you've broken that by adding an offset.
> > > > > >
> > > > > > Just to clarify, the timestamp needs to match what UMD reads directly
> > > > > > from hw in various different ways (timestamp queries, shader_clock,
> > > > > > etc)..
> > > > > >
> > > > > > If we can _restore_ the value (ie. write the hw reg) on resume, that
> > > > > > would be ok.. but it doesn't look possible.
> > > > >
> > > > > Yes, a quick check shows that all relevant registers are write-ignore.
> > > > >
> > > > > > Otherwise I guess we need
> > > > > > some scheme where an offset value is stored in memory somewhere, and
> > > > > > some coordination with userspace to add the offset in various
> > > > > > different paths.  But the kernel shouldn't be adding the offset to
> > > > > > what it returns to userspace itself.
> > > > >
> > > > > Ack, thanks for the explanation. I tried to match what kgsl was doing.
> > > > > It reads the GPU timers directly from the GPU, but for the CPU timers it
> > > > > reads the value from KGSL driver which accumulates the offset.
> > > > >
> > > > > If I understand you correctly, we need to export the offset to the
> > > > > userspace via some extra page (like vdso) and let Mesa read it. Correct?
> > > >
> > > > Yeah, it doesn't look like there is an existing GPU_READONLY && !APRIV
> > > > BO we could re-use, so I think we need to create a new global (TTBR1)
> > > > "vdso" BO which is read-only to the GPU but otherwise accessible to
> > > > IB1+, along w/ MSM_PARAM_VDSO for UMD to query the address of this BO.
> > > > (And update msm_mapping test in igt to validate that it is read-only.)
> > > >
> > > > (I'm not married to the name "vdso" but not creative enough this
> > > > morning to think of something better)
> > > >
> > > > I'm a bit curious if kgsl handles this differently.. maybe there is
> > > > something I'm missing.  But somehow timestamps need to agree between
> > > > what is read on the GPU and what is read on CPU.
> > > >
> > > > BR,
> > > > -R
> > >
> > > I think the kgsl timestamp offset stuff was only added for the new
> > > A750+ CX GMU counter which is not read-only. We do need to port that
> > > over to drm/msm. If the offset is applied for older GPUs, it's
> > > probably another case of kgsl not bothering to keep support for older
> > > GPUs in newer branches.
> >
> > I've been looking at the kgsl drivers from msm-3.18 / 4.4 as those
> > kernels targeted MSM8996/98.
>
> If I understand correctly, for those GPUs glGetInteger64v(GL_TIMESTAMP)
> reads the ALWAYSON perf counter through IOCTL_KGSL_PERFCOUNTER_READ. The
> perfcounters are all saved on suspend and summed towards the current
> value read from HW.
>
> --
> With best wishes
> Dmitry

I think GL_TIMESTAMP just wasn't ever exposed. The equivalent GLES
extension GL_EXT_disjoint_timer_query was written by Qualcomm, and it
explicitly allows zeroing on suspend/resume. IIUC it also doesn't use
the perfcounter, it probably submits a dummy job and reads
kgsl_cmdbatch_profiling_buffer::gpu_ticks_queued.

Connor