Re: [PATCH v3 08/33] drm/ttm: Bound page (re)allocation per defragmentation move
From: Niranjana Vishwanathapura
Date: Fri Nov 20 2026 - 15:37:40 EST
On Thu, Nov 19, 2026 at 06:14:23PM -0800, Matthew Brost wrote:
On Mon, Nov 09, 2026 at 06:44:06PM -0800, Niranjana Vishwanathapura wrote:
On Fri, Jul 10, 2026 at 07:55:54PM -0700, Matthew Brost wrote:
> A defragmentation move reallocates an object's backing at the device's
> beneficial order and harvests whatever the allocator cannot upgrade from
> the old tt. For a very large object this can reallocate (and have the
> driver re-copy) the entire backing in one move, which is expensive and
> can stall concurrent work for an unbounded stretch.
Assisted-by: LLM
NIT...So, a time budge would also be helpful here along with the size
budge?
That could be option but a size budget seems like reasonable place to start.
ok
>
> Add an optional per-move byte budget so a large object can be upgraded in
> slices across successive moves. Plumb a new
> ttm_operation_ctx::defrag_bytes_remaining through to the pool allocator:
> only pages actually (re)allocated at the beneficial order are charged
> against it; pages harvested (borrowed) from defrag_old_tt are free and do
> not count. Once the budget is exhausted the allocator stops upgrading and
> completes the new tt by harvesting the remainder of the old tt as-is,
> marking the tt sub-optimal (TTM_TT_FLAG_BENEFICIAL_ORDER_FAILED) so the
> driver re-queues it and a later move resumes the upgrade. The slices
> already upgraded come back as free already-beneficial harvests on the
> next pass, so each move spends budget only on new forward progress.
>
> The allocator decrements the budget as it allocates and writes the unused
> remainder back on success, letting the caller compute how many bytes were
> actually spent. A value of 0 means unlimited (upgrade the whole object in
> one move), preserving the previous behaviour. This is only consulted on a
> defragmentation move, i.e. when defrag_old_tt is set.
>
> Cc: Carlos Santa <carlos.santa@xxxxxxxxx>
> Cc: Ryan Neph <ryanneph@xxxxxxxxxx>
> Cc: Christian Koenig <christian.koenig@xxxxxxx>
> Cc: Huang Rui <ray.huang@xxxxxxx>
> Cc: Matthew Auld <matthew.auld@xxxxxxxxx>
> Cc: Maarten Lankhorst <maarten.lankhorst@xxxxxxxxxxxxxxx>
> Cc: Maxime Ripard <mripard@xxxxxxxxxx>
> Cc: Thomas Zimmermann <tzimmermann@xxxxxxx>
> Cc: David Airlie <airlied@xxxxxxxxx>
> Cc: Simona Vetter <simona@xxxxxxxx>
> Cc: dri-devel@xxxxxxxxxxxxxxxxxxxxx
> Cc: linux-kernel@xxxxxxxxxxxxxxx
> Cc: Thomas Hellström <thomas.hellstrom@xxxxxxxxxxxxxxx>
> Assisted-by: GitHub_Copilot:claude-opus-4.8
> Signed-off-by: Matthew Brost <matthew.brost@xxxxxxxxx>
> ---
> drivers/gpu/drm/ttm/ttm_pool.c | 101 ++++++++++++++++++++++++++++++++-
> include/drm/ttm/ttm_bo.h | 14 +++++
> 2 files changed, 112 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/gpu/drm/ttm/ttm_pool.c b/drivers/gpu/drm/ttm/ttm_pool.c
> index 52f5e64f05dd..1673e81b8df8 100644
> --- a/drivers/gpu/drm/ttm/ttm_pool.c
> +++ b/drivers/gpu/drm/ttm/ttm_pool.c
> @@ -77,6 +77,11 @@ struct ttm_pool_dma {
> * @dma_addr: Pointer to the next tt dma_address entry to populate if any.
> * @remaining_pages: Remaining pages to populate.
> * @tt_caching: The requested cpu-caching for the pages allocated.
> + * @defrag_bytes_remaining: Byte budget for newly (re)allocated pages on a
> + * defragmentation move, seeded from ttm_operation_ctx::defrag_bytes_remaining.
> + * Pages harvested (borrowed) from the old tt are free and do not count; once
> + * this drops to zero the allocator stops upgrading and harvests the remainder.
> + * On completion it holds the unused budget. 0 means unlimited (no defrag cap).
> */
> struct ttm_pool_alloc_state {
> struct page **pages;
> @@ -84,6 +89,7 @@ struct ttm_pool_alloc_state {
> dma_addr_t *dma_addr;
> pgoff_t remaining_pages;
> enum ttm_caching tt_caching;
> + s64 defrag_bytes_remaining;
> };
>
> /**
> @@ -780,6 +786,7 @@ static void ttm_pool_alloc_state_init(const struct ttm_tt *tt,
> alloc->dma_addr = tt->dma_address;
> alloc->remaining_pages = tt->num_pages;
> alloc->tt_caching = tt->caching;
> + alloc->defrag_bytes_remaining = 0;
> }
>
> /*
> @@ -885,8 +892,12 @@ static int ttm_pool_harvest_page(struct ttm_tt *old_tt, unsigned int order,
>
> static int ttm_pool_harvest_remaining(struct ttm_pool *pool,
> struct ttm_tt *old_tt, pgoff_t off,
> - struct ttm_pool_alloc_state *alloc)
> + struct ttm_pool_alloc_state *alloc,
> + bool *suboptimal)
> {
> + unsigned int beneficial = ttm_pool_beneficial_order(pool);
> + pgoff_t bnr = 1UL << beneficial;
> +
> /*
> * @off is always an old-tt chunk head here, never a tail page, so
> * reading the per-chunk order below is safe. This holds because:
> @@ -906,8 +917,23 @@ static int ttm_pool_harvest_remaining(struct ttm_pool *pool,
> struct page *p = old_tt->pages[off];
> unsigned int order = ttm_pool_page_order(pool, p);
> pgoff_t nr = 1UL << order;
> + pgoff_t num_pages = off + alloc->remaining_pages;
num_pages seems loop invariant. May be move it out of the loop as total_pages
or something?
No, both 'off' and 'alloc->remaining_pages' change each iteration of the loop.
Yes, but the sum 'off + alloc->remaining_pages' is always equals to
it->tt->num_pages, as we do +nr to offset and -nr to alloc->remaining_pages.
> int r;
>
> + /*
> + * Only report a sub-optimal backing when a chunk we cannot
> + * upgrade (out of budget/prealloc) sits in a beneficial-order
> + * aligned region that fully fits in the tt: that region could
> + * have been upgraded with more budget. A chunk below the
> + * beneficial order is expected for the unaligned tail (the last
> + * partial beneficial-order region) and for already beneficial-
> + * order harvests - neither leaves real defrag work, so they must
> + * not flag the tt for re-queue.
> + */
> + if (beneficial && order < beneficial &&
> + round_down(off, bnr) + bnr <= num_pages)
> + *suboptimal = true;
> +
> r = ttm_pool_harvest_page(old_tt, order, off, alloc);
> if (r)
> return r;
e.g., below here (unseen) 'off' is incremented by 'nr'.
yes, but alloc->remaining_pages is decremented by 'nr'.
> @@ -951,6 +977,7 @@ struct ttm_pool_alloc_iter {
> struct ttm_tt *defrag_old_tt;
> unsigned int beneficial_order;
> gfp_t gfp_flags;
> + bool defrag_capped;
>
> /* Mutated as the loop walks the orders. */
> unsigned int order;
> @@ -962,6 +989,44 @@ struct ttm_pool_alloc_iter {
> struct page *p;
> };
>
> +/*
> + * Defrag move budget exhausted: the upgrade can make no further progress this
> + * pass. Snapshot @defrag_capped is set only when a byte budget was in force at
> + * entry.
> + */
> +static bool ttm_pool_iter_defrag_exhausted(const struct ttm_pool_alloc_iter *it)
> +{
> + if (!it->defrag_old_tt)
> + return false;
> +
> + return it->defrag_capped && it->alloc->defrag_bytes_remaining <= 0;
> +}
> +
> +/*
> + * Complete the new tt by harvesting the rest of the old tt as-is (borrowing its
> + * pages, including any still below beneficial order). Never fall back to in-lock
> + * reclaim for the shortfall - that would reintroduce the stall we preallocated
> + * to avoid. Only re-queue (flag) the tt when the harvested remainder is
> + * genuinely sub-optimal; if every remaining old chunk was already at beneficial
> + * order the move fully upgraded the object.
> + */
> +static int ttm_pool_iter_harvest_rest(struct ttm_pool_alloc_iter *it)
> +{
> + pgoff_t off = it->tt->num_pages - it->alloc->remaining_pages;
> + bool suboptimal = false;
> + int r;
> +
> + r = ttm_pool_harvest_remaining(it->pool, it->defrag_old_tt, off,
> + it->alloc, &suboptimal);
> + if (r)
> + return r;
> +
> + if (suboptimal)
> + it->tt->page_flags |= TTM_TT_FLAG_BENEFICIAL_ORDER_FAILED;
> +
> + return 0;
> +}
> +
> /*
> * Defrag move: reuse an already beneficial-order chunk from the old tt at this
> * offset rather than reallocating it. Harvest whatever order the old chunk has
> @@ -1062,11 +1127,12 @@ static int ttm_pool_iter_lower_order(struct ttm_pool_alloc_iter *it)
> if (it->alloc_count && it->defrag_old_tt) {
> pgoff_t off = it->tt->num_pages -
> it->alloc->remaining_pages;
> + bool suboptimal = false;
> int r;
>
> r = ttm_pool_harvest_remaining(it->pool,
> it->defrag_old_tt, off,
> - it->alloc);
> + it->alloc, &suboptimal);
As suboptimal is not required here, probably send a NULL here and have
ttm_pool_harvest_remaining handle that NULL.
That is better will change.
> if (r)
> return r;
>
> @@ -1121,10 +1187,23 @@ static int __ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,
> gfp_flags |= GFP_HIGHUSER;
>
> it.gfp_flags = gfp_flags;
> + it.defrag_capped = it.defrag_old_tt && alloc->defrag_bytes_remaining > 0;
>
> for (it.order = ttm_pool_alloc_find_order(MAX_PAGE_ORDER, alloc);
> alloc->remaining_pages;
> it.order = ttm_pool_alloc_find_order(it.order, alloc)) {
> + /*
> + * Out of defrag budget: harvest the rest of the old tt as-is and
> + * stop (the tt is re-queued if the remainder is still
> + * sub-optimal).
> + */
> + if (ttm_pool_iter_defrag_exhausted(&it)) {
> + r = ttm_pool_iter_harvest_rest(&it);
> + if (r)
> + goto error_free_all;
> + break;
> + }
> +
> /* Reuse an already beneficial-order chunk from the old tt. */
> r = ttm_pool_iter_reuse_old(&it);
> if (r < 0)
> @@ -1155,6 +1234,15 @@ static int __ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,
> goto error_free_all;
> }
>
> + /*
> + * Charge a newly (re)allocated chunk against the defrag move
> + * budget. Harvested chunks borrow the old tt's pages for free
> + * and never reach here.
> + */
> + if (it.defrag_capped)
> + alloc->defrag_bytes_remaining -=
> + (s64)(1UL << it.order) << PAGE_SHIFT;
> +
> it.alloc_count++;
> }
>
> @@ -1204,14 +1292,21 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt,
> struct ttm_operation_ctx *ctx)
> {
> struct ttm_pool_alloc_state alloc;
> + int ret;
>
> if (WARN_ON(ttm_tt_is_backed_up(tt)))
> return -EINVAL;
>
> tt->page_flags &= ~TTM_TT_FLAG_BENEFICIAL_ORDER_FAILED;
> ttm_pool_alloc_state_init(tt, &alloc);
> + alloc.defrag_bytes_remaining = ctx->defrag_bytes_remaining;
> +
> + ret = __ttm_pool_alloc(pool, tt, ctx, &alloc, NULL);
> +
> + /* Report the unused defrag budget back to the caller. */
> + ctx->defrag_bytes_remaining = alloc.defrag_bytes_remaining;
>
> - return __ttm_pool_alloc(pool, tt, ctx, &alloc, NULL);
> + return ret;
> }
> EXPORT_SYMBOL(ttm_pool_alloc);
>
> diff --git a/include/drm/ttm/ttm_bo.h b/include/drm/ttm/ttm_bo.h
> index fcaba5fb8bf7..6124d2627b47 100644
> --- a/include/drm/ttm/ttm_bo.h
> +++ b/include/drm/ttm/ttm_bo.h
> @@ -223,6 +223,20 @@ struct ttm_operation_ctx {
> * NULL for non-defrag operations.
> */
> struct ttm_tt *defrag_old_tt;
> + /**
> + * @defrag_bytes_remaining: Byte budget for newly allocated pages during
> + * a defragmentation move. Only pages that are actually (re)allocated at
> + * the beneficial order count against it; pages harvested (borrowed) from
> + * @defrag_old_tt are free and do not. When the budget is exhausted the
> + * pool allocator stops upgrading and completes the new tt by harvesting
> + * the remainder of @defrag_old_tt, marking the tt sub-optimal so the
> + * driver can resume on a later pass. The allocator decrements this as it
> + * allocates, so on return it holds the unused budget (which the caller
> + * can use to compute how many bytes were spent). A value of 0 means
> + * unlimited (upgrade the whole object). Only consulted when
> + * @defrag_old_tt is set.
> + */
> + s64 defrag_bytes_remaining;
If user specifies -ve value, currently it means unlimited. So, may be update the
documentation here to say "A value of <= 0 means unlimited"? Or, better only accept
> =0 value from the user ctx here.
How about I reject user input defrag_bytes_remaining < 0?
Ok, sounds good.
Niranjana
Matt
Niranjana
> /**
> * @resv: Reservation object to be used together with
> * @allow_res_evict.
> --
> 2.34.1
>