[PATCH 2/2] mm/shmem: trace folio allocation failures and fallbacks

[Xuewen Wang]

The shmem allocation counters cannot identify which tmpfs file is
affected by allocation fallback. They also do not describe order
filtering caused by VMA constraints or existing page-cache entries.

Trace failed calls to shmem_alloc_and_add_folio() and successful calls
that allocate below the highest order permitted by the supplied mask.
Record the original index and the highest permitted, highest suitable
and allocated orders to distinguish filtering from allocator fallback.
A suitable order of -1 identifies an empty filtered mask, where no folio
allocation is attempted; such records use a separate filter stage and keep
the stage error at zero while preserving the helper return value. Omit
successes at the requested order, including ordinary base-page allocations.

Identify failures by stage: allocation, memcg charging, cache insertion
or filesystem accounting. Preserve both the stage's errno and the
returned errno. In particular, a charge failure remains visible even
when a page-cache conflict makes the function return -EEXIST instead.
Stage and errno distinguish this case from a direct insertion race.

Document that each event describes one call to the helper, which may try
multiple candidate orders. The caller may subsequently retry at order 0
or restart a lookup, so an error in this event is not necessarily a failed
user operation. A successful order-0 retry is omitted as a success at that
call's requested order.

Signed-off-by: Xuewen Wang <wangxuewen@xxxxxxxxxx>
---
 Documentation/trace/events-shmem.rst | 45 +++++++++++++++++++++++
 include/trace/events/shmem.h         | 55 ++++++++++++++++++++++++++++
 mm/shmem.c                           | 34 ++++++++++++++++-
 3 files changed, 133 insertions(+), 1 deletion(-)

diff --git a/Documentation/trace/events-shmem.rst b/Documentation/trace/events-shmem.rst
index 45da82761360..4a0e50bdbdf3 100644
--- a/Documentation/trace/events-shmem.rst
+++ b/Documentation/trace/events-shmem.rst
@@ -71,3 +71,48 @@ Filter or aggregate by both ``dev`` and ``ino``; inode numbers alone are
 not unique across mounts. Trace filters use the numeric values shown by
 the event's ``format`` file. Check ``per_cpu/cpu*/stats`` for dropped events
 before interpreting counts.
+
+Allocation attempts
+-------------------
+
+``shmem_alloc_folio_finish`` records failed calls to
+``shmem_alloc_and_add_folio()`` and successful calls that allocate below
+their requested order. Successes at the requested order are omitted,
+including ordinary order-0 allocations.
+
+This is an attempt result, not the final result of a read, write or fault.
+After a higher-order failure the caller may retry at order 0, or restart
+its lookup on ``-EEXIST``. A successful order-0 retry is not recorded.
+Do not count all nonzero returns as failed user operations.
+
+``requested_order`` is the highest order allowed by this call's order mask,
+or 0 for a base-page request. ``suitable_order`` is the highest remaining
+order after VMA and page-cache filtering, or -1 if none remains.
+``actual_order`` is the allocated folio's order, or -1 if allocation did
+not produce a folio. Orders refer to base pages, not bytes.
+
+``stage`` identifies where the attempt ended:
+
+* ``filter``: filtering removed every candidate order, so no folio
+  allocation was attempted. ``suitable_order=-1`` identifies this case.
+* ``alloc``: a suitable order existed, but every allocation failed.
+* ``charge``: memcg charging failed.
+* ``cache``: page-cache insertion failed.
+* ``account``: filesystem block accounting failed after its retry.
+* ``done``: allocation succeeded at a smaller order.
+
+``error`` is the error at that stage; ``ret`` is the error returned to the
+caller. ``stage=filter`` has ``error=0`` because no allocator call occurred,
+while ``ret=-ENOMEM`` preserves the helper's return value. Both are zero for
+``done``. For a charge failure followed by a page-cache conflict, ``error``
+retains the charge error (such as ``-ENOMEM``), while ``ret=-EEXIST``
+requests a lookup retry. A direct insertion race instead has
+``stage=cache error=-EEXIST ret=-EEXIST``. Accounting errors retain
+``-ENOSPC`` or ``-EDQUOT`` as appropriate.
+
+Enable ``events/shmem/shmem_alloc_folio_finish/enable`` in the trace
+instance to collect these records. On successful fallbacks,
+``suitable_order < requested_order`` shows filtering, and
+``actual_order < suitable_order`` shows allocator fallback. Both may
+occur in one attempt. The existing shmem allocation and fallback
+counters remain useful for aggregate statistics.
diff --git a/include/trace/events/shmem.h b/include/trace/events/shmem.h
index 7a0050a1686f..72b20d6b3caf 100644
--- a/include/trace/events/shmem.h
+++ b/include/trace/events/shmem.h
@@ -80,6 +80,61 @@ TRACE_EVENT(shmem_swapin_folio,
 		  __entry->fallback_error, __entry->error)
 );
 
+#define SHMEM_ALLOC_ALLOC	0
+#define SHMEM_ALLOC_CHARGE	1
+#define SHMEM_ALLOC_CACHE		2
+#define SHMEM_ALLOC_ACCOUNT	3
+#define SHMEM_ALLOC_DONE		4
+#define SHMEM_ALLOC_FILTER	5
+
+TRACE_EVENT(shmem_alloc_folio_finish,
+
+	TP_PROTO(struct inode *inode, pgoff_t index,
+		 int requested_order, int suitable_order, int actual_order,
+		 unsigned int stage, int error, int ret),
+
+	TP_ARGS(inode, index, requested_order, suitable_order, actual_order,
+		stage, error, ret),
+
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(unsigned long, ino)
+		__field(unsigned long, index)
+		__field(int, requested_order)
+		__field(int, suitable_order)
+		__field(int, actual_order)
+		__field(unsigned int, stage)
+		__field(int, error)
+		__field(int, ret)
+	),
+
+	TP_fast_assign(
+		__entry->dev = inode->i_sb->s_dev;
+		__entry->ino = inode->i_ino;
+		__entry->index = index;
+		__entry->requested_order = requested_order;
+		__entry->suitable_order = suitable_order;
+		__entry->actual_order = actual_order;
+		__entry->stage = stage;
+		__entry->error = error;
+		__entry->ret = ret;
+	),
+
+	TP_printk("dev=%d:%d ino=%lu index=%lu requested_order=%d suitable_order=%d actual_order=%d stage=%s error=%d ret=%d",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __entry->ino, __entry->index,
+		  __entry->requested_order, __entry->suitable_order,
+		  __entry->actual_order,
+		  __print_symbolic(__entry->stage,
+				   { SHMEM_ALLOC_ALLOC, "alloc" },
+				   { SHMEM_ALLOC_CHARGE, "charge" },
+				   { SHMEM_ALLOC_CACHE, "cache" },
+				   { SHMEM_ALLOC_ACCOUNT, "account" },
+				   { SHMEM_ALLOC_DONE, "done" },
+				   { SHMEM_ALLOC_FILTER, "filter" }),
+		  __entry->error, __entry->ret)
+);
+
 #endif /* _TRACE_SHMEM_H */
 
 /* This part must be outside protection */
diff --git a/mm/shmem.c b/mm/shmem.c
index 3b910aec5bcf..68321d61dfdb 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -2155,17 +2155,23 @@ static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf,
 	unsigned long suitable_orders = 0;
 	struct folio *folio = NULL;
 	pgoff_t aligned_index;
+	pgoff_t orig_index = index;
 	long pages;
 	int error, order;
+	int requested_order, suitable_order;
+	int charge_error = 0;
+	unsigned int stage;
 
 	if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
 		orders = 0;
 
+	requested_order = orders ? highest_order(orders) : 0;
 	if (orders > 0) {
 		suitable_orders = shmem_suitable_orders(inode, vmf,
 							mapping, index, orders);
 
 		order = highest_order(suitable_orders);
+		suitable_order = order;
 		while (suitable_orders) {
 			pages = 1UL << order;
 			aligned_index = round_down(index, pages);
@@ -2181,19 +2187,35 @@ static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf,
 			order = next_order(&suitable_orders, order);
 		}
 	} else {
+		suitable_order = 0;
 		pages = 1;
 		folio = shmem_alloc_folio(gfp, 0, info, index);
 	}
-	if (!folio)
+	if (!folio) {
+		if (suitable_order < 0)
+			trace_shmem_alloc_folio_finish(inode, orig_index,
+						       requested_order,
+						       suitable_order, -1,
+						       SHMEM_ALLOC_FILTER, 0,
+						       -ENOMEM);
+		else
+			trace_shmem_alloc_folio_finish(inode, orig_index,
+						       requested_order,
+						       suitable_order, -1,
+						       SHMEM_ALLOC_ALLOC, -ENOMEM,
+						       -ENOMEM);
 		return ERR_PTR(-ENOMEM);
+	}
 
 allocated:
 	__folio_set_locked(folio);
 	__folio_set_swapbacked(folio);
 
 	gfp &= GFP_RECLAIM_MASK;
+	stage = SHMEM_ALLOC_CHARGE;
 	error = mem_cgroup_charge(folio, fault_mm, gfp);
 	if (error) {
+		charge_error = error;
 		if (xa_find(&mapping->i_pages, &index,
 				index + pages - 1, XA_PRESENT)) {
 			error = -EEXIST;
@@ -2208,10 +2230,12 @@ static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf,
 		goto unlock;
 	}
 
+	stage = SHMEM_ALLOC_CACHE;
 	error = shmem_add_to_page_cache(folio, mapping, index, NULL, gfp);
 	if (error)
 		goto unlock;
 
+	stage = SHMEM_ALLOC_ACCOUNT;
 	error = shmem_inode_acct_blocks(inode, pages);
 	if (error) {
 		struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb);
@@ -2242,9 +2266,17 @@ static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf,
 
 	shmem_recalc_inode(inode, pages, 0);
 	folio_add_lru(folio);
+	if (folio_order(folio) < requested_order)
+		trace_shmem_alloc_folio_finish(inode, orig_index,
+					       requested_order, suitable_order,
+					       folio_order(folio),
+					       SHMEM_ALLOC_DONE, 0, 0);
 	return folio;
 
 unlock:
+	trace_shmem_alloc_folio_finish(inode, orig_index, requested_order,
+				       suitable_order, folio_order(folio), stage,
+				       charge_error ?: error, error);
 	folio_unlock(folio);
 	folio_put(folio);
 	return ERR_PTR(error);
-- 
2.25.1