[PATCH 1/2] mm/shmem: trace swapin results and order fallbacks

[Xuewen Wang]

PSWPIN and ZSWPIN describe swap traffic but cannot identify which tmpfs
file caused it. Shmem swapins initiated through read() do not increment
PGMAJFAULT. This makes it difficult to identify which shared-memory file
is being brought back under memory pressure.

Add an event at each return from shmem_swapin_folio(), with the
filesystem device, inode, original caller index and XArray entry, entry
and folio orders, path and return value. Include entry validation
failures and poison markers so failed attempts are visible too.

Record explicit downgrades from a higher order to order 0 when the
faulting VMA has userfaultfd armed or zswap_never_enabled() returns false.
Record both reasons when both restrictions apply. The zswap restriction
follows a static key, not the current enable setting: it persists after
zswap is disabled and also applies with CONFIG_ZSWAP_DEFAULT_ON=y when
booting with zswap.enabled=0. Initial order-0 requests do not set policy
reason bits.

Also record the initial error when a higher-order synchronous attempt
fails and is retried at order 0. Use sync_fail for this case: the swap
layer can reject a range with -EBUSY before allocating memory, and
-ENOMEM can also come from memcg charging. A racing swap-cache insertion
may supply a folio of a different order, so the fallback fields describe
the request, not its outcome.

Existing swpin_fallback and swpin_fallback_charge counters report
failures on the swap allocation path, but lack inode attribution and do
not describe these policy restrictions. Carry the fallback decision and
final result in the same event so users do not need to correlate
separate records.

Document attempt semantics, swap-cache hits and order sentinels. A
successful event represents a folio installed in the file page cache;
it need not involve disk I/O or a major fault. Keep the existing return
and retry behavior.

Signed-off-by: Xuewen Wang <wangxuewen@xxxxxxxxxx>
---
 Documentation/trace/events-shmem.rst | 73 +++++++++++++++++++++++
 Documentation/trace/index.rst        |  1 +
 MAINTAINERS                          |  2 +
 include/trace/events/shmem.h         | 86 ++++++++++++++++++++++++++++
 mm/shmem.c                           | 64 +++++++++++++++------
 5 files changed, 207 insertions(+), 19 deletions(-)
 create mode 100644 Documentation/trace/events-shmem.rst
 create mode 100644 include/trace/events/shmem.h

diff --git a/Documentation/trace/events-shmem.rst b/Documentation/trace/events-shmem.rst
new file mode 100644
index 000000000000..45da82761360
--- /dev/null
+++ b/Documentation/trace/events-shmem.rst
@@ -0,0 +1,73 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Tmpfs and shmem events
+=====================
+
+The events in ``events/shmem/`` attribute swapin and allocation activity to
+``dev`` (the filesystem device) and ``ino`` (the inode number). They also
+cover anonymous shared memory and memfd files. ``index`` is the page offset
+requested by the caller, before any alignment for a large folio.
+
+Swapin
+------
+
+``shmem_swapin_folio`` records each return from ``shmem_swapin_folio()``.
+``error`` is the return value: zero means a folio was installed in the file
+page cache; ``-EEXIST`` normally asks the caller to retry after a race.
+Entry validation failures and poison markers also produce events.
+
+``entry`` contains the original XArray entry's type and offset. For a large
+entry, this is its base swap slot even when ``index`` lies in its middle.
+For ``path=poison``, it encodes a marker, not a real swap slot.
+
+``requested_order`` is the confirmed XArray entry order, or -1 when unknown.
+``actual_order`` is the order of the folio obtained, or -1 if none was
+obtained. A nonnegative actual order does not imply successful insertion:
+always check ``error``. One successful event may cover multiple pages.
+
+``path`` identifies the branch taken: ``lookup`` for entry validation
+failure, ``cache`` for the initial swap cache hit, ``sync`` for synchronous
+swapin, ``readahead`` for the readahead helper, or ``poison`` for a marker.
+The helpers may themselves find a cached folio, so path does not prove I/O
+occurred. Event counts are not disk-I/O or major-fault counts.
+
+``fallback`` is a bitmask recording explicit downgrades from a higher order
+to order 0 in the synchronous helper. An initial order-0 request does not
+set policy reason bits, even when the policy conditions hold. Set reasons
+are printed separated by ``|``:
+
+* ``none``: no explicit downgrade in the synchronous helper.
+* ``userfaultfd``: the VMA has userfaultfd armed.
+* ``zswap``: ``zswap_never_enabled()`` is false, restricting the request
+  to order 0.
+* ``sync_fail``: the initial higher-order synchronous attempt failed.
+
+The zswap restriction follows a static key, not the current enable setting.
+It persists after zswap is disabled and also applies with
+``CONFIG_ZSWAP_DEFAULT_ON=y`` even when booting with ``zswap.enabled=0``.
+The policy reasons can occur together, so both ``userfaultfd`` and ``zswap``
+are recorded when both restrictions apply to a higher-order request.
+``fallback_error`` preserves the initial synchronous error, including
+range conflicts such as ``-EBUSY``; it is zero for policy downgrades and
+when fallback is ``none``. ``-ENOMEM`` can indicate allocation or memcg
+charging failure.
+
+These fields describe a decision to request order 0, not a guarantee about
+the returned folio: a racing swap-cache insertion may provide a different
+order. Swap-cache and readahead paths can also return a smaller folio
+without setting ``fallback``.
+
+For example, after mounting tracefs at ``/sys/kernel/tracing``, use a private
+instance to capture a workload's swapins::
+
+  mkdir /sys/kernel/tracing/instances/shmem
+  cd /sys/kernel/tracing/instances/shmem
+  echo 1 > events/shmem/shmem_swapin_folio/enable
+  # Run the workload, then stop recording before reading the buffer.
+  echo 0 > tracing_on
+  cat trace
+
+Filter or aggregate by both ``dev`` and ``ino``; inode numbers alone are
+not unique across mounts. Trace filters use the numeric values shown by
+the event's ``format`` file. Check ``per_cpu/cpu*/stats`` for dropped events
+before interpreting counts.
diff --git a/Documentation/trace/index.rst b/Documentation/trace/index.rst
index f4058e8e92e3..dbf407744430 100644
--- a/Documentation/trace/index.rst
+++ b/Documentation/trace/index.rst
@@ -51,6 +51,7 @@ applications.
 
    events
    events-kmem
+   events-shmem
    events-power
    events-nmi
    events-msr
diff --git a/MAINTAINERS b/MAINTAINERS
index d2fc3d8364b3..8f6a02dcf9c7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -27602,8 +27602,10 @@ M:	Hugh Dickins <hughd@xxxxxxxxxx>
 R:	Baolin Wang <baolin.wang@xxxxxxxxxxxxxxxxx>
 L:	linux-mm@xxxxxxxxx
 S:	Maintained
+F:	Documentation/trace/events-shmem.rst
 F:	include/linux/memfd.h
 F:	include/linux/shmem_fs.h
+F:	include/trace/events/shmem.h
 F:	mm/memfd.c
 F:	mm/shmem.c
 F:	mm/shmem_quota.c
diff --git a/include/trace/events/shmem.h b/include/trace/events/shmem.h
new file mode 100644
index 000000000000..7a0050a1686f
--- /dev/null
+++ b/include/trace/events/shmem.h
@@ -0,0 +1,86 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM shmem
+
+#if !defined(_TRACE_SHMEM_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_SHMEM_H
+
+#include <linux/bits.h>
+#include <linux/fs.h>
+#include <linux/kdev_t.h>
+#include <linux/tracepoint.h>
+
+#define SHMEM_SWAPIN_LOOKUP	0
+#define SHMEM_SWAPIN_CACHE	1
+#define SHMEM_SWAPIN_SYNC		2
+#define SHMEM_SWAPIN_READAHEAD	3
+#define SHMEM_SWAPIN_POISON	4
+
+#define SHMEM_SWAPIN_FALLBACK_NONE		0
+#define SHMEM_SWAPIN_FALLBACK_USERFAULTFD	BIT(0)
+#define SHMEM_SWAPIN_FALLBACK_ZSWAP		BIT(1)
+#define SHMEM_SWAPIN_FALLBACK_SYNC		BIT(2)
+
+#define show_shmem_swapin_fallback(flags)			\
+	(flags) ? __print_flags(flags, "|",		\
+	{ SHMEM_SWAPIN_FALLBACK_USERFAULTFD, "userfaultfd" },\
+	{ SHMEM_SWAPIN_FALLBACK_ZSWAP, "zswap" },	\
+	{ SHMEM_SWAPIN_FALLBACK_SYNC, "sync_fail" }) : "none"
+
+TRACE_EVENT(shmem_swapin_folio,
+
+	TP_PROTO(struct inode *inode, pgoff_t index,
+		 unsigned int entry_type, pgoff_t entry_offset,
+		 int requested_order, int actual_order, unsigned int path,
+		 unsigned int fallback, int fallback_error, int error),
+
+	TP_ARGS(inode, index, entry_type, entry_offset, requested_order,
+		actual_order, path, fallback, fallback_error, error),
+
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(unsigned long, ino)
+		__field(unsigned long, index)
+		__field(unsigned int, entry_type)
+		__field(unsigned long, entry_offset)
+		__field(int, requested_order)
+		__field(int, actual_order)
+		__field(unsigned int, path)
+		__field(unsigned int, fallback)
+		__field(int, fallback_error)
+		__field(int, error)
+	),
+
+	TP_fast_assign(
+		__entry->dev = inode->i_sb->s_dev;
+		__entry->ino = inode->i_ino;
+		__entry->index = index;
+		__entry->entry_type = entry_type;
+		__entry->entry_offset = entry_offset;
+		__entry->requested_order = requested_order;
+		__entry->actual_order = actual_order;
+		__entry->path = path;
+		__entry->fallback = fallback;
+		__entry->fallback_error = fallback_error;
+		__entry->error = error;
+	),
+
+	TP_printk("dev=%d:%d ino=%lu index=%lu entry=%u:%lu requested_order=%d actual_order=%d path=%s fallback=%s fallback_error=%d error=%d",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __entry->ino, __entry->index,
+		  __entry->entry_type, __entry->entry_offset,
+		  __entry->requested_order, __entry->actual_order,
+		  __print_symbolic(__entry->path,
+				   { SHMEM_SWAPIN_LOOKUP, "lookup" },
+				   { SHMEM_SWAPIN_CACHE, "cache" },
+				   { SHMEM_SWAPIN_SYNC, "sync" },
+				   { SHMEM_SWAPIN_READAHEAD, "readahead" },
+				   { SHMEM_SWAPIN_POISON, "poison" }),
+		  show_shmem_swapin_fallback(__entry->fallback),
+		  __entry->fallback_error, __entry->error)
+);
+
+#endif /* _TRACE_SHMEM_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
diff --git a/mm/shmem.c b/mm/shmem.c
index 9f0965b8b59c..3b910aec5bcf 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -87,6 +87,9 @@ static struct vfsmount *shm_mnt __ro_after_init;
 
 #include "internal.h"
 
+#define CREATE_TRACE_POINTS
+#include <trace/events/shmem.h>
+
 #define VM_ACCT(size)    (PAGE_ALIGN(size) >> PAGE_SHIFT)
 
 /* Pretend that each entry is of this size in directory's i_size */
@@ -2249,16 +2252,22 @@ static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf,
 
 static struct folio *shmem_swap_alloc_folio(struct inode *inode,
 		struct vm_fault *vmf, pgoff_t index,
-		swp_entry_t entry, int order, gfp_t gfp)
+		swp_entry_t entry, int order, gfp_t gfp,
+		unsigned int *fallback, int *fallback_error)
 {
 	pgoff_t ilx;
 	struct folio *folio;
 	struct mempolicy *mpol;
 	struct shmem_inode_info *info = SHMEM_I(inode);
 
-	if ((vmf && unlikely(userfaultfd_armed(vmf->vma))) ||
-	     !zswap_never_enabled())
-		order = 0;
+	if (order) {
+		if (vmf && unlikely(userfaultfd_armed(vmf->vma)))
+			*fallback |= SHMEM_SWAPIN_FALLBACK_USERFAULTFD;
+		if (!zswap_never_enabled())
+			*fallback |= SHMEM_SWAPIN_FALLBACK_ZSWAP;
+		if (*fallback)
+			order = 0;
+	}
 
 again:
 	mpol = shmem_get_pgoff_policy(info, index, order, &ilx);
@@ -2269,6 +2278,8 @@ static struct folio *shmem_swap_alloc_folio(struct inode *inode,
 		return folio;
 
 	if (order) {
+		*fallback |= SHMEM_SWAPIN_FALLBACK_SYNC;
+		*fallback_error = PTR_ERR(folio);
 		order = 0;
 		goto again;
 	}
@@ -2471,7 +2482,12 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index,
 	softleaf_t index_entry;
 	struct swap_info_struct *si;
 	struct folio *folio = NULL;
-	int error, nr_pages, order;
+	int error, nr_pages, order = -1;
+	int actual_order = -1;
+	unsigned int path = SHMEM_SWAPIN_LOOKUP;
+	unsigned int fallback = SHMEM_SWAPIN_FALLBACK_NONE;
+	int fallback_error = 0;
+	pgoff_t orig_index = index;
 	pgoff_t offset;
 
 	VM_BUG_ON(!*foliop || !xa_is_value(*foliop));
@@ -2479,20 +2495,21 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index,
 	swap = index_entry;
 	*foliop = NULL;
 
-	if (softleaf_is_poison_marker(index_entry))
-		return -EIO;
+	if (softleaf_is_poison_marker(index_entry)) {
+		path = SHMEM_SWAPIN_POISON;
+		error = -EIO;
+		goto out;
+	}
 
 	si = get_swap_device(index_entry);
 	order = shmem_confirm_swap(mapping, index, index_entry);
 	if (IS_ERR_OR_NULL(si)) {
-		if (order < 0)
-			return -EEXIST;
-		else
-			return -EINVAL;
+		error = order < 0 ? -EEXIST : -EINVAL;
+		goto out;
 	}
 	if (unlikely(order < 0)) {
-		put_swap_device(si);
-		return -EEXIST;
+		error = -EEXIST;
+		goto put_swap;
 	}
 
 	/* index may point to the middle of a large entry, get the sub entry */
@@ -2506,10 +2523,13 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index,
 	if (!folio) {
 		if (data_race(si->flags & SWP_SYNCHRONOUS_IO)) {
 			/* Direct swapin skipping swap cache & readahead */
+			path = SHMEM_SWAPIN_SYNC;
 			folio = shmem_swap_alloc_folio(inode, vmf, index,
-						       swap, order, gfp);
+						       swap, order, gfp, &fallback,
+						       &fallback_error);
 		} else {
 			/* Cached swapin only supports order 0 folio */
+			path = SHMEM_SWAPIN_READAHEAD;
 			folio = shmem_swapin_cluster(swap, gfp, info, index);
 		}
 		if (IS_ERR_OR_NULL(folio)) {
@@ -2526,6 +2546,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index,
 			count_memcg_event_mm(fault_mm, PGMAJFAULT);
 		}
 	} else {
+		path = SHMEM_SWAPIN_CACHE;
 		swap_update_readahead(folio, NULL, 0);
 	}
 
@@ -2603,10 +2624,9 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index,
 	folio_put_swap(folio, NULL);
 	swap_cache_del_folio(folio);
 	folio_mark_dirty(folio);
-	put_swap_device(si);
-
+	actual_order = folio_order(folio);
 	*foliop = folio;
-	return 0;
+	goto put_swap;
 failed:
 	if (shmem_confirm_swap(mapping, index, swap) < 0)
 		error = -EEXIST;
@@ -2616,10 +2636,16 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index,
 	if (folio)
 		folio_unlock(folio);
 failed_nolock:
-	if (folio)
+	if (folio) {
+		actual_order = folio_order(folio);
 		folio_put(folio);
+	}
+put_swap:
 	put_swap_device(si);
-
+out:
+	trace_shmem_swapin_folio(inode, orig_index, swp_type(index_entry),
+				 swp_offset(index_entry), order, actual_order,
+				 path, fallback, fallback_error, error);
 	return error;
 }
 
-- 
2.25.1