[PATCH v6 0/5] perf script: Lazy symbol loading

[Alireza Haghdoost via B4 Relay]

perf script loads the entire ELF symbol table of every DSO that appears
in a sample, and keeps each symbol, with its demangled name, until
process exit. Most of those symbols are never sampled. On a large
profile this turns symbol loading into the main memory cost of perf
script, and in a memory-constrained cgroup into an OOM kill.

This series adds an opt-in lazy loader for userspace ELF DSOs, after a
regression fix and two preparatory patches:

  [1/5] Fix a broken "#ifdef ELF_C_READ_MMAP" guard so perf actually
        mmaps ELF files instead of malloc'ing section data. This is a
        standalone regression fix for 22dd1ac91a77, which Ian's series
        [1] also depends on.

  [2/5] Split the size, type and binding checks out of
        choose_best_symbol(), so that the loader can tell which aliases
        need their names. No functional change.

  [3/5] Make the symbol name a pointer instead of a flexible array, so
        that a name can be set after the symbol is in the rb-tree.

  [4/5] Add --lazy-load-symbols to load symbol names lazily.

  [5/5] Unit and shell tests.

On a perf script run over a 45,073-sample callchain profile of a large
C++ service (about 1M symbols), the output is identical in all four
cases (medians of five runs):

                                      time    max RSS
  perf-tools-next                     3.3 s   334 MiB
  v6                                  3.2 s   341 MiB
  v6 --lazy-load-symbols              1.5 s   154 MiB
  v5 --lazy-load-symbols              1.4 s   136 MiB

Eager loading remains the default, and costs 8 more bytes per symbol for
the name pointer. Lazy loading covers userspace ELF .symtab and .dynsym.
The kernel, modules, PPC64 .opd and .gnu_debugdata still load names
eagerly.

Ian's series [1], which removes the priv area and replaces the rb-tree
with an array, would shrink each symbol further. This series does not
depend on it, and I'll rebase on it if it lands first.

[1] https://lore.kernel.org/all/20261027231513.816148-1-irogers@xxxxxxxxxx/

Changes in v6:

- Keep the symbols in struct symbol and load only their names lazily,
  instead of v5's separate per-DSO index (Namhyung).
- Drop v5 2/5 ("perf dso: Allow reading DSO data from an explicit
  file"). Names are read with pread() from the symbol-table file that
  dso__get_filename() returns. The file remains open only while perf
  reads the names. This path does not use the data cache (Namhyung).
- Replace v5 3/5 ("perf symbols: Factor out duplicate symbol
  selection") with 2/5, which only splits out the name-independent
  checks. struct symbol_candidate is gone, and choose_best_symbol()
  still takes struct symbol (Ian's v4 question).
- New 3/5: point to the symbol name instead of embedding it.
- 4/5: rewritten for the new design.
- 5/5: replace the index tests (truncated names, lazy address and name
  lookup, nested range lookup, address lookups racing name lookups,
  lookups racing DSO data reads) with tests of long and unreadable lazy
  names and of names loaded concurrently. The duplicate selection test
  now checks which pairs need names. The eager/lazy parity unit test,
  the parity shell test and the perf script shell test stay. The parity
  shell test drops its nested symbol case, since both modes now share
  the same address lookup. The perf script shell test compares the
  whole output, and reports a skip only if none of its cases can run.
- Rebase onto current perf-tools-next.

Previous versions:

v5: https://lore.kernel.org/all/20261005-perf-symbol-memory-send-v5-0-165dceb2b049@xxxxxxxx/
v4: https://lore.kernel.org/all/20261002-perf-symbol-memory-send-v4-0-0a592bb2539e@xxxxxxxx/
v3: https://lore.kernel.org/all/20260925-perf-symbol-memory-send-v3-0-3e4e234c363b@xxxxxxxx/
v2: https://lore.kernel.org/all/20260919-perf-symbol-memory-send-v2-0-495b8f00ad7c@xxxxxxxx/
v1: https://lore.kernel.org/all/20260915-perf-symbol-memory-send-v1-0-1d3360e21f07@xxxxxxxx/

---
Alireza Haghdoost (5):
      perf symbols: Fix broken ELF_C_READ_MMAP fallback guard
      perf symbols: Split out name-independent duplicate symbol selection
      perf symbols: Point to the symbol name instead of embedding it
      perf script: Add --lazy-load-symbols to load symbol names lazily
      perf test: Test lazy symbol name loading

 tools/perf/Documentation/perf-script.txt           |   7 +
 tools/perf/builtin-script.c                        |   2 +
 tools/perf/tests/Build                             |   1 +
 tools/perf/tests/builtin-test.c                    |   1 +
 tools/perf/tests/shell/lazy_load_symbols_parity.sh | 284 +++++++++++
 tools/perf/tests/shell/script_lazy_load_symbols.sh | 177 +++++++
 tools/perf/tests/symbol-lazy.c                     | 544 +++++++++++++++++++++
 tools/perf/tests/tests.h                           |   1 +
 tools/perf/ui/browsers/map.c                       |   2 +-
 tools/perf/ui/stdio/hist.c                         |   1 +
 tools/perf/util/dso.c                              |   2 +-
 tools/perf/util/dso.h                              |   3 +-
 tools/perf/util/symbol-elf.c                       | 251 +++++++++-
 tools/perf/util/symbol-minimal.c                   |   6 +
 tools/perf/util/symbol.c                           | 140 +++++-
 tools/perf/util/symbol.h                           |  40 +-
 tools/perf/util/symbol_conf.h                      |   1 +
 17 files changed, 1429 insertions(+), 34 deletions(-)
---
base-commit: e74880764bf94b738102b4b6b6e8304d0c8375d3
change-id: 20260915-perf-symbol-memory-send-e7cfca1ac3d9

Best regards,
--  
Alireza Haghdoost <haghdoost@xxxxxxxx>