Userspace notifications for observing userfaultfd faults

[Robert O'Callahan]

For rr (https://rr-project.org) to support recording and replaying
applications that use userfaultfd, we need to observe that a task we
are controlling has blocked on a userfault. Currently this is very
difficult to do, especially if a task blocks on a userfault on a page
where some other task has already triggered a userfault, so no new
userfaultfd event is generated. We also need to observe which page has
been faulted on so we can determine when the fault has been serviced
and the task is ready to run again.

I've tried to find workarounds with existing APIs and it doesn't seem
tractable. See https://github.com/rr-debugger/rr/issues/2852#issuecomment-837514946
for some thoughts about that.

It seems to me that a sufficient API for us would be a new software
perf event, e.g. PERF_COUNT_SW_USERFAULTS, with an associated
PERF_SAMPLE_ADDR that would give us the address of the page. Does that
sounds like a reasonable thing to add?

Robert O'Callahan