[PATCH] pid: make ENOMEM return value more obvious

[Christian Brauner]

The alloc_pid() codepath used to be simpler. With the introducation of the
ability to choose specific pids in 49cb2fc42ce4 ("fork: extend clone3() to
support setting a PID") it got more complex. It hasn't been super obvious
that ENOMEM is returned when the pid namespace init process/child subreaper
of the pid namespace has died. As can be seen from multiple attempts to
improve this see e.g. [1] and most recently [2].
We regressed returning ENOMEM in [3] and [2] restored it. Let's add a
comment on top explaining that this is historic and documented behavior and
cannot easily be changed.
The unconditional initialization of retval when declaring it can be removed
since it is initialized on ever failure path in the loop and unconditionaly
set to ENOMEM right after it.

[1]: 35f71bc0a09a ("fork: report pid reservation failure properly")
[2]: b26ebfe12f34 ("pid: Fix error return value in some cases")
[3]: 49cb2fc42ce4 ("fork: extend clone3() to support setting a PID")
Signed-off-by: Christian Brauner <christian.brauner@xxxxxxxxxx>
---
 kernel/pid.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/kernel/pid.c b/kernel/pid.c
index 19645b25b77c..be43122eb876 100644
--- a/kernel/pid.c
+++ b/kernel/pid.c
@@ -165,7 +165,7 @@ struct pid *alloc_pid(struct pid_namespace *ns, pid_t *set_tid,
 	int i, nr;
 	struct pid_namespace *tmp;
 	struct upid *upid;
-	int retval = -ENOMEM;
+	int retval;
 
 	/*
 	 * set_tid_size contains the size of the set_tid array. Starting at
@@ -247,6 +247,14 @@ struct pid *alloc_pid(struct pid_namespace *ns, pid_t *set_tid,
 		tmp = tmp->parent;
 	}
 
+	/*
+	 * ENOMEM is not the most obvious choice especially for the case
+	 * where the child subreaper has already exited and the pid
+	 * namespace denies the creation of any new processes. But ENOMEM
+	 * is what we have exposed to userspace for a long time and it is
+	 * documented behavior for pid namespaces. So we can't easily
+	 * change it even if there were an error code better suited.
+	 */
 	retval = -ENOMEM;
 
 	if (unlikely(is_child_reaper(pid))) {

base-commit: b26ebfe12f34f372cf041c6f801fa49c3fb382c5
-- 
2.25.1