Re: [PATCH v2 09/10] tty: Consolidate noctty checks in tty_open()

[Peter Hurley]

Hi Richard,

On 03/26/2016 10:58 AM, Richard Weinberger wrote:
> On Sun, Jan 10, 2016 at 6:13 AM, Peter Hurley <peter@xxxxxxxxxxxxxxxxxx> wrote:
>> Evaluate the conditions which prevent this tty being the controlling
>> terminal in one place, just before setting the controlling terminal.
>>
>> Signed-off-by: Peter Hurley <peter@xxxxxxxxxxxxxxxxxx>
>> ---
>>  drivers/tty/tty_io.c | 17 ++++++++---------
>>  1 file changed, 8 insertions(+), 9 deletions(-)
> 
> Peter,
> 
> This commit breaks existing userspace.
> I noticed that on UserModeLinux with Debian Squeeze as userspace, getty does not
> give me a controlling tty upon login.
> It does not seem to happen on newer distros. But still this needs
> further investigation.
> 
> Please me know what debug information you need.

Sorry about that; I hadn't considered the implications of UML console.
Can you test the blob below?

Regards,
Peter Hurley

--- >% ---
diff --git a/drivers/tty/tty_io.c b/drivers/tty/tty_io.c
index 608beb6..a361c61 100644
--- a/drivers/tty/tty_io.c
+++ b/drivers/tty/tty_io.c
@@ -2158,7 +2157,9 @@ retry_open:
 	read_lock(&tasklist_lock);
 	spin_lock_irq(&current->sighand->siglock);
 	noctty = (filp->f_flags & O_NOCTTY) ||
+#ifdef CONFIG_VT
 			device == MKDEV(TTY_MAJOR, 0) ||
+#endif
 			device == MKDEV(TTYAUX_MAJOR, 1) ||
 			(tty->driver->type == TTY_DRIVER_TYPE_PTY &&
 			 tty->driver->subtype == PTY_TYPE_MASTER);