PTY handling patch for 1.3.29

Jeff Noxon (jeff@mrsagw.mrsa.com)
Wed, 27 Sep 1995 10:54:07 -0500 (CDT)

About a month ago I posted to linux-kernel about a problem I was having with
the pty handling. The problem is that someone can telnet to a machine and
start an ill-behaved daemon process that inherits a slave pty, but
the master is released when the telnet session (or xterm or whatever)
is terminated. The open slave can cause problems for programs that try to
use that pty later.

The following patch circumvents the problem by preventing a master pty from
being opened if it has any slaves open. The patch may be horribly
broken, but it fixes a lot of problems for me. Assuming there's not a better
way to do this, I would like to see this go into the kernel. (Read: I
don't know what I am doing with this code, but I needed a fix. It works for
me. :-)

The same patch works with 1.2.13, but will probably need to be applied by
hand.

All comments appreciated!

Thanks,

Jeff

--- linux/drivers/char/tty_io.c Mon Sep 18 00:54:08 1995
+++ linux/drivers/char/tty_io.c Mon Sep 25 13:48:46 1995
@@ -38,6 +38,9 @@
*
* Reorganized FASYNC support so mouse code can share it.
* -- ctm@ardi.com, 9Sep95
+ *
+ * Don't open a master pty if slaves are open.
+ * -- Jeff Noxon <jeff@mrsagw.mrsa.com>, 25 Sep 95
*/

#include <linux/config.h>
@@ -765,10 +768,10 @@
*/
static int init_dev(kdev_t device, struct tty_struct **ret_tty)
{
- struct tty_struct *tty, **tty_loc, *o_tty, **o_tty_loc;
+ struct tty_struct *tty, **tty_loc, *o_tty, **o_tty_loc, **s_tty_loc;
struct termios *tp, **tp_loc, *o_tp, **o_tp_loc;
struct termios *ltp, **ltp_loc, *o_ltp, **o_ltp_loc;
- struct tty_driver *driver;
+ struct tty_driver *driver, *slave;
int retval;
int idx;

@@ -793,6 +796,15 @@
driver->subtype == PTY_TYPE_MASTER &&
*tty_loc && (*tty_loc)->count)
goto end_init;
+ if (driver->type == TTY_DRIVER_TYPE_PTY &&
+ driver->subtype == PTY_TYPE_MASTER)
+ {
+ slave = get_tty_driver(device+64);
+ if (!slave) goto end_init;
+ s_tty_loc = &slave->table[idx];
+ if (*s_tty_loc && (*s_tty_loc)->count)
+ goto end_init;
+ }
retval = -ENOMEM;
if (!*tty_loc && !tty) {
if (!(tty = (struct tty_struct*) get_free_page(GFP_KERNEL)))