Problem with ptys, Linux version 2.6 and above

Richard B. Johnson (root@chaos.analogic.com)
Sat, 25 Sep 1999 13:51:30 -0400 (EDT)


Hello!

This is a bit long, but shows a problem with Linux terminal processing
using ptys. I need someone who knows about both terminal I/O and
the new ptys to see if there is a problem with the kernel, or if there
is a new way to set terminals to raw mode.

A program that reads/writes using stdio, saves the current tty state
and sets it to raw mode. This program usually runs off from RS-232C,
but will also run off the network. Therefore, fd 0 -> 2 can be a
pty.

This should set it to raw mode, and it does, when communicating with
a RS-232C terminal.

if(tcgetattr(0, &save_term) < 0)
ERRORS;
c_cflag = save_term.c_cflag & CBAUD;
memset(&naked_term, 0x00, sizeof(naked_term));
naked_term.c_cc[VMIN] = 1;
naked_term.c_iflag = IGNBRK|IGNPAR;
naked_term.c_cflag = c_cflag|CS8|CREAD;
if(tcsetattr(0, TCSADRAIN, &naked_term) < 0)
ERRORS;

The program writes 64 binary bytes to fd 0. The return value is 64
so it must have succeeded.

if(write(0, buf, len) != len)
ERRORS;

In the meantime, the network daemon which reads the master tty, receives
** 65 ** bytes, not 64 bytes. There is a leading byte of zero.

Later, when the program completes, it restores the terminal to its
original state. This works correctly also.

if(tcsetattr(0, TCSADRAIN, &save_term) < 0)
ERRORS;

The master terminal in the network daemon was set up as:

if(tcgetattr(0, &mem->tt) < 0)
ERRORS;
mem->tt.c_cflag &= CBAUD;
mem->tt.c_cflag |= CS8|CREAD;
mem->tt.c_oflag &= ~(VTDLY|FFDLY|BSDLY|TABDLY|CRDLY|NLDLY);
if(ioctl(0, TIOCGWINSZ, &mem->win) < 0)
ERRORS;

Then I created the task with obsolete forkpty(). I used BSD's source.
It works. The terminal parameters are set from what was saved and
modified above.

switch(forkpty(&master, mem->term, &mem->tt, &mem->win))
{
case 0:
(void)close(fd);
if(chown(mem->term, getuid(), getgid()) < 0)
ERRORS;
strcpy(mem->buf, "TERM=");
strcat(mem->buf, mem->term);
mem->env[0] = mem->buf;
mem->env[1] = NULL;
mem->argv[0] = "/sbin/monitor";
mem->argv[1] = NULL;
execve(mem->argv[0], mem->argv, mem->env);
break;
case -1:
ERRORS;
break;
default:
/*
* Okay, now set up the master and slave terminals.
*/
(void)ioctl(fd, FIONBIO, ON);
(void)ioctl(master, FIONBIO, ON);
(void)ioctl(master, TIOCPKT, ON);
/*
* Run the RX/TX binary state-machine until there is a disconnection
* or any other error.
*/
sfd = MAX(fd, master) +1;
for(;;)
{
FD_ZERO(&mem->rfds);
FD_SET(fd, &mem->rfds);
FD_SET(master, &mem->rfds);
if(select(sfd, &mem->rfds, NULL, NULL, NULL))
{
if(FD_ISSET(fd, &mem->rfds))
{
if((i = read(fd, mem->buf, BUF_LEN)) <= 0)
break;
if((i = write(master, mem->buf, i)) <= 0)
break;
}
if(FD_ISSET(master, &mem->rfds))
{
if((i = read(master, mem->buf, BUF_LEN)) <= 0)
break;

Here I read one more character than was sent by the child that
wrote to his terminal. There is always a leading '\0'. The
virtual link is, otherwise, 8-bit clean.
(If I eat the extra leading byte everything works. This is my
temporary work-around.)

if((i = write(fd, mem->buf, i)) <= 0)
break;
}
}
}
(void)close(fd);
exit(EXIT_SUCCESS);
}

The whole thing works correctly on my Sun, was in fact ported from the
Sun.

I note that telnet has the same problem. I can no longer set telnet
to binary mode and execute `sz` or `rz`. There is always an extra
leading byte of zero. It may be that my version of telnet doesn't
"know" how to set the line to binary because things may have changed.

Telnet uses ptys just like my code.

The problem with the leading byte of zero doesn't show up with normal
terminal operations, but it's still there! The terminals just ignore it.

So the question is, "How do I set the terminal to be 8-bit binary
clean with no cannocal processing?". I can't just eat the extra
byte because, code necessary to do this will not work on other
platforms.

Cheers,
Dick Johnson
**** FILE SYSTEM WAS MODIFIED ****
Penguin : Linux version 2.3.13 on an i686 machine (400.59 BogoMips).
Warning : It's hard to remain at the trailing edge of technology.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/