Technical article 105040

Running a legacy application on OpenServer 5 results in screen corruption.

OpenServer 5.0.0OpenServer 5.0.2OpenServer 5.0.4

Written · Last revised

Problem

I have an application that has been compiled and run on earlier releases of SCO UNIX System V/386. However, when I run this same application on SCO OpenServer Release 5, it experiences screen corruption.

If I pipe the output of the original application through cat(C) or tee(C), however, this will also run successfully on SCO OpenServer Release 5.

Cause

This problem has been, in most cases, related to a programming error in the application which was not apparent when executed on previous releases of SCO UNIX System V/386.

It has been found that, in these types of applications, in order to set the stdin stream into non-blocking mode, the following code would be used:

int     flags;
...

/* Obtain current flags */ flags = fcntl(0, F_GETFL, 0);

/* Make standard input non-blocking */
fcntl(0, F_SETFL, flags | O_NDELAY);

If such code is executed under SCO UNIX System V/386 or SCO XENIX System V, it will run perfectly on these systems, but will lose amounts of output data when run under SCO OpenServer 5. If the setting of the non-blocking input flag isn't done, it works fine in all cases.

The reason for this is that the stdin, stdout and stderr streams
all reference the _same_ internal File Table entry since they are,
in fact, dup(S) copies of the same file descriptor. This means that
setting O_NDELAY (or O_NONBLOCK) on one will also set it on the
others. Hence, in the above code fragment, O_NDELAY will also be
set on stdout and stderr, resulting in all three I/O streams being
set into non-blocking mode.

When a process is initially created, stdin, stdout and stderr are by default all attached to the same physical tty device (also known as the 'controlling terminal'). This tty device may not be capable of writing an entire output buffer in a single operation.

In previous releases, writing data in non-blocking mode to such a tty device (for example, via stdout or stderr in the above code example) may result in the UNIX kernel blocking the process if there was insufficient kernel buffers available for this operation. This behavior, however, contravenes POSIX standards.

This was resolved in the XPG4 EFS, and later in SCO OpenServer 5.

So now, in SCO OpenServer 5, non-blocking write operations to tty
devices can return without all of the requested data having been
written. Therefore, the kernel will instead perform a "short
write," writing only as much as the device can accept in a single
operation.  This is now correct behavior; it is normal for the
write(S) system call to return after having written only part of
the buffer.

It is the application's responsibility to verify the actual amount of data written. The problem arises because many applications do not, in fact, check for successful writes to terminal devices. They assume that all writes to a terminal, even large writes done with what was supposed to be a non-blocking descriptor, complete successfully.

This type of programming practice can therefore result in apparent data corruption on output to terminal devices.

Note that, in practice, very few programs actually require non-blocking output.

Solution

To address this screen corruption problem, consider these solutions:

- If you do not have access to the original source code, then pipe the application output through cat(C) or tee(C):

my_app | cat -
or:
my_app | tee /dev/null

Both cat(C) and tee(C) check the return values of their write requests which usually resolves the problem.

- Recompiling your source code on SCO OpenServer 5 may address this problem.

- Your application code must always check the results of write(S) system calls. By enabling non-blocking I/O (O_NDELAY or O_NONBLOCK) on stdout, write requests may return indicating that not all the data was output. In this case, when issuing a write request, it is imperative to verify that the desired number of bytes was written, and to issue additional writes as necessary to complete the desired output. For example:

int do_write(int fd, void *buffer, int bytes) { int result, written = 0;

while (bytes > 0) {
        result = write(fd, buffer, bytes);
        if (result < 0)
                return result; /* pass back error code */
        if (result == 0)
                nap(50); /* don't do a hard busy-loop */
        else {          /* NOTE: link with -lx for nap(S) */
                bytes -= result;
                buffer = (void *)((char *)buffer + result);
                written += result;
        }
}
return written;
}

- Close stdout and stderr, and then reopen them using open(S) and dup(S). This will create separate File Table entries for each one of them:

int fd;
char *tty;
...
close(1);
close(2);
tty = (char *) ttyname(0);
fd = open(tty, O_WRONLY);
dup(fd);        /* Reopens stdout (fd 1) */
                /* Alternatively "dup2(fd, 1)" would work */
close(fd);

/* Use open(S), dup(S) or dup2(S) to set up stderr (fd 2) accordingly */

- If the application is curses-based, consider using the curses routine nodelay(S) instead of fcntl(S). For example, if the original code was:

flags = fcntl(0, F_GETFL, 0);      /* Obtain current flags */
fcntl(0, F_SETFL, flags|O_NDELAY); /* Make input non-blocking */
ch = wgetch(win);                  /* Read a character */
fcntl(0, F_SETFL, flags);          /* Restore original flags */
it could be changed to:
nodelay(win, 1);                   /* Make input non-blocking */
ch = wgetch(win);                  /* Read a character */
nodelay(win, 0);                   /* Input is blocking again */

From the Xinuos technical article database, record 105040. Reproduced with permission. The original is at www.sco.com/ta/105040, which is reachable but blocked from search engines. Xinuos marks this article against a retired product line. That refers to the product’s lifecycle, not to whether the advice still works — plenty of it still applies to running systems.

Next step

Need this fixed rather than explained?

Tell us the release, what the system runs and what it is doing. We will reply by email with what we would do next.

  • You deal with the people who do the work
  • No obligation, and no upgrade required before we will help
  • We tell you plainly if something is not worth doing

Send an enquiry

We reply by email. If a server is down, say so in the first line. You can also write to help@scoexpert.com.