Technical article 109287

How do I set up a serial console on SCO OpenServer 5?

OpenServer 5.0.0OpenServer 5.0.2OpenServer 5.0.4

Written · Last revised

Problem

I need to configure a serial terminal.  I also need it to run
as the main console screen.

Cause

There are a number of reasons why a serial terminal can be helpful.
One reason is to attempt to regain control of a system that has
hung.  The serial driver operates at a higher level of priority
than the standard keyboard driver and may be operational if the
keyboard is no longer functional.

Solution

To set up the terminal you will need to connect it to a serial
port.  The examples here are for "COM 1", "tty1a".  You will need
to modify the examples to suit a need other than for COM 1.

NOTE: To be a serial console the terminal must be connected to either COM 1 or COM 2.

The initial setup to be able to access the operating system from a serial terminal will be:

1.  Enable the appropriate COM port in the systems BIOS.  Note
the IRQ and Address for later reference.
2.  Boot the system and go into single-user mode.
3.  Run "hwconfig -hc" to see if the serial port you will be
using is already configured into the kernel.

If the serial port is already configured you will see (among other devices):

device
======
serial -- with the associated address, vec and comment
4.  If the serial port is not displayed in "hwconfig -hc" you
will need to run the following to configure the serial port:
mkdev serial
The option to use is "IBM - COM 1" or "IBM - COM 2".  Be sure
that the Address and IRQ match the settings that are in the
system CMOS.

Once the serial port is configured, the kernel will be relinked and the system will need to be shut down and restarted.

For assistance with configuring the serial device, see the man page for mkdev:

man ADM mkdev
5.  When the system restarts, go into multiuser mode.  You will
need to run the following command to enable the serial port for
login use:
# enable /dev/tty1a

Note: Again this example is for COM port 1 (serial port 1); if you are using an alternate port please modify the command to meet your needs.

If the above command is issued in single-user mode, the terminal will not be active until you go into multiuser mode since only the console is active in single-user mode.

6.  If the terminal is properly connected to the serial port,
you should see a login prompt on the terminal as soon as the
port is enabled.
******************* IMPORTANT ****************************

You must see a login prompt and have the serial terminal functional before going on the next section.

**********************************************************
===========================================================
The next section is for configuring the system so that the
serial terminal will be the console (the normal multi-screens
will remain operational).  Note: To be console the serial
terminal must be connected to COM 1 or COM 2.
Do the following in single-user mode:
7.  Using your favorite editor, edit the file "/etc/default/login"
and change the line:
OVERRIDE=tty01

to add "#" so that it looks like this:

#OVERRIDE=tty01
Then add this line:
OVERRIDE=tty1a

Note: tty1a is again for COM port 1.

Once the changes have been made you can save and exit this file.

8.  Edit the file "/etc/default/boot".

At the end of the file, add this line:

SYSTTY=1
Note: The "1" is for COM 1.  "0" would be for the video console,
which is the default setting.  No other values of SYSTTY are
allowed.  However, it is still possible to specify a COM port
other than COM 1 for use as a serial console.  In order to do so,
you must add "systty=sio(MINOR)" to the default bootstring. Here,
MINOR is the minor number of the lower-case (non-modem-control) port
device name.  For example, if currently you have a line in
/etc/default/boot which reads "DEFBOOTSTR=hd(40)unix", and you
you wish to configure COM 2 as the console, modify this line
so that it reads "DEFBOOTSTR=hd(40)unix systty=sio(8)", where 8
is the minor number of /dev/tty2a. This value may vary, so it
is important to verify the minor number with "ls -l /dev/tty2a".
9.  If you have not enabled the serial port the terminal is
connected to yet, do so before going any further.
Example: "enable /dev/tty1a"
10. Make sure that the terminal is set up for it to function
properly.  A common setup is:
DUPLEX=full
BAUD=9600
DATA BITS=8
STOP BITS=1
PARITY=NONE

Note: the above settings are the settings OpenServer will use by default.

See the reference documentation or contact the terminal maker for any specific setup options that may be needed for specific terminals.

11. Note: The SCO Help section on setting up a serial terminal
indicates that you should disable tty01 - tty12.  However, this
is not necessary for the terminal to be the console.

Upon reboot the serial terminal will be the console and should be used for operations requiring the console.

You can now shut down and restart the system.  If the serial
terminal is acting as the console, the initial bootup messages
and control options will be displayed to the terminal instead
of the normal monitor.
============================================================
Disabling the serial terminal

To disable the serial terminal as the console you will need to reset the files discussed in the section above. Here we consider the COM 1 case as a predominant example. The steps are the same for COM2, with "tty2a" replacing "tty1a" below, _except_ that instead of removing "SYSTTY=1" from /etc/default/boot, you would remove the string "systty=sio(MINOR)" from DEFBOOTSTR, where MINOR is the minor number of /dev/tty2a. See the previous note under step 8 for more details.

Edit "/etc/default/login" and change:
OVERRIDE=tty1a
to read:
OVERRIDE=tty01
Edit "/etc/default/boot" and remove the line:
SYSTTY=1

To disable the serial terminal as a login port, enter the command:

# disable /dev/tty1a

using the tty## that is applicable for your setup.

==============================================================

See also

SCO OpenServer Handbook, Chapter 24, Adding serial terminals. (This can also be accessed via the SCO Help icon).

man pages mkdev(ADM), boot(F)

TA106049, "Nine to twenty-five pin adapters for serial communications"

TA107165, "How should I set up my cable for a serial printer or terminal?"

From the Xinuos technical article database, record 109287. Reproduced with permission. The original is at www.sco.com/ta/109287, which is reachable but blocked from search engines. Xinuos marks this article against a retired product line. That refers to the product’s lifecycle, not to whether the advice still works — plenty of it still applies to running systems.

Next step

Need this fixed rather than explained?

Tell us the release, what the system runs and what it is doing. We will reply by email with what we would do next.

  • You deal with the people who do the work
  • No obligation, and no upgrade required before we will help
  • We tell you plainly if something is not worth doing

Send an enquiry

We reply by email. If a server is down, say so in the first line. You can also write to help@scoexpert.com.