Technical article 116102

SECURITY: OpenServer, What is CSSA-2001-SCO.35.2, concerning the "setcontext and sysi86 vulnerabilities"?

OpenServer 5.0.4OpenServer 5.0.5OpenServer 5.0.6

Written · Last revised

Problem

The previous release of this fix was flawed, and required a recut. If CSSA-2001-SCO-35 or CSSA-2001-SCO-35.1 has already been applied, SCO strongly recommends that you install the updated fix.

Please note: Before installing CSSA-2001-SCO.35.2, any previously installed version of CSSA-2001-SCO.35 must be removed.

This patch closes a family of security holes present in SCO OpenServer 5.0.6 and previous releases, which stem from the ability of regular user processes to reprogram segment descriptors and certain other CPU registers.

This patch can be applied on SCO OpenServer 5.0.4, 5.0.4c, 5.0.5,
5.0.5a, 5.0.6, or 5.0.6a.  This fix will be included in releases
subsequent to 5.0.6a.
Closing this family of security holes does, however, result in
a functionality change, which may prevent certain applications from
running -- they will exit with an error, or dump core, instead of
running properly.  For example, the i286emul and x286emul emulators
will not work, so any '286 executables that require those emulators
will not work.  It is our intention to enhance this patch in the
future, such that it enables full functionality of such applications
while still closing the security holes.
If this patch breaks any crucial applications, the system
administrator may choose to disable the patch, by editing the file
/etc/conf/pack.d/kernel/space.c and setting the value of the
allow_dscr_remap parameter to 1.  This will return the kernel to the
old (non-secure) behavior.

Solution

          Please see the security advisory below for details.
___________________________________________________________________________
SCO, Inc. Security Advisory
Subject:         REVISED: OpenServer: setcontext and sysi86 vulnerabilities
Advisory number: CSSA-2001-SCO.35.2
Issue date:      2002 January 21
Cross reference: CSSA-2001-SCO-35 CSSA-2001-SCO.35.1
___________________________________________________________________________
1. Problem Description

The previous release of this fix was flawed, and required a recut. If CSSA-2001-SCO-35 or CSSA-2001-SCO-35.1 has already been applied, SCO stongly recommends that you install the updated fix.

Please note: Before installing CSSA-2001-SCO.35.2, any previously installed version of CSSA-2001-SCO.35 must be removed.

This patch closes a family of security holes present in SCO OpenServer 5.0.6 and previous releases, which stem from the ability of regular user processes to reprogram segment descriptors and certain other CPU registers.

This patch can be applied on SCO OpenServer 5.0.4, 5.0.4c,
5.0.5, 5.0.5a, 5.0.6, or 5.0.6a.  This fix will be included in
releases subsequent to 5.0.6a.
Closing this family of security holes does, however, result in
a functionality change which may prevent certain applications
from running -- they will exit with an error, or dump core,
instead of running properly.  For example, the i286emul and
x286emul emulators will not work, so any '286 executables
which require those emulators will not work.  It is our
intention to enhance this patch in the future, such that it
enables full functionality of such applications while still
closing the security holes.
If this patch breaks any crucial applications, the system
administrator may choose to disable the patch, by editing the
file /etc/conf/pack.d/kernel/space.c and setting the value of
the allow_dscr_remap parameter to 1.  This will return the
kernel to the old (non-secure) behavior.
2. Vulnerable Versions
Operating System        Version  Affected Files
------------------------------------------------------------------
OpenServer              All     /etc/conf/pack.d/kernel/os.a(machdep.o)
                                /etc/conf/pack.d/kernel/os.a(sysi86.o)
3. Workaround
None.
4. OpenServer
4.1 Location of Fixed Binaries
ftp://stage.sco.com/pub/security/openserver/CSSA-2001-SCO.35.2/
4.2 Verification
md5 checksums:
MD5 (CSSA-2001-SCO.35.2.tar) = 08a06921bfc5050020a459595f05a146
md5 is available for download from
ftp://stage.sco.com/pub/security/tools/
4.3 Installing Fixed Binaries
***  Before installing CSSA-2001-SCO.35.2, any previously    ***
***  installed version of CSSA-2001-SCO.35 must be removed.  ***

Upgrade the affected binaries with the following commands:

Download the CSSA-2001-SCO.35.2.tar file to /tmp.
# cd /tmp
# tar xvf CSSA-2001-SCO.35.2.tar
# custom

Instruct custom to install from images, and supply /tmp as the directory of the VOL image.

5. References
ftp.netbsd.org/pub/NetBSD/security/advisories/NetBSD-SA2001-002.txt.asc

This and other advisories are located at [link no longer available]

This advisory addresses SCO Security internal incidents sr855993, sr855994, SCO-559-1328, SCO-559-1329, erg711906 and erg711905.

6. Disclaimer

SCO, Inc. is not responsible for the misuse of any of the information we provide on our website and/or through our security advisories. Our advisories are a service to our customers intended to promote secure installation and use of SCO products.

7. Acknowledgements
        SCO wishes to thank the Last Stage of Delirium Research
        Group (contact@lsd-pl.net) for their discovering of, and
        research into, these issues.
___________________________________________________________________________

From the Xinuos technical article database, record 116102. Reproduced with permission. The original is at www.sco.com/ta/116102, which is reachable but blocked from search engines. Xinuos marks this article against a retired product line. That refers to the product’s lifecycle, not to whether the advice still works — plenty of it still applies to running systems.

Next step

Need this fixed rather than explained?

Tell us the release, what the system runs and what it is doing. We will reply by email with what we would do next.

  • You deal with the people who do the work
  • No obligation, and no upgrade required before we will help
  • We tell you plainly if something is not worth doing

Send an enquiry

We reply by email. If a server is down, say so in the first line. You can also write to help@scoexpert.com.