Technical article 109567

run_setkey function does not work with crypt(C) utility furnished by OSS425C.

OpenServer 5.0.0OpenServer 5.0.2OpenServer 5.0.4

Written · Last revised

Problem

I have installed the SCO OpenServer Development System and Support
Level Supplement (SLS) OSS425C, the Domestic Crypt Supplement, in
order to gain the ability to create user applications that can
encrypt and decrypt data strings.  I have created an encrypted
version of the file "plain_text" called "confidential" with the
crypt(C) command:
crypt < plain_text > confidential

When invoking this command, crypt(C) prompts me for a crypt key, which I can use later to obtain decrypted data from "confidential". I specify "topsecret" as my crypt key.

Now I want to write an application that grabs a line of encrypted text from "confidential" and uses the key "topsecret" to decrypt the line.

The following program grabs the first text line from "confidential"
and employs the functions run_setkey() and run_crypt() (see crypt(S)
for details) to decrypt this text.  However, run_setkey() fails
and returns -1, as does the subsequent call to run_crypt().
/*****Begin sample program*****/
/*  compile with -lcrypt */
#include <stdio.h>
void main()
{
FILE   *fp,*fpn;
char   s[81];
int    r;
int    p[2];

if ((fp= fopen("confidential", "r")) == NULL) { fprintf(stderr, "Cannot open file confidential\n"); exit(1); } fgets(s,80,fp);

p[0]=p[1]=0;

/* run_setkey() will fail and return -1 */ printf("run_setkey returned: %d\n",run_setkey(p,"topsecret"));

        r=strlen(s);
        printf("%s\n",s);
        printf("run_crypt returned: %d\n", run_crypt(0L,s,r,p));
        printf("%s\n",s);
        crypt_close(p);
        fclose(fp);
 }
/*****End sample program*****/

Cause

The function run_setkey() is supposed to invoke the crypt(C) utility
and open a communication channel between the current process and
crypt(C).  This functionality is broken in the SCO OpenServer
Development System releases stated above.

Solution

This has been fixed in SCO OpenServer 5.0.5 with the SCO OpenServer 5.0.5 Development System and SLS OSS425C.

NOTE:     For OpenServer 5.0.4, a workaround is to create a link between
          /bin/sh and /sbin/sh:
mkdir /sbin
ln /bin/sh /sbin/sh

The above program should now work, as should more significant applications of run_setkey() and run_crypt().

From the Xinuos technical article database, record 109567. Reproduced with permission. The original is at www.sco.com/ta/109567, which is reachable but blocked from search engines. Xinuos marks this article against a retired product line. That refers to the product’s lifecycle, not to whether the advice still works — plenty of it still applies to running systems.

Next step

Need this fixed rather than explained?

Tell us the release, what the system runs and what it is doing. We will reply by email with what we would do next.

  • You deal with the people who do the work
  • No obligation, and no upgrade required before we will help
  • We tell you plainly if something is not worth doing

Send an enquiry

We reply by email. If a server is down, say so in the first line. You can also write to help@scoexpert.com.