Technical article 112723

How do I set my disk buffering for SCO OpenServer 5 to the optimum values?

OpenServer 5.0.0OpenServer 5.0.2OpenServer 5.0.4OpenServer 5.0.5OpenServer 5.0.6OpenServer 5.0.7

Written · Last revised

Problem

I have identified a disk buffering bottleneck using sar. How do I increase disk buffering for SCO OpenServer 5?

Cause

sar -b reports buffer cache activity:
# sar -b 1 5
SCO_SV tiger5 3.2v5.0.6 Pentium    03/08/2001
11:26:50 bread/s lread/s %rcache bwrit/s lwrit/s %wcache pread/s pwrit/s (-b)
11:26:51     145     145       0       0       0       0       0       0
11:26:52     134     134       0       0       0       0       0       0
11:26:53     130     522      75     121     196      38       0       0
11:26:54      96      96       0       0       0       0       0       0
11:26:55     106     106       0       0       0       0       0       0
Average      122     200      39      24      39      38       0       0
The columns show the numbers of actual disk read/write
operations (cache misses), and the number of buffer cache
read/write operations (cache hits). Each operation is a
1 Kbyte transfer.  The columns also show read/write hit
ratios, and the number of raw I/O requests.
The most useful statistic is the read hit ratio (%rcache column),
which shows you the effectiveness of the buffer cache. On a typical
system you should expect a hit ratio of 85% OR HIGHER.  If this
degrades significantly, try to increase the size of the buffer
cache.
You would probably need to increase buffer cache if %wcache is
less than circa 65%, even if %rcache is 100%, as long as the
system is not paging or swapping.  You can see this with the
"sar -p" command.
Alternatively, you could get a faster drive to increase the
throughput onto the disk or reorganize your application so
that several spindles can be utilized.  See sar(ADM).

The disk load caused by the actual disk read/write operations can be investigated with "sar -d":

# sar -d 1 5
SCO_SV tiger5 3.2v5.0.6 Pentium    03/08/2001
11:27:26  device   %busy     avque     r+w/s    blks/s    avwait    avserv (-d)
11:27:27 Sdsk-0   100.00      1.43    100.00   1797.03      8.91     20.79
11:27:28 Sdsk-0   100.00      1.38    113.86   2208.91      6.70     17.83
11:27:29 Sdsk-0   100.00      1.44    112.87   2168.32      7.46     16.93
11:27:30 Sdsk-0   100.00      1.34    109.90   2417.82      7.39     21.71
11:27:31 Sdsk-0   100.00      1.52    121.57   2741.18     10.56     20.32
Average  Sdsk-0   100.00      1.42    111.66   2267.59      8.23     19.49
To simulate this load, NBUF value was set deliberately low, to
"200".  Then the following commands were run to demonstrate
some heavy test load on the server:
dd if=/dev/zero of=/tmp/testfile bs=1024 count=500000 &
dd if=/dev/zero of=/tmp/testfile1 bs=1024 count=500000 &

This generated 2 x 500MB files in /tmp.

Solution

Prior to SCO OpenServer 5.0.6, the automatic buffer cache setting would set aside 10% of memory to disk buffers and cap at about 6.3MB (6652 1k buffers).

For example, SCO OpenServer 5.0.5 would allocate the following I/O buffers for 64MB of RAM:

mem: total = 65148k, kernel = 11188k, user = 53960k swapdev = 1/41, swplo = 0, nswap = 300000, swapmem = 150000k rootdev = 1/42, pipedev = 1/42, dumpdev = 1/41 kernel: Hz = 100, i/o bufs = 6300k

The i/o bufs are 10% of memory.

For OpenServer 5.0.6 this cap has been removed and will scale up to a new limit of 450MB.

However, in order to achieve better results with your buffer cache,
you may wish to increase the amount of memory used for the buffer
cache.  In this example, 25% of memory is allocated:

mem: total = 65148k, kernel = 23240k, user = 41908k swapdev = 1/41, swplo = 0, nswap = 300000, swapmem = 150000k Autoboot from rootdev = 1/42, pipedev = 1/42, dumpdev = 1/41 kernel: Hz = 100, i/o bufs = 16384k (high bufs = 11472k)

The i/o bufs are now 25% of memory.

These disk buffering caches can be increased manually.

Changing Kernel parameters to anything above this may improve the performance of the disk buffering, but may direct valuable memory from an important resource to disk buffering. Any changes to kernel parameters should be undertaken with caution.

To change these values, use the following command:

/etc/conf/cf.d/configure

Select 1 - Buffer Management Enter the figure for NBUF, when prompted Enter the figure for NHBUF, when prompted Enter RETURN to accept the current values for the remaining parameters OR you can simply enter "q" to skip the remaining parameters.

If NHBUF is not a power of 2 then this will be highlighted when you attempt to "q" from the main menu and answer "y" to update the system configuration files with your changes.

To adjust the value of NBUF so that it takes up 25% of memory, just take .25 times the amount of memory on your system, expressed in MB, and multiply by 1024 to get the value of NBUF, as:

RAM in MB x (1024*1024) bytes/1MB x (1 disk buffer/1024 bytes) x 25%

Example: NBUF  <25% of memory> eg, 64MB = ((64 x 1024) x 0.25) = 16384
NBUF has a companion parameter, NHBUF.  This is best left at 0,
telling the kernel to choose an appropriate value based on NBUF.

If you do change NHBUF remember that it must be an exact power of 2.

The algorithms for the various versions of SCO OpenServer, detailing what value NHBUF should be set to, are:

For OpenServer 5.0.0, 5.0.2, 5.0.4, and 5.0.5:

NHBUF   <HALF NBUF (to the next power of 2)
                for Single Processor Servers>
        <Next exact power of 2 larger than NBUF
                for Multi Processor Servers>
For OpenServer 5.0.6/5.0.7:

The autotune algorithm for NHBUF changed in OpenServer 5.0.6/5.0.7 so that Single and MultiProcessor servers use the same values:

NHBUF    <Next exact power of 2 larger than NBUF>
Typical settings from the field are:
MEM      NBUF      ONE CPU NHBUF    MULTI-CPU NHBUF
64MB     16384     8192             n/a - leave at auto values
128MB    32768     16384            - as above
256MB    65536     32768            - as above
512MB    131072    65536            - as above
1GB      262144    131072           - as above
2GB      450000    262144           - as above
3GB      450000    262144           - as above
4GB      450000    262144           - as above

Now relink the kernel with this command:

/etc/conf/cf.d/link_unix

Reboot your server for the changes to take effect.

NOTE:     25% is just an example.  The actual needs of individual systems
          vary.  Also, it is not possible to assign 25% of the largest
          systems' RAM to buffers since the upper limit on NBUF is 450000,
          while OpenServer 5.0.4 (and later) support up to 4GB of RAM.

By default, "sar" is not enabled on SCO OpenServer. To enable it, use this command:

/usr/lib/sa/sar_enable -y

See also

ta103492, "Tuning large systems for performance."

ta106129, "Suggestions for improving UNIX disk performance."

ta107228, "Large document for tuning SCO UNIX System and SCO Open Desktop kernels."

ta110628, "Large document for tuning SCO UNIX System and SCO Open Desktop kernels - cont'd."

The Online Performance Guide at:
http://your_server:457/PERFORM/CONTENTS.html

This web site provides UNIX performance tuning tools:

http://www.sarcheck.com

From the Xinuos technical article database, record 112723. Reproduced with permission. The original is at www.sco.com/ta/112723, which is reachable but blocked from search engines.

Next step

Need this fixed rather than explained?

Tell us the release, what the system runs and what it is doing. We will reply by email with what we would do next.

  • You deal with the people who do the work
  • No obligation, and no upgrade required before we will help
  • We tell you plainly if something is not worth doing

Send an enquiry

We reply by email. If a server is down, say so in the first line. You can also write to help@scoexpert.com.