Technical article 109214

"WARNING: timeout - timeout table overflow (5000 exceeded)" in OpenServer 5.x.

OpenServer 5.0.0OpenServer 5.0.6

Written · Last revised

Problem

The following warning message appears on the console:

WARNING: timeout - timeout table overflow (5000 exceeded)

and all of the ttys are locked.

I have searched the Technical Articles and found an article with this same issue. However, it says to increase the NCALL kernel parameter. I have searched all of the options in the /etc/conf/cf.d/configure script, but there is no NCALL. Where can I find it or how can I resolve the problem?

Cause

NCALL is a kernel parameter that pertains to a release prior to SCO OpenServer 5 (SCO UNIX System V/386 Version 4.x). In SCO OpenServer 5, it has been replaced by MAXCALL. This parameter is not configurable. It has been set to a maximum of 5000, which is the maximum number of pending callouts.

If a kernel routine has to execute a function at some specified time in the future, it uses the timeout(K) function. This routine arranges for a function to be called with a specified argument in a specified number of clock ticks. In order for a timeout to take place, the timeout function must be triggered into operation when the specified number of clock ticks expires. There may be many instances of functions to time out, so the system must keep them in a table to process appropriately. This table is called the "callout table". If the callout[] table overflows, the system panics with the message "Timeout table overflow".

Solution

The callout[] table is set to a reasonably high number, and for it
to overflow means that you have more than 5000 entries waiting to
be triggered. Take note of the time the system panics and the
corrosponding warning message. It could be an indication that all
the functions are set to run at once, or of applications that have
set timers to perform tasks at a later time.  You will need to find
out why there are more than 5000 entries in the callout[] table.

Check all applications that set timers, alarms, or will perform a task at a specified time in the future. Cron jobs and backup schedules are usually set to go off at a specific time. If you have a lot of these jobs scheduled, in addition to 3rd party drivers or applications that use the timeout(K) function, the callout[] table will eventually get filled and will panic.

crash(ADM) is a utility that will help in determining which functions are to be called at some point in the future. The "callout" command in crash(ADM) prints the contents of the callout[] table. It will print the entries in the order in which they appear in the table, which is also the order in which they will be triggered by the clock(S) function.

The following is an output of the crash(ADM) utility calling the callout command:

# crash
dumpfile = /dev/mem, namelist = /unix, outfile = stdout
> callout
FUNCTION          ARGUMENT        TIME          ID        PROC
itsendalarm     0xfbf52c60           4    43230607           0
wdtimeout                0           7    43230544           0
e3e_tick        0xf02de4a4           0    43230545           0
in_fasttimo              0           5    43230604           0
in_slowtimo              0          29    43230603           0
unselect        0xfb1199a8         134    43230437           0
unselect        0xfb11bb40         517    43229227           0
unselect        0xfb11e4e8         323    43229233           0
unselect        0xfb117ac0          70    43229627           0
ip_pmtu_age              0         492    43228302           0
unselect        0xfb117d70        3078    43230282           0
unselect        0xfb118428         750    43230300           0
arptimer                 0       20143    43228281           0
unselect        0xfb11e640       21251    43229232           0
kmwakeup                 0        8763    43213340           0
unselect        0xfb11e798       21911    43224262           0
unselect        0xfb11a5c0      190623    42743793           0
unselect        0xfb119b00     1292921    40651162           0
unselect        0xfb1182d0      639490    43185957           0
unselect        0xfb11b330     1098616    41217201           0

FUNCTION - is the name of the function to call. The sample output shows that the itsendalarm() will be the next function called, followed by wdtimeout(), and so on.

ARGUMENT - is the single argument that will be passed to the function when it is called.

TIME - is the time in HZ that will elapse before the function is called. The output shows that itsendalarm() will be called in 0.4 seconds time. The time between each function call is incremental. Therefore, the second function wdtimeout() will be called in 0.11 second's time (0.4 + 0.7).

ID - is the timeout ID. This value is used when a function is cancelled by the untimeout(K) function. The timeout ID is incremented every time timeout(K) is called.

NOTE:     After finding out which functions are in the callout[] table, the
          task now is to identify the location of the function. Two utilities
          that are helpful for searching binary files or the kernel are
          strings(C) and nm(CP).

- strings(C) can be used to locate error messages within binary files such as Driver.o or the UNIX kernel. It will not help identify functions.

- nm(CP) is useful for locating functions, but only comes with the Development System and is frequently not available.

There is also a script that will search text files from the current directory through subdirectories for the string indicated. It is included below.

Note: This is for text files only. This will not locate anything in binary files.

Place it in the path (we recommend /usr/bin) and set permissions for execute:

-rwxr-xr-x   1 root  group   634 Mar 12  1997 /usr/bin/findstring
------------------------------------<findstring>--------------------------
----
:
# findstring    05/17/96 dond
#
# to use:
# cd <dir to search>
# nohup nice -10 findstring <search string (spaces ok)> &
#
# output in $HOME:
# findstring.out - files and line#s where strings are found
# findstringDONE - empty file flagging completion of findstring
#
trap 'rm core; exit' 3
findfile=$HOME/findstring.out
donefile=$HOME/findstringDONE
exec 2>/dev/null
rm $findfile $donefile
echo "$*" > $findfile
pwd >> $findfile
date >> $findfile
echo "" >> $findfile
find . -type f -follow ! -name findstring.out -print   \
 | xargs grep -ny "$*" >> $findfile
echo "" >> $findfile
echo '*** DONE ***' >> $findfile
touch $donefile
--------------------------------<done>--------------------------------

From the Xinuos technical article database, record 109214. Reproduced with permission. The original is at www.sco.com/ta/109214, which is reachable but blocked from search engines. Xinuos marks this article against a retired product line. That refers to the product’s lifecycle, not to whether the advice still works — plenty of it still applies to running systems.

Next step

Need this fixed rather than explained?

Tell us the release, what the system runs and what it is doing. We will reply by email with what we would do next.

  • You deal with the people who do the work
  • No obligation, and no upgrade required before we will help
  • We tell you plainly if something is not worth doing

Send an enquiry

We reply by email. If a server is down, say so in the first line. You can also write to help@scoexpert.com.