Technical article 127183

OMS 4.2:Now can I archive my log files?

OpenServer 6.0.0UnixWare 7.1.4

Written · Last revised

Problem

How can I rotate my log files manually using OMS 4.2?

Solution

OMS 4.2 installs the following in the cron for root:

0 6 * * * /opt/insight/etc/logrotate.cron >/dev/null 2>&1

This can be run manually to compress you log files in /opt/insight/logs.

The configuration file for this
/opt/insight/etc/logrotate.conf

The default size=1024M which is 1GB file size.

"vi" is not likely to be able to edit a file beyond 30Mb so you may wish to edit the file to set a 30M file size for log rotation.

NOTES:    Please note that the man page for "logrotate" will not be available
          on OpenServer6.
See:
LOGROTATE(8)          MAINTENANCE COMMANDS           LOGROTATE(8)

NAME logrotate - rotates, compresses, and mails system logs

SYNOPSIS
     logrotate [-dv] [-f|--force] [-s|--state file] config_file+
DESCRIPTION
     logrotate is designed to ease administration of systems that
     generate  large  numbers  of log files.  It allows automatic
     rotation, compression, removal, and mailing  of  log  files.
     Each log file may be handled daily, weekly, monthly, or when
     it grows too large.  Normally, logrotate is run as  a  daily
     cron  job.   It  will not modify a log multiple times in one
     day unless the criterium for that log is based on the  log's
     size  and logrotate is being run multiple times each day, or
     unless the -f or -force option is used. Any number of config
     files  may  be given on the command line. Later config files
     may override the options given  in  earlier  files,  so  the
     order  in  which the logrotate config files are listed in is
     important.  Normally, a single config  file  which  includes
     any other config files which are needed should be used.  See
     below for more information on how to use the include  direc-
     tive  to  accomplish  this.   If a directory is given on the
     command line, every file in that directory is used as a con-
     fig file.  If no command line arguments are given, logrotate
     will print version and copyright information, along  with  a
     short  usage  summary.   If  any errors occur while rotating
     logs, logrotate will exit with non-zero status.
OPTIONS
     -d   Turns on debug mode and implies -v.  In debug mode,  no
          changes  will  be  made to the logs or to the logrotate
          state file.
-f, --force
     Tells logrotate to  force  the  rotation,  even  if  it
     doesn't  think  this  is  necessary.  Sometimes this is
     useful after adding new entries to logrotate, or if old
     log  files  have been removed by hand, as the new files
     will be created, and logging will continue correctly.
-m, --mail <command>
     Tells logrotate which command to use when mailing logs.
     This  command  should accept two arguments: 1) the sub-
     ject of the message, and 2) the recipient. The  command
     must  then read a message on standard input and mail it
     to the recipient. The default mail command is /bin/mail
     -s.
-s, --state <statefile>
     Tells logrotate to use an alternate state  file.   This
     is useful if logrotate is being run as a different user
     for various sets of log files.  The default state  file
     is /var/lib/logrotate/status.
--usage
     Prints a short usage message.
CONFIGURATION FILE
     logrotate reads everything about the log files it should  be
     handling from the series of configuration files specified on
     the command line.  Each configuration file  can  set  global
     options  (local  definitions override global ones, and later
     definitions override earlier ones) and specify a logfile  to
     rotate. A simple configuration file looks like this:
# sample logrotate configuration file
compress
/var/log/messages {
    rotate 5
    weekly
    postrotate
                              /sbin/killall -HUP syslogd
    endscript
}
"/var/log/httpd/access.log" /var/log/httpd/error.log {
    rotate 5
    mail www@my.org
    size=100k
    sharedscripts
    postrotate
                              /sbin/killall -HUP httpd
    endscript
}
/var/log/news/news.crit {
    monthly
    rotate 2
    olddir /var/log/news/old
    missingok
    postrotate
                              kill -HUP `cat /var/run/inn.pid`
    endscript
    nocompress
}
The first few lines set global options; in the example, logs
are  compressed  after they are rotated.  Note that comments
may appear anywhere in the config file as long as the  first
non-whitespace character on the line is a #.
The next section of the config files defined how  to  handle
the log file /var/log/messages. The log will go through five
weekly rotations before being removed. After  the  log  file
has  been rotated (but before the old version of the log has
been compressed), the  command  /sbin/killall  -HUP  syslogd
will be executed.
The  next  section   defines   the   parameters   for   both
/var/log/httpd/access.log    and   /var/log/httpd/error.log.
They are rotated whenever is grows over 100k  is  size,  and
the  old  logs files are mailed (uncompressed) to www@my.org
after going through 5 rotations, rather then being  removed.
The sharedscripts means that the postrotate script will only
be run once, not once for each log which  is  rotated.  Note
that  the  double  quotes  around  the first filename at the
beginning of this section allows logrotate  to  rotate  logs
with  spaces  in the name. Normal shell quoting rules apply,
with ', ", and \ characters supported.
The last section defines the parameters for all of the files
in  /var/log/news.  Each file is rotated on a monthly basis.
This is considered a single rotation directive and if errors
occur  for  more  then  one  file,  the  log  files  are not
compressed.
Please use wildcards with caution.  If you specify *, logro-
tate  will  rotate  all  files, including previously rotated
ones.  A way around this is to use the olddir directive or a
more exact wildcard (such as *.log).
Here is more information on  the  directives  which  may  be
included in a logrotate configuration file:
compress
     Old versions of log files are compressed with  gzip  by
     default. See also nocompress.
compresscmd
     Specifies which command to use to compress  log  files.
     The default is gzip.  See also compress.

uncompresscmd Specifies which command to use to uncompress log files. The default is gunzip.

compressext
     Specifies which extension to  use  on  compressed  log-
     files,  if compression is enabled.  The default follows
     that of the configured compression command.
compressoptions
     Command line options may be passed to  the  compression
     program,  if  one is in use.  The default, for gzip, is
     "-9" (maximum compression).
copy Make a copy of the log file, but don't change the  ori-
     ginal  at  all.  This option can be used, for instance,
     to make a snapshot of the current  log  file,  or  when
     some  other utility needs to truncate or pare the file.
     When this option is used, the create option  will  have
     no effect, as the old log file stays in place.
copytruncate
     Truncate the original log file in place after  creating
     a  copy, instead of moving the old log file and option-
     ally creating a new one, It can be used when some  pro-
     gram  can  not  be  told  to close its logfile and thus
     might continue writing (appending) to the previous  log
     file  forever.   Note  that  there is a very small time
     slice between copying the file and  truncating  it,  so
     some  logging  data might be lost.  When this option is
     used, the create option will have no effect, as the old
     log file stays in place.
create mode owner group
     Immediately  after  rotation  (before  the   postrotate
     script  is  run) the log file is created (with the same
     name as the log file just rotated).  mode specifies the
     mode  for the log file in octal (the same as chmod(2)),
     owner specifies the user name  who  will  own  the  log
     file,  and  group specifies the group the log file will
     belong to. Any of the log file attributes may be  omit-
     ted,  in  which  case those attributes for the new file
     will use the same values as the original log  file  for
     the  omitted  attributes.  This  option can be disabled
     using the nocreate option.
daily
     Log files are rotated every day.
delaycompress
Postpone compression of the previous log  file  to  the
next rotation cycle.  This has only effect when used in
combination with compress.  It can be  used  when  some
program  can  not be told to close its logfile and thus
might continue writing to the  previous  log  file  for
some time.
extension ext
     Log files are given the final extension ext after rota-
     tion. If compression is used, the compression extension
     (normally .gz) appears after ext.
ifempty
     Rotate the log file even if it is empty, overiding  the
     notifempty option (ifempty is the default).
include file_or_directory
     Reads the file given  as  an  argument  as  if  it  was
     included inline where the include directive appears. If
     a directory is given, most of the files in that  direc-
     tory  are read in alphabetic order before processing of
     the including file continues. The only files which  are
     ignored  are files which are not regular files (such as
     directories and named pipes) and files whose names  end
     with  one  of the taboo extensions, as specified by the
     tabooext directive.   The  include  directive  may  not
     appear inside of a log file definition.

mail address When a log is rotated out-of-existence, it is mailed to address. If no mail should be generated by a particular log, the nomail directive may be used.

mailfirst
     When using the  mail  command,  mail  the  just-rotated
     file, instead of the about-to-expire file.
maillast
     When using the mail command, mail  the  about-to-expire
     file,  instead  of  the  just-rotated file (this is the
     default).
missingok
     If the log file is missing,  go  on  to  the  next  one
     without issuing an error message. See also nomissingok.
monthly
     Log files are rotated the first time logrotate  is  run
     in  a  month  (this is normally on the first day of the
     month).

nocompress Old versions of log files are not compressed with gzip. See also compress.

nocopy
     Do not copy the original  log  file  and  leave  it  in
     place.  (this overrides the copy option).
nocopytruncate
     Do not truncate the original log file  in  place  after
     creating   a  copy  (this  overrides  the  copytruncate
     option).
nocreate
     New log files  are  not  created  (this  overrides  the
     create option).
nodelaycompress
     Do not postpone compression of the previous log file to
     the  next  rotation  cycle  (this  overrides the delay-
     compress option).

nomail Don't mail old log files to any address.

nomissingok
     If a log file does not exist, issue an error.  This  is
     the default.

noolddir Logs are rotated in the same directory the log normally resides in (this overrides the olddir option).

nosharedscripts
     Run prerotate and postrotate scripts for  every  script
     which  is  rotated  (this is the default, and overrides
     the sharedscripts option).
notifempty
     Do not rotate the log if it is  empty  (this  overrides
     the ifempty option).
olddir directory
     Logs are moved into directory for rotation. The  direc-
     tory  must  be  on  the same physical device as the log
     file being rotated, and is assumed to  be  relative  to
     the  directory  holding the log file unless an absolute
     path name is specified. When this option  is  used  all
     old  versions  of  the  log  end up in directory.  This
     option may be overriden by the noolddir option.
postrotate/endscript
     The lines between postrotate  and  endscript  (both  of
     which  must appear on lines by themselves) are executed
     after the log file is  rotated.  These  directives  may
     only  appear  inside  of  a  log  file definition.  See
     prerotate as well.
prerotate/endscript
     The lines between  prerotate  and  endscript  (both  of
     which  must appear on lines by themselves) are executed
     before the log file is rotated and only if the log will
     actually  be  rotated. These directives may only appear
     inside of a log file  definition.   See  postrotate  as
     well.
firstaction/endscript
     The lines between firstaction and  endscript  (both  of
     which  must appear on lines by themselves) are executed
     once before all log files  that  match  the  wildcarded
     pattern are rotated, before prerotate script is run and
     only if at least one  log  will  actually  be  rotated.
     These  directives  may only appear inside of a log file
     definition. See lastaction as well.
lastaction/endscript
     The lines between lastaction  and  endscript  (both  of
     which  must appear on lines by themselves) are executed
     once after all log files that match the wildcarded pat-
     tern  are  rotated,  after postrotate script is run and
     only if at least one log is rotated.  These  directives
     may  only  appear  inside of a log file definition. See
     lastaction as well.
rotate count
     Log  files  are  rotated  <count>  times  before  being
     removed  or  mailed  to the address specified in a mail
     directive. If count is  0,  old  versions  are  removed
     rather then rotated.
size size
     Log files are rotated when they grow bigger  then  size
     bytes. If size is followed by M, the size if assumed to
     be in megabytes.  If the k is  used,  the  size  is  in
     kilobytes.  So  size  100, size 100k, and size 100M are
     all valid.
sharedscripts
     Normally, prescript and postscript scripts are run  for
     each log which is rotated, meaning that a single script
     may be run multiple times for log  file  entries  which
     match multiple files (such as the /var/log/news/* exam-
     ple). If sharedscript is  specified,  the  scripts  are
     only  run once, no matter how many logs match the wild-
     carded pattern.  However, if none of the  logs  in  the
     pattern  require  rotating, the scripts will not be run
     at  all.  This  option  overrides  the  nosharedscripts
     option.
start count
     This is the number to use as the base for rotation. For
     example,  if  you  specify  0, the logs will be created
     with a .0 extension as they are rotated from the origi-
     nal  log  files.   If  you specify 9, log files will be
     created with a .9, skipping 0-8.  Files will  still  be
     rotated  the  number  of times specified with the count
     directive.
tabooext [+] list
     The current taboo extension list is  changed  (see  the
     include  directive  for information on the taboo exten-
     sions). If a + precedes the  list  of  extensions,  the
     current taboo extension list is augmented, otherwise it
     is replaced. At startup, the taboo extension list  con-
     tains .rpmorig, .rpmsave, ,v, .swp, .rpmnew, and ~.
weekly
     Log files are rotated if the current  weekday  is  less
     then the weekday of the last rotation or if more then a
     week has passed since the last rotation. This  is  nor-
     mally the same as rotating logs on the first day of the
     week, but it works better if logrotate is not run every
     night.
FILES
     /var/lib/logrotate/status  Default state file.
     /etc/logrotate.conf        Configuration options.
SEE ALSO
     gzip(1)
NOTE:     LogRotate is also available for other System files and can be
          installed, for example, with an open source perl script that allows
          you to rotate multiple log files which you list in a single
          configuration file. It also has a number of other useful features.

To use logrotate on UW7.1.4 first install the following packages from the 7.1.4 MP4 CD, including MP4:

- perl
- perl modules
- gzip
Once these are installed, make the directory /usr/local/bin/logrotate

Copy the the two files in the files shown below:

(perl-logrotate.pl and logrotate.conf) to /usr/local/bin/logrotate
## Configuration for logrotate
# Please read perl-logrotate.pl for more information
logfile /var/adm/syslog
logfile /var/adm/log/osmlog

rotate-time 23 hours keep-files 7 compress yes status-file /var/adm/logrotate.status dryrun no

Make sure perl-logrotate.pl is executable:
# cd /usr/local/bin/logrotate
# chmod +x perl-logrotate.pl

Edit logrotate.conf and change to your personal preferences.

Each option is explained in the header of perl-logrotate.pl so its worth reading that first.

Whilst you're getting used to how it works, in logrotate.conf change dryrun to yes, this give you a chance to try the options without making any changes to the system.

To run logrotate manually simply run the perl script:

# cd /usr/local/bin/logrotate
# ./perl-logrotate.pl

Once you have a configuration that meets your requirements, add a cron entry to /var/spool/cron/crontabs/root that runs /usr/local/bin/logrotate/perl-logrotate.pl at whatever time suits on a nightly basis.

#!/usr/bin/perl
#####
## perl-logrotate.pl by Aki Tossavainen <cmouse@youzen.ext.b2.fi> (c) 2004
##
# Does a log file rotation as defined by config file.
# Does not break open logfiles, just truncates them.
#
# If you find bugs or have suggestions, please email.
#
### Config file format
# Following keywords are supported by the config file
#
## logfile <filename>
# Defines a filename to rotate
#
## rotate-time <number> <hour(s)/day(s)/week(s)>
# Defines the time to wait before next rotation. If no unit is specified
# seconds is assumed!
#
## keep-files <number>
# Defines how many log files are kept. 5 means that you'll have files 0->4.
#
## compress <yes,1,no,0>
# Should the files be compressed. You need Compress::Zlib if you enable this
#
## status-file <filename>
# Where the rotation status will be written.
#
## dryrun <yes,1,no,0>
# Whether the logs would be really rotated. This is defaulted to 1 unless
# you explicitly set it to no.
#
# Config file can have empty lines and comments beginning with #
# Do not add comments after config lines homever, that will break things.
####
use strict;
# Configuration file to use.
our $config_file = '/usr/local/bin/logrotate/logrotate.conf';
# Init variables
our @logfiles;
# no compression by default
our $compress = 0;
# dryrun - if people do not read config files...
our $dryrun = 1;
# 1 week.
our $rotate_time = 604800;
# 5 files
our $rotate_files = 5;
# locations
our $status_file = '/var/adm/logrotate.status';
# to avoid complaints
our %status;
sub parse_time($) {
    my ($t) = @_;
    my @time = split / /,$t;
    return 0 if (@time == 0);
    return 0 if ($time[0] =~ /\D/);
    my $number = $time[0];
    if (@time > 1) {
	$number = $time[0];
	# we have some definition after the number
	# hours
	$number *= 3600 if ($time[1] eq 'hour');
        $number *= 3600 if ($time[1] eq 'hours');
	# days
	$number *= 86400 if ($time[1] eq 'day');
	$number *= 86400 if ($time[1] eq 'days');
	# weeks
	$number *= 604800 if ($time[1] eq 'week');
	$number *= 604800 if ($time[1] eq 'weeks');
    }
    return $number;
}
sub parse_config($) {
    my ($file) = @_;
    open CONFIG,'<'.$file or die("Unable to open config file $file\n");
    while(<CONFIG>) {
	chomp;
	next if ($_[0] eq '#');
	my @line = split / /,$_,2;
	if (@line > 1) {
	    # they all take args
	    if ($line[0] eq 'compress') {
		if ($line[1] eq 'yes' || $line[1] eq '1') {
		    $compress = 1;
		    next;
		} elsif ($line[1] eq 'no' || $line[1] eq '0') {
		    $compress = 0;
		    next;
		}
		die("Invalid value for compress: must be one of yes,1,no,0");
	    }
	    if ($line[0] eq 'logfile') {
		push(@logfiles,$line[1]);
		next;
	    }
	    if ($line[0] eq 'rotate-time') {
		die("Invalid rotate-time '".$line[1]."'\n") if (($rotate_time = parse_time($line[1]))==0);
		next;
	    }
	    if ($line[0] eq 'keep-files') {
		die("Invalid keep-files") if ($line[1] =~ /\D/);
		$rotate_files = $line[1];
		next;
	    }
	    if ($line[0] eq 'status-file') {
		die("No status file given") if ($line[1] eq '');
		$status_file = $line[1];
		next;
	    }
	    if ($line[0] eq 'dryrun') {
                if ($line[1] eq 'yes' || $line[1] eq '1') {
                    $dryrun = 1;
                    next;
                } elsif ($line[1] eq 'no' || $line[1] eq '0') {
                    $dryrun = 0;
                    next;
                }
                die("Invalid value for dryrun: must be one of yes,1,no,0");
	    }
	} elsif (@line) {
	    die("All options have at least 1 parameter");
	}
    }
    close CONFIG;
}
parse_config($config_file);
if ($compress) {
    use Compress::Zlib;
}
# read status file
if (!-e $status_file) {
    open STATUS,'>'.$status_file or die("Unable to create status file\n");
    close STATUS;
}
open STATUS,'<'.$status_file or die("Unable to open status file\n");
while(<STATUS>) {
    chomp;
    my @line = split / /;
    if (@line == 2) {
	$status{$line[0]} = $line[1];
    }
}
close STATUS;
# now that we know what we are expected to do, we'll start
for my $logfile (@logfiles) {
    # first we try to read the entire log file into memory. then we
    # instantly truncate it, 'move' all the existing logfiles +1 forward
    # and create possibly compressed .0.bz2 file.
    # simple eh?
    # first we check the status file, maybe it doesn't need rotating as of yet
    next if (defined($status{$logfile}) && (time - $status{$logfile} < $rotate_time));
    # open file.
    print "Would rotate $logfile\n" if ($dryrun);
    if (open LOGFILE,'<'.$logfile) {
	# good... reel it in.
	my @data = <LOGFILE>;
	close LOGFILE;
	# if dryrun is one this should not execute.
	if (($dryrun == 1)||(open LOGFILE,'>',$logfile)) {
	    close LOGFILE if ($dryrun == 0);
	    # truncate worked. proceed.
	    my $str = join "",@data;
	    if ($compress) {
		$str = Compress::Zlib::memGzip($str);
	    }
	    # juggle files
	    for my $i (0..($rotate_files-2)) {
		# to avoid making keep-files + 1 files...
		my $n = $rotate_files - $i - 1;
		# if we use compressed files, we check this.
		if ($compress) {
		    if (-e $logfile.'.'.$n.'.gz') {
			if ($dryrun) {
			    print "Would unlink ".$logfile.'.'.$n.".gz\n";
			} else {
			    unlink $logfile.'.'.$n.'.gz';
			}
		    }
		    if ($dryrun) {
			print "Would rename ".$logfile.'.'.($n-1).'.gz to '.$logfile.'.'.$n.".gz\n";
		    } else {
			rename $logfile.'.'.($n-1).'.gz',$logfile.'.'.$n.'.gz';
		    }
		} else {
		    # and if not, this.
		    if (-e $logfile.'.'.$n) {
			if ($dryrun) {
                            print "Would unlink ".$logfile.'.'.$n."\n";
			} else {
			    unlink $logfile.'.'.$n;
			}
		    }
		    if ($dryrun) {
                        print "Would rename ".$logfile.'.'.($n-1)." to ".$logfile.'.'.$n."\n";
		    } else {
			rename $logfile.'.'.($n-1),$logfile.'.'.$n;
		    }
		}
	    }
	    # create the .0 file
	    if ($dryrun) {
		if ($compress) {
		    print "Would create ".$logfile.".0.gz\n";
		} else {
		    print "Would create ".$logfile.".0\n";
		}
	    } else {
		if ($compress) {
		    if (!(open LOGFILE,'>'.$logfile.'.0.gz')) {
			print "Unable to open $logfile.0.gz - placing it back to your logfile\n";
			open LOGFILE,'>>'.$logfile;
			print LOGFILE Compress::Zlib::memGunzip($str);
			close LOGFILE;
			next;
		    }
		} else {
		    if (!(open LOGFILE,'>'.$logfile.'.0')) {
			print "Unable to open $logfile.0.gz - placing it back to your logfile\n";
			open LOGFILE,'>>'.$logfile;
			print LOGFILE Compress::Zlib::memGunzip($str);
			close LOGFILE;
			next;
		    }
		}
		print LOGFILE $str;
		close LOGFILE;
	    }
	    # marking it rotated if it's really rotated.
	    $status{$logfile} = time;
	    # and that's it.
	} else {
	    print "Unable to truncate file $logfile\n";
	}
    } else {
	print "Unable to open file $logfile\n";
    }
}
if ($dryrun) {
    print "Would update status file\n";
} else {
# write status file.
    open STATUS,'>'.$status_file or die("Unable to write status information to $status_file\n");

for my $logfile (@logfiles) { print STATUS $logfile.' '.$status{$logfile}."\n"; } close STATUS; }

From the Xinuos technical article database, record 127183. Reproduced with permission. The original is at www.sco.com/ta/127183, which is reachable but blocked from search engines.

Next step

Need this fixed rather than explained?

Tell us the release, what the system runs and what it is doing. We will reply by email with what we would do next.

  • You deal with the people who do the work
  • No obligation, and no upgrade required before we will help
  • We tell you plainly if something is not worth doing

Send an enquiry

We reply by email. If a server is down, say so in the first line. You can also write to help@scoexpert.com.