Technical article 110488

My filesystem is corrupt even after running fsck. What can I do before restoring from backup or consulting a data recovery service?

OpenServer 6.0.0OpenServer 5.0.0OpenServer 5.0.2OpenServer 5.0.4OpenServer 5.0.5OpenServer 5.0.6OpenServer 5.0.7UnixWare 7.0.0UnixWare 7.0.1UnixWare 7.1.0UnixWare 7.1.1UnixWare 7.1.2UnixWare 7.1.3UnixWare 7.1.4Open UNIX 8

Written · Last revised

Problem

Even after running fsck, perhaps several times, and with a variety of options, such as -ofull for HTFS (OpenServer) or VXFS (UnixWare) filesystems, or -s for EAFS filesystems, I still cannot mount and access data in a corrupted filesystem. Typically, the filesystem check utility, fsck, keeps finding errors it cannot repair, or it can even quit or hang while checking the filesystem.

I don't want to lose the most recent modifications to the filesystem by restoring the contents from a backup, nor do I want to incur the expense of a data recovery consultancy. Is there anything left that I can do?

Cause

Some filesystem corruption may not be repairable by fsck:

This has been seen with errors such as "fsck-free invalid buffer".

fsck -ofull -y <device node>
or for OSR5/6 use Debug mode:
fsck -s -y -t recover <device node>
fsck -ofull -d <device node>

For UnixWare7/OpenServer6, you can also attempt an fsck using the -onolog option as:

         fsck -F vxfs -ofull -onolog -y <device node>
See:
http://uw714doc.xinuos.com/en/man/html.1M/fsck_vxfs.1M.html
http://osr600doc.xinuos.com/en/FS_admin/_Checking_vxfs_File_Systems.html

For OpenServer5, you can also attempt a mount using the -nolog option as:

         mount -v -o nolog /dev/u /u
See:
http://osr507doc.xinuos.com/en/man/html.ADM/mount.ADM.html
http://osr507doc.xinuos.com/en/man/html.ADM/fsck.ADM.html

However, it is possible that the type of corruption that persists is immaterial as long as you do not perform any further _writing_ to the filesystem.

It is therefore reasonable to attempt to read and recover the data in the filesystem by mounting the filesystem read-only. If the mount utility, in checking the status of the filesystem to be mounted, determines that the existing corruption is passable in read-only mode, the mount will succeed and data can be explicitly examined and saved.

Solution

Try to mount the filesystem read-only with the -r option. For example, for a non-root VXFS filesystem /dev/u on an SCO UnixWare system, and a mount-point directory of /u:

mount -r -F vxfs /dev/u

And for the root HTFS filesystem on an SCO OpenServer 5 system, while booted from a set of emergency boot/root floppies:

mount -r /dev/hd0root /mnt
If the mount succeeds, then chances are good that the data in
the filesystem can be read and recovered.  After recovering
the desired data, it is strongly recommended that you re-create
the filesystem and restore the contents from the latest backup.

Before re-creating the filesystem it is recommended to:

a) Ensure you do have a valid backup from which to restore

b) Are the users able to work on a seperate filesystem or server while you effect repairs?

c) Make a note of the filesystem's setting prior to rebuiding it:

For OpenServer5/6: Use 'divvy' to note the filesystem details and use 'divvy' to re-create individual filesystems or "mkdev hd" to re-create individual disks.

For UnixWare7: Use "fsadm <filesystem>" and "mkfs -m <device>" to determine the features of the filesystem (can also be used on OpenServer6) and either "mkfs" to re-create individual filesystems ( see TA 110908 or below) or "diskrm <device>" and "diskadd <device>" to re-create individual disks.

NOTE:     To re-add a filesystem in UnixWare7:
For example:
/u1 and /u2 sit on /dev/dsk/c0b0t1d0:
/u1                 : Disk space:   467.63 MB of   500.00 MB available (93.53%)
/u2                 : Disk space:   488.25 MB of   522.00 MB available (93.54%)
and in /etc/vfstab:
/dev/dsk/c0b0t1d0s1     /dev/rdsk/c0b0t1d0s1    /u1     vxfs    1       yes     -       SYS_RANGE_MAX
/dev/dsk/c0b0t1d0s2     /dev/rdsk/c0b0t1d0s2    /u2     vxfs    1       yes     -       SYS_RANGE_MAX
We can remake /u2 with:
# umount /u2
# mkfs -m /dev/dsk/c0b0t1d0s2 > /local/create_u2.sh
          # cat /local/create_u2.sh
mkfs -F vxfs -o ninode=unlimited,bsize=1024,version=4,inosize=256,logsize=1024,nolargefiles /dev/dsk/c0b0t1d0s2 1069056
          # ./sh /local/create_u2.sh
Mkfs: make vxfs file system?
(DEL if wrong)

version 4 layout 1069056 sectors, 534528 blocks of size 1024, log size 1024 blocks unlimited inodes, largefiles supported 534528 data blocks, 533312 free data blocks 17 allocation units of 32768 blocks, 32768 data blocks last allocation unit has 10240 data blocks

Enter RETURN when prompted or DEL to abort.

Now, run:
# mountall

and you will see /u2 re-created when you run:

# dfspace
/u1                 : Disk space:   467.63 MB of   500.00 MB available (93.53%)
/u2                 : Disk space:   488.25 MB of   522.00 MB available (93.54%)
NOTE:     For a full listing of file system errors visit:
For OpenUnix8:
-------------
http://ou800doc.sco.com/ODM_FSadmin/fssag-11.html#HEADING11-29
For UnixWare7:
-------------
http://uw714doc.xinuos.com/en/ODM_FSadmin/fssag-11.html
For OpenServer6:
---------------
http://osr600doc.xinuos.com/en/ODM_FSadmin/fssag-11.html
For OpenServer5:
---------------
http://osr507doc.xinuos.com/en/man/html.M/messages.M.html
NOTE:     To check the inodes on a server, use "ncheck", eg:
# ncheck /dev/rroot
See /etc/default/filesys or /etc/vfstab
NOTE:     Additional Kernel Messages can be found at:
http://uw714doc.xinuos.com/en/SM_trouble/saT.check_errmsg.html
http://uw714doc.xinuos.com/en/ODM_FSadmin/fssag-11.html
http://uw714doc.xinuos.com/en/ODM_VMadmin/sag-8.html
http://uw714doc.xinuos.com/en/SM_perform/_Kernel_Messages_about_Exceeding.html
http://uw714doc.xinuos.com/en/HAUserG/appa.htm

See also

ta105411, "Filesystem Repair Utilities for SCO OpenServer 5.0.0, 5.0.2, and 5.0.4."

ta105410, "Filesystem Repair Utilities for SCO UNIX Release 3.2 Version 4.2."

ta107202, "How to recover from a system crash under SCO UNIX System
V/386."

ta109259, "What crash recovery features are provided with UnixWare 7?"

ta104411, "How to recover root filesystem after crash if backups, boot/root exist. 4.2."

ta105315, "UnixWare 2.0.*, How to recover from a system crash if the root filesystem is unmountable."

ta104781, "fsck(ADM) is reporting read errors on block 0 of my root device."

ta104767, "Recover OpenServer 5 root filesystem after crash if backups, boot/root exist."

ta103182, "How to contact a data recovery service provider."

ta105316 "UnixWare 2.1.x, "How to clean root filesystem after crash if system won't boot and emergency boot or backups don't work."

From the Xinuos technical article database, record 110488. Reproduced with permission. The original is at www.sco.com/ta/110488, which is reachable but blocked from search engines.

Next step

Need this fixed rather than explained?

Tell us the release, what the system runs and what it is doing. We will reply by email with what we would do next.

  • You deal with the people who do the work
  • No obligation, and no upgrade required before we will help
  • We tell you plainly if something is not worth doing

Send an enquiry

We reply by email. If a server is down, say so in the first line. You can also write to help@scoexpert.com.