Problem
How can I configure and administrate ODM under SCO OpenServer 6.0.0 for Raid-5 only?
Also, what to do in case of a failed disk in the ODM Raid-5?
NOTE: Note 1:
Before you start, make sure you have the following:- In our example, we are going to configure a Raid-5 set ONLY, as this is what is needed.
Therefore you need to obtain the FULL SCO OpenServer 6 Online Data Manager License.
As this license will enable your ODM software for Disk Management, Raid Level 0,1,5 & 10).
If you only need to configuring RAID-1 (Mirror), then you need to obtain the SCO OpenServer 6 Disk Mirroring License.
This is explained in a separate TA# 126802.
- SCO OpenServer 6.0.0 MP2 installed.- SCO VERITAS Online Disk Manager (ODM) ver 3.2Aa installed.
- OSS706C Supplement installed.http://www.sco.com/support/update/download/release.php?rid=238This will have some bug fixes for ODM under OSR6.
- Your "scologin X-session" is working.Note 2:You need to be aware of the following info, which will help you to understand how ODM Mirroring works.
ODM Mirroring does not mirror the whole physical disk to the other, like in a hardware mirror. But it only mirrors the filesystems on the disks.
So VxFS filesystems needs to be first configured on a disk via ODM and then mirror these filesystems across to the other disk.
On our example system, we have four disks:
# sdiconfig -l
0:0,7,0: HBA : (mpt,1)LSI Logic 1030 10414
0,0,0: DISK : FUJITSU MAS3184NC 5205
0,1,0: DISK : MAXTOR ATLASU320_36_SCAB434
0,2,0: DISK : MAXTOR ATLASU320_36_SCAB434
0,3,0: DISK : MAXTOR ATLASU320_36_SCAB434
0,6,0: TAPE : SONY SDT-10000 01u1
0,8,0: HBA : SDR GEM318 0
1:0,2,0: HBA : (ide,1) Generic IDE/ATAPI
0,0,0: CDROM : HL-DT-STDVD-ROM GDR8163B0F15Disk1 - 18 GB, FUJITSU MAS3184NC will be used only for the SCO OpenServer6 installation (boot disk).
Disk-2,3&4 - 36 GB MAXTOR ATLASU320_36_SCAB434 will be used as
critical data disk and will be configured to Raid-5 via ODM.It is not recommended to encapsulate the root disk with ODM unless
there is a specific need and has been tested thoroughly. This
generally fails because the SCSI controller in use is not able to
boot from the second disk or there is no vtoc written to the disk
first prior to ODM's installation. Solution
In our example, we are going to leave disk1 alone and not allow ODM to control it.
This disk will host the OSR6 installation (Boot disk).
The other three 36 GB HD, will be used under ODM control as Raid-5
set.Setting UP a RAID-5 set
-----------------------As root, perform the following steps:1-You need first to initialize the three HD's that will be used under ODM by running "mkdev hd":
# mkdev hdMass storage devices are recognized automatically when the system is booted with a new device present.
Current hard disk configuration:Alias - Address - Inquiry String
-------- -------- --------------
hba1 - c0b0t7 - (mpt,1)LSI Logic 1030 10414
disk1 - c0b0t0l0 - FUJITSU MAS3184NC 5205
disk2 - c0b0t1l0 - MAXTOR ATLASU320_36_SCAB434
disk3 - c0b0t2l0 - MAXTOR ATLASU320_36_SCAB434
disk4 - c0b0t3l0 - MAXTOR ATLASU320_36_SCAB434Please enter the alias of the disk you wish to initialize ( type quit to exit ): <disk2>
Please Note: do not type <disk1>, as this is your boot disk.
If your disk was brand new and formatted then you will get this screen:
The recommended default partitioning for your disk is:
a 100% "UNIX System" partition.To select this, please type "y". To partition your disk differently, type "n" and the "fdisk" program will let you select other partitions. y
There are 35907743 blocks available in the UNIX area. Please enter the number of file systems you want this area to be divided into, or press <Return> to get the default of 7 file system(s) <0>
Here type "1"The layout of the filesystems and swap area is now prepared.
Do you wish to make any manual adjustments to the sizes or names of the filesystems or swap area before they are created on the hard disk? (y/n) <y>
you end up with the divvy table with 1 divisions with names d00110 or similar. Do (enter) the following there:
"p" to Prevent a new file system from being created
on this division. (We do not want a filesystem
to be created)."0" For Division the first division."q" to quit and then"i" for Install (Install the division set-up shown).
This will prevent any divisions to be created on the disks. As these disks will be under ODM control, no need to create OpenServer6 divisions on them.
You will then land back in the "mkdev hd" utility menu again.
If your disk was not brand new and had some other data on it (another partition), then you will get additional messages that needs answers.
In our example the third disk had UnixWare7 partition on it:
Alias - Address - Inquiry String
-------- -------- --------------
hba1 - c0b0t7 - (mpt,1)LSI Logic 1030 10414
disk1 - c0b0t0l0 - FUJITSU MAS3184NC 5205
disk2 - c0b0t1l0 - MAXTOR ATLASU320_36_SCAB434
disk3 - c0b0t2l0 - MAXTOR ATLASU320_36_SCAB434
disk4 - c0b0t3l0 - MAXTOR ATLASU320_36_SCAB434Please enter the alias of the disk you wish to initialize ( type quit to exit ): <disk3>
You will then enter the OSR6 partition utility (fdisk).
Choose the following option in the order:
- Option 3, delete them. - Option 1, create a Unix Partition (use 100% for Unix) - Option 4, Exit and update the disk configuration.
Next screenYou have chosen a hard disk partition that was last administered on a UnixWare system. The division layout of this hard disk partition cannot be changed, using SCO OpenServer 6 tools, if you wish to preserve the existing data on this disk partition.
You have two options.If you wish to access data that is already on this disk, you should simply exit this tool and use the Scoadmin Filesystem Manager to update /etc/default/filesys.
If you wish to just create new filesystems on this disk partition, permanently discarding any data already on this disk partition, you should continue using this tool.
Type quit to exit this tool or create to proceed: <create>
Here type "create"WARNING: You are about to permanently destroy the data contained in this disk partition.
Are you sure you wish to proceed?Type quit to exit this tool or destroy to proceed: <destroy>
Here type "destroy"Then you will enter the OSR6 division table information as above for disk2 and follow the same procedure.
Repeat the same procedure for disk4.Then quit the "mkdev hd" utility.You are now ready to start the ODM configuration.
Please Note: If you fail to follow the above procedure in step 1, ODM will fail with the initialization setup.
2-As root, run from the command line:
# vxinstallyou will get the following screen:Volume Manager Installation
Menu: VolumeManager/InstallThe Volume Manager names disks on your system using the controller and disk number of the disk, substituting them into the following pattern:
c<controller>b<bus>t<target>d<disk>Some examples would be:c0b0t0d0 - first controller, bus 0, first target, first disk c1b0t0d0 - second controller, bus 0, first target, first disk c1b0t1d0 - second controller, bus 0, second target, first disk
The Volume Manager has detected the following controllers on your system:
c0:Hit RETURN to continue. <RETURN>since in our example, we only have one disk controller, ODM displayed "c0", first disk controller with Valid HD
Next screen:Volume Manager Installation
Menu: VolumeManager/InstallYou will now be asked if you wish to use Quick Installation or Custom Installation. Custom Installation allows you to select how the Volume Manager will handle the installation of each disk attached to your system.
Quick Installation examines each disk attached to your system and attempts to create volumes to cover all disk partitions that might be used for file systems or for other similar purposes.
If you do not wish to use some disks with the Volume Manager, or if you wish to reinitialize some disks, use the Custom Installation option Otherwise, we suggest that you use the Quick Installation option.
Hit RETURN to continue. <RETURN>Next screen:Volume Manager Installation Options
Menu: VolumeManager/Install1 Quick Installation
2 Custom Installation? Display help about menu
?? Display help about the menuing system
q Exit from menusSelect an operation to perform:We will choose "2 - custom Installation", as we want to control which disks are to be configured under ODM for mirroring.
Note: With custom, we will leave the boot/root disk alone.
Next screen:Volume Manager Custom Installation
Menu: VolumeManager/Install/CustomThe c0b0t0d0 disk is your Boot Disk. You can not add it as a new disk. If you encapsulate it, you will make your root filesystem and other system areas on the Boot Disk into volumes. This is required if you wish to mirror your root filesystem or system swap area.
Encapsulate Boot Disk [y,n,q,?] (default: n) <RETURN>Next screen:Volume Manager Custom Installation Menu: VolumeManager/Install/Custom/c0 Generating list of attached disks on c0....
<excluding root disk c0b0t0d0>The Volume Manager has detected the following disks on controller c0:
c0b0t1d0 c0b0t2d0 c0b0t3d0Hit RETURN to continue. <RETURN>Next screen:Installation options for controller c0
Menu: VolumeManager/Install/Custom/c01 Install all disks as pre-existing disks.
(encapsulate)
2 Install all disks as new disks. (discards data on
disks!)
3 Install one disk at a time.
4 Leave these disks alone.? Display help about menu
?? Display help about the menuing system
q Exit from menusSelect an operation to perform: <2>Choose 2, install all disks as new disks.
Next screen:Volume Manager Custom Installation
Menu: VolumeManager/Install/Custom/c0/InitUse default disk names for these disks? [y,n,q,?] (default: y) <RETURN>
choose the defaultThe c0b0t1d0 disk will be given disk name disk01
Next screen:Perform surface analysis (highly recommended)
[y,n,q,?] (default: y) <n>PLEASE CHOOSE "n" and ignore the recommended, as this will take ages to finish. Not really needed for SCSI HDs.
do the same for disk02 & disk03.
Next screen:Volume Manager Custom Installation
Menu: VolumeManager/Install/CustomThe following is a summary of your choices.
c0b0t1d0 New Disk
c0b0t2d0 New Disk
c0b0t3d0 New DiskIs this correct [y,n,q,?] (default: y) <RETURN>Choose the default.Next screen:The Volume Manager is now reconfiguring (partition
phase)...Volume Manager: Partitioning c0b0t1d0 as a new disk.
Volume Manager: Partitioning c0b0t2d0 as a new disk.
The Volume Manager is now reconfiguring (initialization
phase)...Volume Manager: Adding disk01 (c0b0t1d0) as a new disk.
The Volume Daemon has been enabled for transactions.
Volume Manager: Adding disk02 (c0b0t2d0) as a new disk.
Volume Manager: Adding disk03 (c0b0t3d0) as a new disk.
The system now must be shut down and rebooted in order to continue the reconfiguration.
Shutdown and reboot now [y,n,q,?] (default: n) <y>Choose "y" for the server to reboot.
3-After reboot to check that "xvinstall" was successful, run the following command as root:
# vxdiskadmType "list" or "l" to choose the "List disk information" then RETURN to choose the default "all"
you will see this output:DEVICE DISK GROUP STATUS
c0b0t0d0 - - error
c0b0t1d0 disk01 rootdg online
c0b0t2d0 disk02 rootdg onlineDo not get worried about the "error" status of the Device c0b0t0d0, as this is our root boot disk and we did not want ODM to have control of it. this is why ODM if flagging it as status "error", because it can't use it.
The other devices, c0b0t1d0 (disk01), c0b0t2d0 (disk02) and c0b0t3d0 (disk03) are flagged as status "online", which is good.
OR run the "vxprint" from the command line:
# vxprintyou will see this output:Disk group: rootdgTY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
dg rootdg rootdg - - - - - -dm disk01 c0b0t1d0s7 - 71793462 - - - -
dm disk02 c0b0t2d0s7 - 71793462 - - - -
dm disk03 c0b0t3d0s7 - 71793462 - - - -In both of the above outputs indicates that "vxinstall" was successful and our three ODM disks are ready be used for RAID-5.
Now to define a RAID-5 set via ODM.
4-The best and comfortable way to define a RAID-5 set on ODM and associate filesystems on it is via the ODM graphical interface.
This is why it is recommended that you have X-windows working on OSR6 as noted in the "before you start" section.
You can use the ODM command line Interface CLI, but the syntax is long and complicated.
Login as root in the X-scologin screen.
To start ODM GUI Manager, click on the "System Administration" icon, and then the "VERITAS Volume Manager" ico. Or From the xterm, run:
# vxva &This will open "VERITAS Visual Administrator" window.
To define a simple mirror, please follow the steps:
- Click on the <rootdg> button. this will open a "view of rootdg (<server name>)" window. There you will see three cylinder like icons of disk01, disk02 & disk03.
- From the above menu¿s, choose: Options --> Popup the Command window This will open a "command Info window" so that we can see what command are run and their progress. This is good, because if any error happens, we will know at which stage and command it failed....etc
- On the "View of rootdg" window, hold down the <Ctrl> key and click with the left mouse button on disk01, disk02 & disk03 icons. All three ODM disks will then be highlighted in green colour.
- In our example, We now will create a RAID-5 set:
Basic-Ops --> Volume Operations --> Create --> RAID-5This will open a "RAID-5 Volume/FS Create Form" window. update the following:
Volume name = u1 Volume size = <use the max> Create File system = Yes Impose inode limit = No Mount Point = /u1
Then click on the "Apply" button. You will then see on the "command History" & "Commands Running" window, the commands lines that ODM is running runs in order to generate that RAID-5 set. Check that all command are successful and have finished with "DONE"
Note: This operation will take some time, depending on your Harddrive sizes.
Once it has finished, you can see from the VERITAS GUI if the RAID-5 setup was success or not.
You can also check it from the command line:
# vxprintDisk group: rootdgTY NAME ASSOC KSTATE LENGTH PLOFFS STATE
dg rootdg rootdg - - - -dm disk01 c0b0t1d0s7 - 71793462 - -
dm disk02 c0b0t2d0s7 - 71793462 - -
dm disk03 c0b0t3d0s7 - 71793462 - -v u1 raid5 ENABLED 71792640 - ACTIVE
pl u1-01 u1 ENABLED 71792640 - ACTIVE
sd disk01-03 u1-01 ENABLED 71792640 0 -
sd disk02-03 u1-01 ENABLED 71792640 0 -
pl u1-02 u1 ENABLED 640 - ACTIVE
sd disk03-03 u1-02 ENABLED 640 0 -The "Active" is what we are looking for to indicate the success of the RAID-5 and its status.
For more options on the vxprint command, see man pages vxprint(ADM).
Recovering from a broken Faulty disk
------------------------------------In the event of a faulty disk, ODM will mark it as faulty and it gets detached.
Lets assume that Disk02 (ODM Naming) got faulty, you will see this:
# vxprint -dDisk group: rootdgTY NAME ASSOC KSTATE LENGTH PLOFFS STATE
dg rootdg rootdg - - - -dm disk01 c0b0t1d0s7 - 71793462 - -
dm disk02 - - - - NODEVICE
dm disk01 c0b0t3d0s7 - 71793462 - -
....etcAll reference to disk02, you see "NODEVICE".
To replace the faulty disk02 you need to carry out the following steps:
1- Run as root the "vxdiskadm" command line:
# vxdiskadmFrom the menu, choose "4 - Remove a disk for replacement"
Remove a disk for replacement
Menu: VolumeManager/Disk/RemoveForReplaceUse this menu operation to remove a physical disk from a disk group, while retaining the disk name. This changes the state for the disk name to a "removed" disk. If there are any initialized disks that are not part of a disk group, you will be given the option of using one of these disks as a replacement.
Enter disk name [<disk>,list,q,?] <disk02>Enter "disk02" as the name. If you do not know the name,
enter "l" for list of disks in the "rootdg" and the list
below will be shown:Disk group: rootdgDM NAME DEVICE TYPE PRIVLEN PUBLEN STATEdm disk01 c0b0t1d0s7 sliced 1024 71793462 -
dm disk02 - - - - NODEVICE
dm disk03 c0b0t3d0s7 sliced 1024 71793462 -Enter disk name [<disk>,list,q,?] <disk02>Type "disk02".The following volumes will lose mirrors as a result of this operation:
u1No data on these volumes will be lost.
The requested operation is to remove disk disk02 from disk group rootdg. The disk name will be kept, along with any volumes using the disk, allowing replacement of the disk.
Select "Replace a failed or removed disk" from the main menu when you wish to replace the disk.
Continue with operation? [y,n,q,?] (default: y) <RETURN>Hit RETURN to choose the default "y".
Remove another disk? [y,n,q,?] (default: n) <RETURN>Hit RETURN to choose the default "n", then q to quit "vxdiskadm".
"vxprint" will now show this output:# vxprint -dDisk group: rootdgTY NAME ASSOC KSTATE LENGTH PLOFFS STATE
dg rootdg rootdg - - - -dm disk01 c0b0t1d0s7 - 71793462 - -
dm disk02 - - - - REMOVED
dm disk03 c0b0t3d0s7 - 71793462 - -
....etcShutdown the server and power off.2- Remove the faulty disk and replace it with a new one.
Note: IMPORTANT!!! The new disk MUST be the exact same size (or more) as the active disks. If the replaced disk is even one cylinder less then the active disks, then the ODM can't repair the RAID-5. Therefore it is highly recommended to have the replaced disk from the same size and make of the active disk.
3- Boot the Server.As root, you need to initialize the replaced disk under SCO OpenServer 6.0.0. Follow the same procedure in the Section "Setting UP a Mirror" - Step 1.
4- Run the following commands as root:
# /usr/sbin/vxdctl stop
# /usr/sbin/vxconfigd -k -m enable
# /usr/sbin/vxdctl enableThe above commands will restart ODM so that it will be aware of the new replaced disk.
5- Run the "vxdiskadm" from the command line.
# vxdiskadmFrom the menu, choose "5 - Replace a failed or removed disk"
Replace a failed or removed disk
Menu: VolumeManager/Disk/ReplaceDiskUse this menu operation to specify a replacement disk for a disk that you removed with the "Remove a disk for replacement" menu operation, or that failed during use. You will be prompted for a disk name to replace and a disk device to use as a replacement. You can choose an uninitialized disk, in which case the disk will be initialized, or you can choose a disk that you have already initialized using the Add or initialize a disk menu operation.
Select a removed or failed disk [<disk>,list,q,?]
<disk02>Enter "disk02" (Our replaced disk).Select disk device to initialize [<address>,list,q,?]
<c0b0t2d0>Enter "c0b0t2d0", which is our Device for the replaced disk. With "l" you can see all the list of Devices ODM has knowledge of.
If the new replaced disk had before a valid VTOC, then ODM will allow you to preserve the data via encapsulation.
Then you get the below message. We do not want to encapsulate but initialize.
If the disk was brand new, then you are asked to initialize the disk:
The following disk device has a valid VTOC, but does not appear to have been initialized for the Volume Manager. If there is data on the disk that should NOT be destroyed you should encapsulate the existing disk partitions as volumes instead of adding the disk as a new disk. Output format: [Device_Name]
c0b0t2d0Encapsulate this device? [y,n,q,?] (default: y) <n>Enter "n", for not encapsulating.Instead of encapsulating, initialize? [y,n,q,?] (default: n)
<y>Enter "y" for initialization.The requested operation is to initialize disk device c0b0t2d0 and to then use that device to replace the removed or failed disk disk02 in disk group rootdg.
Continue with operation? [y,n,q,?] (default: y) <RETURN>Hit RETURN for the default "y"Perform surface analysis (highly recommended)
[y,n,q,?] (default: y) <n>PLEASE CHOOSE "n" and ignore the recommended, as this will take ages to finish. Not really needed for SCSI HDs.
Replace another disk? [y,n,q,?] (default: n) <RETURN>Hit RETURN for default "n".Still in the "vxdiskadm" menu, you can see if the above operation was successful or not, by choosing "l" for "List disk information" and then default "all".
You will see this output, where disk02 now is reporting online for ODM:
DEVICE DISK GROUP STATUS
c0b0t0d0 - - error
c0b0t1d0 disk01 rootdg online
c0b0t2d0 disk02 rootdg online
c0b0t3d0 disk03 rootdg online6- ODM is now re-syncing the RAID-5 set. To check, run "vxprint":
# vxprintThe syncing will take a while depending on the filesystem sizes.
NOTES: As a reminder, these point are important to remember when
configuring Mirroring under ODM and recovery: See also
http://osr600doc.xinuos.com/en/ODM_oi/_Online_Data_Manager_oi.html