Technical article 110169

How to enable large file support?

OpenServer 6.0.0UnixWare 7.0.0UnixWare 7.0.1UnixWare 7.1.0UnixWare 7.1.1UnixWare 7.1.3UnixWare 7.1.4Open UNIX 8

Written · Last revised

Problem

By default, the maximum file length on a UW7/OSR6 system is 1 gigabyte but many os utilities have a limitation of 2GB, but the maximum possible file length supported by VXFS filesystems on UW7/OSR6 systems is 1 terabyte.

How do I enable this large file support on a VXFS filesystem?

The maximum supported filesystem size for UnixWare7 and OpenServer6 is 1 terabyte (1TB). This support is enabled by default.

SEE:
http://uw714doc.xinuos.com/en/FS_admin/MFST_FmttStorageDev.html#_Size_Limitations
_on_a_File_Syst
However, please note that it is not advisable to try to create a root
disk with this capacity and the Initial System Load (ISL) install
screen will limit you to less than a 1TB.  Considering how long an
"fsck" would likely take to check such a filesystem then it might be
advisable to split your data up between different filesystems.

Solution

To enable large files support on the root filesystem:

1. Run fsadm to enable the large file support:

# fsadm -F vxfs -o largefiles /

2. Run fsadm to confirm that the support has been enabled:

# fsadm /
This should report "largefiles".
3. Increase ULIMIT to unlimited:
# ulimit unlimited

4. Ensure SFSZLIM and HFSZLIM are set to "infinity" (which is represented by 0x7FFFFFFF) and relink the kernel:

# cd /etc/conf/bin
# ./idtune SFSZLIM 0x7FFFFFFF
# ./idtune HFSZLIM 0x7FFFFFFF
# ./idbuild -B
5. Reboot the system using shutdown(1M).

You should now be able to create files larger than 2GB on root.

To test for a successful creation of a file greater than 2GB:

# dd if=/dev/zero of=big bs=1024k count=2100
# ls -l big
     -rw-r--r--    1 root     sys      2202009600 Oct  4 14:34 big
The 2GB Limit is (1024x1024)x2048 = 2147483648
# dd if=/dev/zero of=file1.dd bs=1024k count=2048
# l file1.dd
-rw-r--r--    1 root     sys      2147483648 Apr 22 06:29 file1.dd

So trying to create a file with a count of 2049 where largefile support is not present would give:

# dd if=/dev/zero of=file1.dd bs=1024k count=2049
dd: write error
2048+0 records in
2047+1 records out
# l
-rw-r--r--   1 root     sys      2147483136 Apr 22 09:47 file1.dd

Creating a file with a count of 2047 would be the largest file size and gives:

# l
-rw-r--r--    1 root     sys      2146435072 Apr 22 06:40 file1.dd

Not all utilities are largefiles aware (and shell redirection does not support greater than 2GB files).

For example, you can't create a file greater than 2GB by using
'cat':
# cat file1 file2 >> file3
if file3 exceeds 2GB.

or 'tar', and here's a little more detail on the limitations:

Here's an example trying to 'tar' an exact 2GB file, created with:

# dd if=/dev/zero of=file1.dd bs=1024k count=2048
# l file1.dd
-rw-r--r--    1 root     sys      2147483648 Apr 22 06:29 file1.dd
# tar cvbf 20 file1.tar file1.dd
  UX:tar: ERROR: Cannot access file1.dd: Value too large for
  defined data type

However, 'tar' will support a file count of 2047 as:

# l
-rw-r--r--    1 root     sys      2146435072 Apr 22 06:40 file1.dd
# tar cvbf 20 file1.tar file1.dd
a file1.dd 4192256 tape blocks
# l
-rw-r--r--    1 root     sys      2146435072 Apr 22 06:40 file1.dd
-rw-r--r--    1 root     sys      2146447360 Apr 22 06:54 file1.tar
Now, here's the 'untar':
# rm file1.dd
# tar xvbf 20 file1.tar
x file1.dd, 2146435072 bytes, 4192256 tape blocks
# l
-rw-r--r--    1 root     sys      2146435072 Apr 22 06:40 file1.dd
-rw-r--r--    1 root     sys      2146447360 Apr 22 06:54 file1.tar

Therefore, it is not recommended to use a filesize of greater than 2146435072 with 'tar'.

However, OpenServer6's and UnixWare7's Skunkware 'bash' is designed to be large file aware.

For a list of commands, utilities and APIs that support largefiles, see the Intro(2) man page on your UW7/OSR6 system.

- To ensure that non-root users have ULIMIT=unlimited, change the ULIMIT parameter in /etc/default/login to unlimited.

For OSR6, this is set to 2GB by default.

Take care not to put a value in here too low otherwise copying files around on the filesystems could result in the error: "cp cannot copy"

Running "ulimit" after running the changes above should return the value:

4194303
- For earlier versions of UnixWare7:
SFNOLIM 1024
HFNOLIM 1024

These are the maximum number of file descriptors or NOFILES equivalent if you were using OpenServer5.

Increasing SFNOLIM to above 1024 may also cause "netcfg" to take
several minutes to load each component.  This is the soft limit
specifying the maximum number of open files the process can have.
Check the default value of these parameters by searching the
file 'mtune' in /etc/conf/cf.d/mtune.   In the case of UW713
onwards, the default value is 2048 and so, therefore, would not
need changing.

It is recommended to put the server into Single User Mode (init 1 or at the boot logo press space and enter "INITSTATE=S" then press ENTER followed by "go").

You can also increase SCORLIM and HCORLIM but you should be aware that if your application core dumps then the size of the core file will equal that of memory.

- It is not recommended to set the file size to unlimited on the root filesystem in order to ensure that the root file system does not run out of disk space unnecessarily.

- Do not restrict the permissions of /etc/default/login as this will prevent other non root users from logging into the GUI desktop, if this is required, as the entry DEFAULTWINDOWMANAGER=cde can not be read by the user and they will need to pick the Window Manager manually in the "Options" section of SCOLOGIN.

- OpenServer5 is limited to a 2GB filesize.

See also

fsadm_vxfs(1M) and Intro(2)

Page 29 of the UnixWare 7 Getting Started Guide.

ta105805, "How can I set up extra filesystems or slices on my boot disk?"

ta105408, "UnixWare 2.0.*, How can I tune my system to allow a process to use more memory?"

ta117654, "What binaries of UnixWare7 are "Large File" aware?"

TA126131 "I see the error "value too large for defined data type"

From the Xinuos technical article database, record 110169. Reproduced with permission. The original is at www.sco.com/ta/110169, which is reachable but blocked from search engines.

Next step

Need this fixed rather than explained?

Tell us the release, what the system runs and what it is doing. We will reply by email with what we would do next.

  • You deal with the people who do the work
  • No obligation, and no upgrade required before we will help
  • We tell you plainly if something is not worth doing

Send an enquiry

We reply by email. If a server is down, say so in the first line. You can also write to help@scoexpert.com.