Problem
My file is too large to transfer over ftp, how can I get around this problem?
Cause
Transferring large files over ftp can fail for any of these reasons:
* Some ftp servers won't store files larger than 1GB. * Network connection poor or unstable. * Long inactive terminal session times out. * Transfer completes, but results in corrupted files.
Solution
Use compression utilities like compress(C), gzip(C), or bzip2(C) to reduce the size of the file being transferred.
You can use the split(C) utility to break the file into smaller pieces and then once transferred to the target machine, use cat(C) to combine the pieces back into a single file.
To keep the transferring terminal session active use ftp's 'hash' option to provide hash marks as transfer status.
To avoid corruption use ftp's 'bin' option so the transfer uses binary mode.
Here is a step-by-step outline showing how to transfer a large file.
1) First reduce the file sizes by using compress and split:
$ compress big_file
$ split -b 500m big_file.Z big_file.Z.partThis will create separate files, each 500MB with the names: big_file.Z.partaa big_file.Z.partab big_file.Z.partac big_file.Z.partad
2) transfer these smaller files through ftp in binary mode with hash marks:
$ ftp -p ftp.destination.com
ftp> bin
ftp> hash
ftp> cd /target
ftp> mput big_file.Z.part*3) On the target machine, use cat to combine the files into one:
$ cat big_file.Z.partaa big_file.Z.partab big_file.Z.partac
big_file.Z.partad > big_file.Z
$ uncompress big_file.ZIf you are having problems retrieving a file through ftp due to network timeouts, try using ftp's 'reget' command along with 'bin' and 'hash':
$ ftp -p ftp.server.com
ftp> bin
ftp> hash
ftp> reget big_file.Z.partaaThe reget command will continue from where the last transfer failed and append to the end of the existing file on the local system so the transfer doesn't need to start over at the beginning.
NOTE: split(C) is limited to files 2GB or less at this time, a
request has been filed with SCO Engineering to have split
made largefile aware. On Openserver6 and UnixWare7
compress is largefile aware, so if you can compress the
file to less than 2GB, you can still split it and use this
method to transfer the file.NOTE: No additional utilities such as BZIP2, GZIP, INFOZIP or ZIP
support file sizes greater than 2Gb. See also
TA110169 "How to enable large file support?" TA126131 "I see the error "value too large for defined data type" TA105852 "What are the filesystem types that can be used in SCO OpenServer 5?"