5. Unpacking and Compiling
5.1 File locations
5.2 Modifying defaults.h
6. Terminology
7. Tape Devices
8. Tape drives
8.1 ftape
8.2 zftape
8.3 SCSI Drives
8.4 IDE tape drives
8.5 Floppies and other removable media
9. Quick fast forwards
10. Can seek
11. Formatting and Erasing Tapes
12. /proc Filesystem
13. Compression
14. Running Taper
14.1 Command line options
14.2 Backup Module
14.2.1 Backup modes
14.3 Restore
14.3.1 Restore modes
14.3.1.1 Full restore
14.3.1.2 Most recent volume restore
14.3.1.3 Fixed volume restore
14.4 Recreate info file
14.5 Verify archive
14.6 Utilities
14.6.1 Make tape
14.6.2 Test make tape
14.6.3 Which proc
14.6.4 Test fast fsf
14.6.5 Test can seek
14.6.6 Test end of tape
14.6.7 Erase volumes
14.6.8 Reindex info file
14.6.9 Look for recursive links
14.6.10 Attempt Recovery
15. Unattended Backup
16. File Sets
19. Screen Colours
20. Multiple Tape Backups
20.1 Writing
20.2 Reading
21. Cross platform support
22. Limitations
23. Bugs
24. Major bugs in early versions
25. Finally
______________________________________________________________________
1. Disclaimer
Copyright (C) 1996-2000 by Yusuf Nagree (yusuf@e-survey.net.au)
This program is free software; you can redistribute it and/or
modify
it under the terms of the GNU General Public License as pub
lished by
the Free Software Foundation;
This program is distributed in the hope that it will be useful,
but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU
General Public License for more details.
You should have received a copy of the GNU General Public Li
cense
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
2. Warning
Please note that this is BETA software. This means that it is
still
under development. This program works fine on my setup (DX2-66,
Jumbo
250, Floppy controller, 32Mb RAM, 3 IDE hard drives (120mb,
120mb,
540mb), Suprafax modem, Mitsumi Quad Speed IDE CD-ROM, serial
mouse).
Until then, please also backup using a method you KNOW works
for you.
Having said this, from the comments I have received so far, it
does
seem to work on a fair few systems so I hope it does work on
yours.
If it doesn't, I'd like to know.
3. Introduction
Taper is a user friendly archive program especially designed
for
backing up to tape drives. It also supports backing up to files
on a
hard disk
It, I hope, overcomes the lack of programs available currently
for
Linux for user friendly archiving.
Tar/cpio, apart from having an unfriendly user interface, lack
one
essential feature that I needed - there was no information
stored in
the beginning of the archive about what files were on the
archive. The
only way to get such a list was to traverse the whole archive,
getting
cpio/tar to print out a list of files that it found. With a
250mB
archive, this could easily take 5 min. It was a real hassle to
maintain lists of files on all my tapes and it could take 20
min to
locate a file if I didn't know which tape it was on - printing
out
file lists of each tape.
I solved this by maintaining an archive information file on the
hard
disk. This file contains all the information about files
stored on
the archive. When files are added to the archive, the archive
information file is updated. This archive information file can
be
reconstructed from the archive should the file get
deleted/corrupted/lost - it just takes time.
To speed access to the files, the info file cluster consists of
four
files - two main files and two indexes. The names of the main
files
are taper_x and taper_x.0, while the two indexes are taper_x.1
and
taper_x.2. The files are placed in the default info file direc
tory.
file on your target machine. If you do not want to reindex,
then
take all four files on the floppy with you.
o reconstruct the archive information file on the restoring
machine.
Since reconstructing the archive information file can take a
while,
the first method is recommended. Info files can be quite long
and
there are three of them. A short housekeeping file, a main file
and an
index file. When transporting info files, on the short file and
the
main file need be transported - the index file can be rein
dexed.
Although these info files can be long, they compress very well
-
typically more than 10:1 so it is important that you have the
compress
info files preference ON.
Note, that as a security measure, archives created by one user
cannot
be read by another user, unless the archive information file is
copied
from the backup user's directory to the restore user's directo
ry.
Thus, if you want to make a backup and don't want others to be
able to
restore it, set your permissions on the archive information
file as
rw------- (which taper does by default). Note that this is not
completely foolproof, since a restorer can run recreate info
files and
create the information file and then access the archive. It is
more a
deterrent rather than secure system.
The second problem with both tar and cpio was that using the
floppy
interface tape drive, you could not append files. The only way
to
append files was using a combination of tar and mt - a very
clumsy and
time consuming method of backing up.
Note that the archive device doesn't have to be a tape drive -
it can
be an ordinary file on a filesystem, or a floppy, or in fact,
any file
that the Linux can write to.
Each archive is given a unique archive ID (which is actually
the
other setups. If you have any problems, contact me an I'll see
if I
can help.
The main program is called taper.
Note that currently taper is not really designed for backing up
more
than about 30,000 files (unless you have a large amount of
physical
memory).
4. Requirements
o ncurses library >= v1.9.6 with the forms library
o kernel >= v1.2.5 (I think it will work for lesser versions
but I
have not tested it) - Your kernel must have SYS_V_IPC sup
port
enabled if you wish to take advantage of the triple buffer
ing
system.
o gcc >= 2.6.1
5. Unpacking and Compiling
Unpack the source file distribution by issuing the following
command:
tar xzf 6.9b.tar.gz
Modify Makefile.common as required and then
make clean
make all
TRIPLE_BUFFER
Enable this option if you wish to take advantage of
taper's new
however, on faster drives, can slow down performance. The
best
thing to do is to compile with triple buffering and look
at your
performance. If it is slow, then recompile with triple
buffering
off.
I consider a fast drive to be one that delivers a
throughput of
more than 8mB/minute.
TAPER_BIG_ENDIAN
Taper stores all data to the tape in little endian for
mat. To
use taper on a big endian machine, define TAPER_BIG_ENDI
AN.
FIFO_PROBLEMS
If you are using libc5, and are getting strange errors
about
"Broken Pipes", segmentation faults etc.., uncomment this
option
GLIBC
If you are using glibc (aka libc6), this must be set to
TRUE
5.1. File locations
To place the binaries and the man page in the appropriate loca
tions,
make install
The binaries are placed in /sbin and the man page is
/usr/man/cat1
which is in accordance with the Linux FSSTND.
To remove the files
make uninstall
5.2. Modifying defaults.h
6. Terminology
An archive refers to all the files on a tape. A volume is the
files
that are written to in one backup session. Thus, if you make
one
backup to a tape, then it will have one volume. If you then ap
pend
files to this tape, it will have two volumes. If you append
again, it
will have three volumes.
For all intents & purposes, volumes are transparent to the us
er. When
selecting files to restore, all the files on all the volumes
are
presented in alphabetical order (not volume order).
7. Tape Devices
This program requires a device driver for the tape drive. For
users of
the floppy controller type (eg. Jumbo), there are now two
drivers
available.
The original one was ftape and was written by Bas Laarhoven
and, at
the time of writing this, is currently at version 2.11a. It is
available from sunsite.unc.edu. It should be in
/pub/Linux/kernel/tapes. At the time of writing, it was still
in
/pub/Linux/Incoming.
A new driver, zftape, was written by Claus-Justus Heine, and
is found
at ftp://sunsite.unc.edu/pub/Linux/kernel/tapes/zf
tape-1.06.tar.gz.
It uses the basic ftape code, however, it contains many en
hancements
over ftape including support for compression and a true seek
able
device. The latest version at time of writing was 1.06a. You
must
use version 1.02 or later.
However, Claus-Justus Heine has taken over maintaining both
ftape and
zftape. His latest project is to integrate the code of ftape
and
zftape into one source, which is has called ftape and have ver
sion
numbers >= 3. That is, ftape-3.xx and above is actually a com
bination
of the old zftape and ftape. The moral of the story is... if
aachen.de/~LBFM/claus/ftape where the latest ftape and zftape
can be
obtained.
Two devices are needed - a rewinding device - when using this
device,
the tape drive is automatically rewound when closed, and a non-
rewinding device - when using this device, the tape drive is
not
rewound when closed but left where is. The names of these de
vices
depend on whether you are using a SCSI drive, ftape or zftape.
Alternatively, you can change the names in the global prefer
ences
menu, via the preference file, from the environment, or via the
command line.
It is important that both a rewinding and a non-rewinding de
vice is
given.
When making an archive as a file on a hard disk, the filename
should
be given for both the rewinding and non-rewinding device. To
make
things easier, you can use the -b command line option or the
BOTH
preference.
8. Tape drives
8.1. ftape
ftape is the original floppy tape driver for Linux and now sup
ports a
wide variety of floppy tape drives and a few enhanced con
trollers. In
the later development kernels, ftape is part of the kernel
source
tree. It is still available separately for those running older
kernels. I am not referring to ftape version 3 or above in this
section. If you are using ftape version >= 3, for the purposes
of this
documentation, you should assume that you are using zftape.
Start taper with the -T ftape option.
The device names are /dev/ftape (which should point to
/dev/rft0) for
the rewinding device, and /dev/nftape (which should point to
/dev/nrft0) for the non-rewinding device. These are the values
that
should drastically improve taper performance. If you have mul
tiple
volume backups made under ftape, however, you should not up
grade
because you will not be able to access volumes other than 1 us
ing
zftape.
8.2. zftape
This section also applies to those people using ftape version
>= 3.
Start taper with the -T zftape option.
This driver supports several modes - one is the generic ftape
mode.
The second is a QIC compliant mode (zftape) and the third is a
compressed QIC compliant mode (czftape). It is possible to ac
cess
archives made with ftape using the generic mode (see below),
however,
if you make a backup using the QIC compliant mode (zftape),
then you
cannot use this if you decide to go back to ftape. As this is
rarely
done, and because the QIC compliant mode is so superior to the
generic
mode, I would recommend that you use it.
Backups made under ftape will work with zftape if you:
1. Make sure that you use the device names /dev/ftape and
/dev/nftape.
You will have to change this manually under Global prefer
ences
2. You only restore files from the first volume. It is not pos
sible to
restore files from other volumes. This is because zftape
does not
support the fsf ioctl for ftape drives. I am currently
corresponding with Claus to see if we can resolve this prob
lem.
3. Do not try and append data to an archive created with ftape
using
zftape
4. You change the set-blksize & get-blksize preferences to yes
and
change the block-size to 10K.
If you have multiple volume backups, then you had best use
fast forward through archives. You should notice quite a dif
ference,
especially when doing a mkinfo and when restoring from multiple
volumes. Although zftape has support for compression, you are
better
off letting taper compress since taper buffers a bit better and
has
the option of using gzip which is more efficient than the algo
rithm
used by zftape.
The devices for this driver should be created using
make mknod
in the Makefile that comes with the driver. Note that the per
missions
will be set up so that only root has write access to the tape
drive.
You will have to change this manually if you want to allow oth
er users
access.
The device names are /dev/qft0 for the rewinding device and
/dev/nqf0
for the non-rewinding device.
If you make a backup with ftape version >= 3, you cannot re
store with
zftape.
8.3. SCSI Drives
Start taper with the -T scsi option.
Unfortunately, I do not have a SCSI drive and therefore am un
able to
test taper code with SCSIs. I rely on SCSI users sending me
information about how taper works for them and then I try and
use that
information to help other SCSI users who are having problems.
The
following information was sent to me by AP Harris
(apharris@onShore.com) and I've included it to help SCSI users.
The SCSI rewinding device is /dev/st0 (or /dev/st1 on the 2nd
tape
drive) and the non-rewinding device is /dev/nst0. You'll prob
ably
have to prepare all your tapes using mktape from the utilies
menu.
If you are using a SCSI tape device, make sure you've complied
the
No additional drivers are needed (although the st-aware mt is
nice).
The block size (-x option below) should be set below the tape
device
driver buffer size (which can be changed at kernel compile time
in the
kernel and defaults at 32K).
SCSI tapes generally do not need formatting - see below for
full
details.
8.4. IDE tape drives
Start taper with the -T ide option.
The default device names are /dev/ht0 and /dev/nht0. Support
for IDE
drives is in alpha stage and I would encourage users to give me
feedback on how taper is working with their IDE drives
8.5. Floppies and other removable media
Start taper with the -T removable option.
When the floppy (or whatever) is full, taper will prompt you
for the
next floppy/tape. This works in exactly the same way as multi
ple tape
backups. The default device names are set to /dev/fd0 - ie. the
first
floppy drive.
If you are using a ZIP drive, then make sure that the whole
disk has
been partitioned as a Linux native partition then start taper:
taper -T removable -b /dev/sda1
Substitute /dev/sda1 with the device name of your ZIP drive.
Note:
Your ZIP drive must have one partition only since taper ignores
the
partition table.
Drives
based on ftape are NOT capable of doing this, however, zftape,
is able
to do quick fast forwards, as are most SCSI drives.
With ftape, therefore, the only way to find the next volume is
to
rewind the tape to the beginning, and then forward to the de
sired
volume. As you can imagine, this constant rewinding and for
warding can
take quite a while.
By default, taper assumes that zftape, SCSIs and IDE tape
drives can
do a quick fast forward, while removable and ftape tape drives
cannot.
To test whether your drive can do quick fast forwards, go into
the
utilies menu and run the Test fast fsf program which will tell
you if
your drive supports quick fast forwards or not. Note that taper
will
overwrite all the data on the tape used for the test.
If you find that your tape drive doesn't support quick fast
forwards,
you will have to change the appropriate preference.
10. Can seek
Some tape drives support a seek ioctl which allows positioning
of the
tape drive at a particular block. This makes restore a lot
quicker,
since taper can calculate which block a file(s) lies on an ad
vance
straight to that block.
By default, taper assumes that zftape, and SCSI drives support
this
ioctl while removable,ftape and IDE don't.
You can test if your tape drive supports seek by running the
Test can
seek utility and changing the appropriate preference.
11. Formatting and Erasing Tapes
there is
no program available to low level format a tape, and thus, you
will
have to rely on DOS/WIN/OS-2 programs to do so. If you are
planning on
using taper, then you do not need to worry about step 2, since
taper
will take care of that for you. If you are not going to be us
ing taper
on that particular tape, then you will need to do a mt erase.
Note
that if you buy pre-formatted tapes (which I'd STRONGLY recom
mend
since formatting can take 2-3 hours), then you by-pass step 1
ONLY -
you still have to prepare the tape for Linux.
People with SCSI drives (esp DAT) may not need to format tapes
for use
under Linux. See your tape drive documentation for details.
Similarly, you may not need to do step 2. Those drives that do
not
require step 2 may still require some information to be written
on the
tape before they will work correctly under taper, therefore,
run
mktape from the utilties menu before running tape. You can use
the
test make tape option from the utilities menu to see if your
tape
drive needs to have mktape run on new tapes. It is important
that you
put a brand new tape in the tape drive when testing.
STOP PRESS: The new ftape drivers (versions greater than or
equal to
3.03) do not require tapes to be erased, however, the older
ones do.
Therefore, if you use the new ftape driver, you can set this
preference OFF.
In addition, the new ftape driver can format tapes but it still
takes
time - save yourself a headache - buy preformatted tapes.
--erase-tape-off to tell taper that your tape doesn't need
erasing.
This is the default behaviour if you have started taper with
--tape-
type scsi.
12. /proc Filesystem
Under Linux, it is possible to have a /proc filesystem. This
taper has an option of trying to detect the /proc system and
ignoring
it when doing the backup. Normally, the /proc system is on de
vice
number 1. If your /proc system is on another device, then you
can tell
taper this with the -j (--proc-device) option, or preference.
If you are not sure what device your /proc is mounted on, then
run the
program which_device which is in the utilities menu. Give the
name
/proc (or whatever name your /proc is) and taper will tell you
the
device your /proc is mounted on.
Before using taper, it is a good idea to confirm that your
/proc is on
device 1 and if it isn't change the appropriate preference.
13. Compression
There are several compression methods available.
External compression - type 1
This method calls an external compression program to do
the
compression. The default is set to gzip but can be
changed in
the defaults.h file at compile time. The compression pro
gram is
expected to read the file from standard input and write
the
compressed output to standard output. If your compression
program doesn't do this, you will have to modify the
taper
sources.
Internal compression - type 2
This is a very good compression method - it is very fast
and
reasonably good at compression. Because of this, it is
the
default compression method. The only problem is that is
uses an
extra 2MB of memory which can degrade system performance
on
heavily loaded machines.
Internal gzip - type 3
This method is basically the gzip compression method,
however,
produce compressed files that can be read by gzip. You
need
taper to read the archives.
In summary then:
o Memory use, type 2 > type 3 > type 1.
o Speed, type 2 > type 3 > type 1.
o Compression ratio, type 3 > type 1 > type 2.
You should use compression 2 unless you have a very heavily
loaded
system or less than 4MB of RAM.
If taper is still too memory hungry for you, try the following:
o Turn off triple buffering
o In defaults.h, reduce the size of DEFAULT_TR_SIZE
o Don't use use compress-type set at 2 because this is a very
memory
hungry compression (but also the quickest of the lot).
Compress-
type set at 3 is a good alternative.
o In defaults.h, change the line
#define COMPRESS2_BUFFER_SIZE 1000000
to:
#define COMPRESS2_BUFFER_SIZE 1
You will not be able to use compress-type 2, but if memory is a
prob-
lem, you shouldn't be using this anyway.
o In defaults.h, add the following line:
#define MAXNAMLEN 125
move
between options, press the up or down arrow (or left & right).
To
select and option, press the ENTER key.
taper accepts the following command line options. Note that
there are
two formats for options - short (one letter) and long (GNU
style).
Either one (or a mixture of both) can be used. Alternatively,
options
can be stored in a preference file (see below for details).
14.1. Command line options
--tape-type (-T)
which tape drive you are using. There are three valid op
tions
(s)csi
scsi drive
(f)tape
ftape based tape drive
(z)ftape
zftape based tape drive (or ftape version >= 3)
(i)de
ide based tape drive
fi(l)e
a regular file on the hard disk
(r)emovable
a device that needs mounting, eg. floppy disk
This option must be specified. NB: Only the bracketed
letter of
the option is required.
--help (-?)
prints help on command line options
--append-on (+a)
Append files to archive if it exists. Default.
--append-off (-a)
Overwrites existing archive if found.
NOTE: The append options are only for unattended backups.
If you
use the main taper program, you will be prompted for
--set-ownership-on (+B)
After restoring a file, restore the ownership attributes
of the
file
--set-ownership-off (-B)
Don't attempt to set the ownership attributes of a file
after
restore
--tape-name file (-f) file
use file as the rewinding archive file. If not specified,
will
use the file specified in the environment variable TAPE.
You can
use a networked format like tar (eg. myhost.com:/dev/tape
to use
the tape drive on myhost.com).
--ntape-name file (-n) file
use file as the non-rewinding archive file. If not speci
fied,
will use the file specified in the environment variable
NTAPE.
You can use a networked format like tar (eg.
myhost.com:/dev/tape to use the tape drive on my
host.com).
--both-devices file (-b) file
equivalent to -n file -f file useful when not using tape
drives,
but archive files. You can use a networked format like
tar (eg.
myhost.com:/dev/tape to use the tape drive on my
host.com).
--exclude-files file (-F) file
a list of files not to include in the backup. Consists of
a
series of suffixes to exclude separated by spaces. It is
not
case sensitive (ie. .o is the same as .O) For example,
the
default setting of ".o ~" excludes all files that end in
.o and
all files that end in ~ .
--bad-checksum-on (+C)
When doing a mkinfo or complete restore, if a file with a
checksum is encountered, will try to continue without
prompting.
--compress-type (-c) num
0 No compression
1 Use an external compression program. The program to
use is
hard compiled and can be changed by editing
defaults.h. The
default is set to use gzip.
2 Use the internal compression program which is similar
to
compress. It doesn't provide as good a compression as
gzip
but it is much quicker. Note that this compression us
es about
2MB of RAM extra so on heavily loaded machines, it can
degrade performance. Default.
3 Use the internal compression program which is similar
to
gzip. It provides the same level of compression as
gzip and
is much quicker than the external version. It is not
as
quick as option 2, but much faster than 1. In addi
tion, it
doesn't use anywhere near the memory or system load
that 2
requires.
--exclude-compress (-X) file
Specifies which group of files to exclude from compress
ing.
Comprises a string of suffixes separated by a space - eg.
``.gz
.gif'' would exclude files ending in .gz and .gif. It is
not
case sensitive (ie. JPG = jPg etc..) Default is ``.gz
.gif .Z
.zip .jpg .bmp''
gzip is used for compression. It assumes that gzip is on
your
path. If you use another compress program, change the en
try in
defaults.h and recompile.
--print-dir (-d) archive_id
Prints the directory of the archive whose id is
your files may be in /usr/local/src/xzy with nothing in
/usr,
/usr/local or /usr/local/src. When you select /usr from
the top
restore directory, you will automatically be placed in to
/usr/local/src/xyz where the files are, and the empty
directories will be skipped. Default.
--auto-descend-off (-D)
Turn the above off.
--prompt-archive-on (+e)
When selecting restore, prints a list of all known
archives and
the user can select which one of these to restore from.
Default.
--prompt-archive-off (-e)
When selecting restore, automatically selects the archive
in the
tape drive (or regular file). User not given the option
of which
archive s/he would like to restore from
--can-seek-on (+E)
Tells taper that your tape drive supports a seek ioctl
for
positioning to a particular block. Default depends on
tape drive
type.
--can-seek-off (-E)
You tape drive doesn't support an ioctl for positioning
to a
particular block. Taper will use reads to position. De
fault
depends on tape drive type.
--volume-title (-g) title
Title of volume. Default is NULL. Only applicable in
backup.
--get-blocksize-on (+G)
If this option is defined, then after taper opens a tape,
it
tries to get the block size using an ioctl call. This is
only
for the new zftape driver. Default is on for zftape and
off for
others.
--soft-links-on (+h)
With soft links, store the file, not the link.
--comp-head-start (+H) minutes
It is useful for taper to always have a full buffer of
data
available to send to the tape drive so that the tape
drive never
has to sit and wait. This can be a problem if compression
is
taking a long time. By giving the compression program a
head
start, you can try and ensure that taper always has data
available to write. A good starting number for large
backups is
10 minutes. Default is 0.
--info-files path (-i) path
directory where archive information files are saved. Also
where
file sets are saved. Default ~/taper_info).
--compress-info-on (+I)
turns on compression of info files. Automatically will
compress
&after creation and will decompress when required.
Default.
--compress-info-off (-I)
Don't compress info files.
--proc-device (-j) num
This specifies the device number of the /proc system.
This is
avoid backing up of the /proc filesystem which is a run
time
filesystem used to store the current state of the operat
ing
system. If this value is set to 0, then no checking is
performed. The default value is 1 (which is the device
number of
the /proc filesystem under Linux).
--min-before-seek (-J) num
When restoring, if the tape needs to be advanced by less
than
min-before-seek blocks, then a seek will not be issued,
rather
the tape will continue streaming. This is because if you
have a
is set, taper will try and erase the tape using a stan
dard ioctl
call. Floppy tape drives (those based on ftape and zftape
require this option to be set.
--erase-tape-off (-k)
Will not try to erase a tape before writing new data to
it.
Most SCSI drives do not require erasing, so you can use
this
option for them.
--illegal-end-of-tape-off (-K)
Tells taper that your tape drive correctly handles the
end of
tape condition. Use the utility test end of tape to see
if your
tape drive handles this properly. Default.
--illegal-end-of-tape-on (+K)
Tells taper that your tape drive doesn't handle the end
of tape
condition as per BSD semantics. Use the utility test end
of tape
to see if your tape drive handles this properly.
When taper encounters an error in the tape, it asks you
whether
this signifies an end of tape or a real error.
--log-file (-l) file
name of log file. Default ~/taper_log.
--log-level (-m) number
logging level (0 = no logging..4 = all). Default = 2.
--size-dirs-on (+M)
When you select a directory in backup, it will calculate
the
directory size. This may take a while for big directo
ries.
Default.
--size-dirs-off (-M)
No sizing of directories on selection. Quicker, but no
indication of size of backup being contemplated.
--limit-log-file (-L) number
specifies how many megabytes the log file should be at a
--overwrite (-o) num
level of overwrite
0 - no overwrite.
If the file exists on the hard disk, then it is not
overwritten
1 - more recent overwrite. Default.
If the file exists on the hard disk, it is only over
written
if the file on the backup device is more recent
2 - unconditional overwrite.
The file on the hard disk is always overwritten
--tape-overwrite-on (+O)
When doing unattended backups, if a tape contains unrec
ognized
data, then if this option is set, then the tape is auto
matically
overwritten.
--tape-overwrite-off (-O)
When doing unattended backups, if a tape contains unrec
ognized
data, then the tape is not overwritten - the backup is
aborted
and a message is mailed to the user. Default.
--preference-file (-p) file
Name of preference file
--tmp-dir (-P) dir
Specifies the directory where temporary files are to be
placed.
Default is /usr/tmp.
--only-volume (-q) num
Tells restore/backup to only show only volume 'num' in
its
window. Default is 0 which means show all volumes.
--fast-fsf-on (+Q)
Quick fast forwards are enabled. See the section of
Quick fast
forwards for more details. Default is on for zftape and
SCSI
and off for ftape.
--exclude-dirs (-R) path
which directories to automatically exclude from the back
up
process. The string consists of a list of space separat
ed
directories. Note that the selection is recursive. For
example,
if you exclude list was "/tmp", the contents of /tmp will
not be
backed up. Also, none of the directories in /tmp (eg.
/tmp/junk)
will be backed up. Users of netscape may also want to
put their
cache directory on the exclude list ( /.netscape/cache)
since
this is where netscape stores its list of recently visit
ed
locations and is a waste to backup. Default is "tmp
/usr/tmp
/var/tmp"/.
--strip-number (-s) number
remove number leading pathnames. When backing up, the
full
pathname is stored in the information file (and on the
tape).
You have the option of removing number leading paths when
restoring. For example, if the file backed up was
/home/yusuf/taper/README, by specifying -s 2, the file
will
appear in the restore menu as taper/README. If -s1 was
specified, the file will appear as yusuf/taper/README.
If there are more strips than paths (eg. specifying -s 6
in the
above example), then the filename only is used (eg.
README).
The value -s 99 is a special value and tells restore to
work out
the optimum number of strips based on common leading
paths
The default is -s 99 (ie. auto strip).
--set-blocksize-on (+S)
If this option is defined, then before taper erase a
tape, it
tries to set the block size using an ioctl call. This is
mainly
for the new zftape driver, but some SCSI drives need it.
Default is on for zftape and off for others.
Title of archive. Default is NULL. Only applicable when
archive
is being created for the first time. If supplied when do
ing an
archive append, then it is ignored.
--incremental-on (+u)
When doing a backup, only backup those files selected
which are
more recent than the files that are on the backup
archive.
Default.
--incremental-off (-u)
When doing a backup, backup all files selected. This op
tion can
be overriden when selecting files for backup.
--unattended-file (-U) file
Gives the name of a file/directory to be backed up in
unattended
mode. If the filename begins with a @, then it is taken
to be
the name of a file set, otherwise, it is a file. If the
first
letter of the filename is a '!', then the file is exclud
ed
rather than selected. You can use as many -U as you like
on the
command line.
--recent-restore-on (+w)
When doing a restore, restore the most recent file with
the
given filename. For example, if /etc/hosts has been
backed up
twice in different volumes, restore the file which is
newer.
Default.
--recent-restore-off (-w)
When doing a restore, restore the file that was selected
- for
example, if you selected /etc/hosts in volume 1, this if
the
file that will be restored, even if there is an
/etc/hosts in
volume 2 which is more recent. This option can be over
riden
when selected files for restoration.
prints version being used. Also prints the options taper
was
compiled with
--unattended-id (-V)
when appending to a backup in unattended mode, if this
value is
not -1, then taper will check that the tape in the drive
belongs
to this archive ID before appending. If another archive
is in
the drive, then the backup will not be made. Default is
-1.
--block-size (-x) num
The size of a block. Apart from the tape header, data is
transferred to the tape device in blocks of 'num' size.
The
default is 28K. If you use a SCSI, see then note above.
Num is
in bytes. Note that this value MUST NOT exceed the size
of
DOUBLE_BUFFER in defaults.h.
--use-eom-on (+y)
Uses the ioctl eom to position to end of tape rather than
fsf.
--use-eom-off (-y)
Uses multiple fsf to position to end of tape. Default.
--min-free (-Y) num
Specifies the minimum amount of free disk space that must
exist
before a file is compressed. num is in kilobytes. Default
is
4096K.
--prompt-dirs-on (+z)
Turns on confirmation when selecting directories for
backing up.
Default.
--prompt-dirs-off (-z)
Turns off confirmation when selecting directories for
backing up
detects the end of tape. Default is 0 (auto-detect).
14.2. Backup Module
Select the backup option from the taper menu.
Four windows are displayed:
The top left represents the file system (hard disk) and you can
move
it using the arrow keys. To enter a directory, press ENTER when
the
highlight is on that directory. To include a file/directory in
the
backup, press 's'. Selecting a directory will select all the
files in
the directory recursively. To unselect a file/directory, press
'u'. If
the file is only selected because it's directory is selected
(eg. if
you select /usr/john, then /usr/john/prefs is indirectly se
lected),
then you must exclude the file/directory. Pressing 'e' will
exclude
the file/subdirectory from the backup. Pressing 'd' will show
details
about the file the highlight is on. Pressing 'j' will allow
you to
jump to a particular directory.
The top right window shows what is currently on the archive.
You can
use the arrow keys to move the display up/down. If the entry
is in
brackets, this means that this is an exclusion. If there is
previous
archive, you can move the arrow keys down to a file/directory
and
press 's' to select this entry.
The bottom left window shows the files that you have selected
for
backup. To unselect a file that you have selected for backing
up,
move the highlight to the file you wish to remove from the
backup set
and then press 'u'.
The bottom right window shows the files that you have excluded
from
Pressing 'q' will abort backup
You can abort a backup by pressing q or Q while the backup is
being
made. This will cleanly stop backing up.
14.2.1. Backup modes
There are two modes for backup - full and incremental. The de
fault
incremental mode (which can be changed by changing the options,
the
preference file or command line). In incremental mode, when you
select
a file for backing up, taper looks at what is on the archive
already.
If the file you have selected for backup is more recent (or
doesn't
exist on the archive) than the one on the archive, it is backed
up: if
the file you have selected for backup is the same or older than
the
one on the archive, it is not backed up. For directories, taper
checks
each file in the directory recursively. To see if a file will
be
backed up, look at it's size in the selection (bottom) window -
if it
is zero, it means that taper will not back up that file.
In full mode, taper backups the file regardless of what's on
the
archive.
The backup mode can be changed for each file selection. Move
the
highlight in the selection window to the file you wish to
change and
then press 's'. To the left of the filename is an 'I' if the
file is
to be saved as incremental, or 'F' if it is full.
14.3. Restore
Select the restore option from the taper menu. If you have se
lected
the 'prompt archives' option (default), then taper prints a
list of
all the archives it knows about and lets you select which one
you
would like to work with. The highlight will be automatically
position
on the archive that is in the tape drive (or which the regular
Three windows are displayed:
The top left window shows details of what is currently on the
archive.
You can move around using the arrow keys. To enter a directory,
press
ENTER when the highlight is on that directory. To select a
file/directory for restoration, press 's'. Selecting a directo
ry will
select all the files in the directory recursively. Pressing 'd'
will
show details about the file the highlight is on. Pressing 'j'
will
allow you to jump to a particular directory.
The top right window shows how the archive was constructed (ie.
what
files/directories were selected for inclusion in the archive).
An
entry in brackets indicates that this was an exclusion. You
can use
the arrow keys to move the display up/down. You can press 's'
when you
are on a file or directory to select that particular file/di
rectory.
The bottom window shows the files that you have selected for
restoration. To unselect a file that you have selected for
restoration up, move the highlight to the file you wish to re
move from
the restore set and then press 'u'.
Pressing TAB moves between the windows. Pressing 'h' will dis
play a
help screen. Pressing 'f' will finish selection and commence
backup
Pressing 'q' will abort backup
While the restore is in progress, you can stop it by pressing q
or Q.
14.3.1. Restore modes
14.3.1.1. Full restore
If taper cannot find an info file for the tape in the drive,
then it
offers you the option of doing a full restore. In full restore
mode,
all files on the archive are restored in all volumes. There is
no
select
a file for restoration, taper automatically selects the file
which is
on the latest volume (eg. you may have three volumes and
/etc/passwd
is on each. Even if you select the /etc/passwd in volume 1,
taper will
automatically select the volume 3 file, not the volume 1 file).
This
is indicated by an 'M' in the volume column of the selection
window.
Note that even if you are in a restricted volume view (using
--only-
vol), most recent restore will still automatically find the
latest
volume of your file on the archive.
14.3.1.3. Fixed volume restore
In fixed volume mode, whichever file you select is the one that
is
restored (eg. in the example above, the /etc/passwd on volume 1
would
be restored, despite the fact that the /etc/passwd on volume 3
is more
recent).
To toggle between fixed volume & most recent mode for a partic
ular
file selection, press 's' when on the selected file in the se
lection
window.
14.4. Recreate info file
Select the recreate info file option from the taper menu.
This recreates archive information files given an archive. It
also
checks that the checksums of all the files are valid. Any er
rors are
printed to the log file.
Basically, it looks for an archive on the devices given. It
prints the
archive ID and archive title on the screen, then prints each
file as
it passes it while recreating the archive information file in
the
directory specified. Because it has to traverse the whole
archive, it
continue the mkinfo or you can tell taper to assume that this
is the
end of the backup. This is useful if a backup got killed by a
power
failure or something.
During the traverse, you can press q or Q to stop it. If you do
that,
you will corrupt your info file and you will have to re-run
mkinfo to
restore your info file.
Mkinfo may return errors when trying to advance to volumes that
do not
exist. Do not worry about this - just select abort and the info
file
will be correctly constructed.
14.5. Verify archive
This option goes through an archive and checks that what's on
the
archive matches what is on the hard disk, byte by byte. If a
file is
on the archive but not on the disk, it is ignored. Any discrep
ancies
are printed to the log file. It also checks the checksums and
if a bad
checksum is encountered, like mkinfo, you have the option of
trying to
continue or to assume that the rest of the backup is bad.
14.6. Utilities
14.6.1. Make tape
This prepares a tape for use by taper. This is not needed by
zftape or
ftape tape drives. Some SCSI drives need this.
14.6.2. Test make tape
This tests whether your tape drive needs to have make tape run
on new
tapes. It is important that a BRAND NEW tape is used when run
ning this
test - ie. one that has never seen the inside of your (or any
one
else's) tape drive before.
--proc-
device (or -J) option. The default is 1.
14.6.4. Test fast fsf
Some tape drives can quickly advance between volumes on the
tape. This
also quicker restoration of files from multiple volumes. Use
this
utility to see if your tape drive can use fast fsf.
14.6.5. Test can seek
Taper can use the seek ioctl to make seeking for files in re
store a
lot quicker. Basically, the block on which the file should lie
is
calculated and then a seek is issued to quickly advance the
tape drive
to this block. However, your tape drive must support the seek
ioctl
for this to happen.
You can use this utility to check if your tape drive supports
the seek
ioctl.
Before using test can seek you must ensure that you have the
correct
setting for test fast fsf.
14.6.6. Test end of tape
According to BSD semantics, when a program requests a further
read
from a tape when at the end of the tape, it should return ei
ther a 0
error or a -1 error with errno set at ENODATA or ENOSPC.
Some tape drives do not do this, but return an I/O error which
means
that taper doesn't know whether an end of tape is reached or a
true
I/O error is encountered.
Therefore, this utility will check to see if your tape drive
handles
the end of tape condition legally. If it does, well and good.
If it doesn't, taper assumes that I/O errors are really end of
tapes.
The problem, of course, is that we don't know when true I/O er
the
number of volumes you wish to delete. Taper then deletes these
volumes. Note that once a volume is deleted, it cannot be re
stored.
14.6.8. Reindex info file
To speed up access, the info file cluster contains a couple of
index
files. These files are arranged as m-tree indexes. Occasional
ly, these
index files can get corrupted. If you think this has happened,
you can
reindex the files.
Also, if you want to restore your files onto a different ma
chine, and
do not want to recreate the info files on that machine, you can
take a
copy of the info files to the new machine. Rather than taking
all four
of them, you can take the two main files, and rebuild the in
dexes
using this option, on the new machine.
14.6.9. Look for recursive links
It is possible on some systems that you have recursive links.
For
example, you may have a soft link that points to "./". This
leads to a
recursive situation which does not matter if you do not follow
links.
If you follow links (via the soft-links option), then taper
will crash
as it encounters this link. Running this utility identifies any
recursive links you may have so that you can either tell taper
not to
use hard links, or so you can remove the offending link.
Just enter the starting directory you wish to test - taper will
look
for recursive links and tell you if you have any. In some situ
ations,
taper will seg fault at the link. This is not a problem since
you will
know where the offending link is by when it seg faults.
14.6.10. Attempt Recovery
If you have an archive that is corrupted, you can try to tell
tape header
volume header
file header ) repeated for each file
file ) in the archive
Once a corruption occurs, you have to find the file header of
the
first file beyond the corruption and tell taper where it is and
then
taper can continue to recover files.
In this utility, you tell taper where you want to start the re
store
(in bytes relative to the beginning of the archive). Taper will
then
print the filename and file header for confirmation. If you get
garbage here, then you are not positioned on the file header
and you
must continue trying until you are.
Once you are correctly positioned, say YES and taper will try
and
recover files beyond the corruption point.
The easiest way to find the correct position is to use an edi
tor such
as midnight commander to scan the file until you can locate the
correct location.
If you are using a tape drive (especially on a volume other
than the
first), it is a good idea to read the tape data into a disk
file using
dd and then work with the file, rather than the tape. It is
just
quicker.
If you are trying to recover data from a volume other than the
first
one, you must position the tape (using the non-rewinding de
vice) on
the volume you wish to recover from, read the tape data into a
disk
file and then work on this disk file.
When you are working with the disk file, start taper using the
-T l
preference.
on, the
files will be appended to the archive. If --append-off,
then the
existing archive will be overwritten.
--erase-tape
If unrecognized data exists on the tape, then if --erase-
tape-
on, the unrecognized data will be overwritten. If
--erase-tape-
off, the data will not be overwritten and the backup ses
sion
will be aborted.
--unattended-id
If this option is not -1, then taper will only append the
to
this archive if it is archive "id". If some other
archive's tape
is in the drive, the backup will not be made and the user
will
be mailed about it.
It is important that these preferences are correctly set, since
taper
is running in unattended mode, you will not be prompted for any
confirmation, rather the action will just happen.
To specify the files/directories for backup, use the -U (or
--unattended-file) command line options. If the filename begins
with a
@, then taper interprets this to mean that a filename is a
fileset and
reads in the appropriate fileset.
The files/directories are backed up either as full backups or
as
incremental backups - it depends on what you have set via com
mand line
options, or preference files. If you haven't specified any
thing, then
by default, incremental backups are used.
With unattended backups, it is not possible to have multi-tape
backups
because taper can't prompt you to insert the next tape. If the
end of
the tape is encountered, taper will stop the backup and send
you a
message.
Mail is sent to the root (can be changed in defaults.h) via the
mail
will cause taper to run at 20:10 (8.10pm) every day, using the
fileset
named set.
16. File Sets
You may have a particular set of files or directories that you
always
wish to backup - eg. /etc/passwd, /usr/local/bin, and /usr/lo
cal/etc.
Rather than selecting these files & directories every time you
want to
make a backup, you can select them once and then save the par
ticular
selection to a file set. Next time you wish to backup this par
ticular
set of files, you need only load in the file set and taper will
automatically select the files for you.
To save a file set, press S in either backup or restore. You
will then
be prompted for a name to give to your file set.
To restore a file set, press L in either backup or restore. You
will
be presented with a list of known file sets. Select which one
you want
using the arrow keys and ENTER and taper will load in the file
set.
16.1. File Set format
A file set file simply is an ASCII file that contains three
lines for
each entry selected.
The first line is a capital I or E. I indicates the entry is to
be
selected. E indicates the entry is to be excluded. the select
ed.
The next line is the selection.
The third line is null (this contains a blank for future sup
port of
regexp).
17. Environment variables
o zftape
o ftape
o reg-file
o ide
TAPER_PREFS
the name of your preference file
TAPER_LOG_FILE
the name of your log file
TAPER_LOG_LEVEL
the level of logging taper should do
TAPER_INFO_FILES
the directory in which your info files and set files are
found
TAPE
the name of the rewinding tape device
NTAPE
the name of the non-rewinding tape device
18. Preferences
Rather than having to issue a whole lot of command line options
every
time you wish to run backup/restore, it is possible to store
commonly
used options in a preference file. Individual users can have
their own
preference file in their ~ directory, or there can be a global
preference file in /usr/local/etc. Both backup & restore look
for a
preference file in the following sequence:
1. Check if a filename is given via -p (or --preference-file)
command
line option
2. Check if a filename is given via environment variable TA
PER_PREFS
3. Check if ~/taper_prefs exists. If it does, use this
4. Default of /usr/local/etc/taper_prefs
If no file is given, then internal defaults (as specified
It is possible to change and save preferences within taper. Se
lect the
appropriate option from the taper menu (ie. either global op
tions,
backup options or restore options). To move between options,
press the
up or down arrows (or left and right). To change an option, use
the
left and right arrow keys. When you have finished, press F10
(to
change this from F10, change the entry in defaults.h).
18.2. Saving preferences
There are two methods of saving your preferences. One is to a
preference file or, the second is to a command line file. To
save to
a preference file, select the appropriate option. Give the
filename of
the preference file (default is ~/taper_prefs) and then press
F. Next
time you start taper, the preferences that you have selected
will be
automatically loaded.
The second method of saving is to a command line file. The
saves the
preferences you have selected to a shell script. Select the
name of
the shell script (default is start_taper). To invoke taper us
ing the
selected preferences, issue a `sh start_taper'. This is useful
if you
have temporary (or multiple) configurations and you don't want
to
overwrite your existing preference file.
18.3. Preference file format
The format of a preference file is a text file with one prefer
ence per
line. A preference is given:
PREFERENCE=VALUE
See the file 'pref_example' for an example of a preference
file.
18.4. Special preferences
The following preferences can be prefixed by a letter to indi
cate
which tape drive they belong to.
o set-blksize
o get-blksize
o erase-tape
o fast-fsf
o block-size
o tape-name
o ntape-name
o can-seek
For example, if you have the following line in your preference
file,
block-size = 12300 z-block-size = 43122 z-tape-name = /dev/qft0
z-
ntape-name = /dev/nqft0 s-tape-name = /dev/scsi s-ntape-name =
/dev/nscsi -z
then the block-size is set to 43122 only if the zftape driver
is
selected, otherwise, it is set to 12300. Also, the device names
are
set to /dev/qft0 & /dev/nqft0 if the zftape driver is selected,
or to
/dev/scsi &/dev/nscsi if the scsi driver is selected. If the
preference given is block-size=43122, then the block-size will
be set
to 43122 regardless of the tape driver selected.
19. Screen Colours
It is possible to change the colours displayed. This can be
done from
the command line or from within a preference file.
from the command line, or
color-normal = yellow,blue
in a preference file.
Below is a list of the possible colors to change
color-title
title bar (blue on white)
color-main
main screen (white on blue)
color-dialog
dialog boxes (white on red)
color-status
status boxes (white on red)
color-directory
screen used to select files (black on green)
color-selected
screen used to show selected files (white on blue)
color-selected
screen used to show unselected files (white on blue)
color-bottom
status bar at bottom of the screen (blue on white)
color-help
screen used to show help screen (black on cyan)
color-on-vol
screen used to show current volume contents (white on
purple)
color-form
screen used to get data (eg. change preferences, enter
volume
title) (white on black).
20. Multiple Tape Backups
For the use of multiple tape backups, these are the following
assumptions made.
20.2. Reading
End of tape is indicated by one of:
o error code == -1 && errno == ENOSPC
o error code == -1 && errno == ENODATA
o returned less bytes than we asked for
End of volume is indicated by
o zero bytes read
o error code == -1 && errno == ENODATA
This is what ftape does for floppy controller tape drives. If
are
using another sort of tape drive, then the multiple tape backup
feature will only work if the above is true. If it is not true
and you
are able to work out what your particular tape drive does, if
you send
me details, I will try and incorporate it into future releases.
I'm
especially interested in users of SCSI drives since I have no
access
to them.
If your tape drive doesn't support the above, you can use the
--tape-
size option to tell taper the size of your tape drive. Tape
size is
specified in megabytes, eg. --tape-size 120 means that your
tape is
120MB, --tape-size 250 means that your tape is 250MB. Note that
these
must be uncompressed sizes. You must specify the same number
every
time you start taper - you can put it in your preference file
if
necessary. If you don't specify the same number, you will get
very
spurious, unpredictable results. Note that you should subtract
about
10% from the manufacturers stated size to account for format
ting etc..
(eg. for a 120MB tape, tell taper that it is 108MB).
21. Cross platform support
you
define the TAPER_BIG_ENDIAN makefile option, taper will con
vert
everything read from the tape to big endian so that your ma
chine
can use it. Even if you make a backup from a big endian ma
chine,
the data on the tape will be in little endian format.
o The info files are also stored in little endian format so
that they
should be portable accross different architectures.
o Data types are defined in config.h. Taper uses 32 bit
signed &
unsigned data, unsigned 16 bit, and 8 bit signed & unsigned
data.
Change the appropriate types for your machine. Attributes
such as
gid_t, uid_t, dev_t, umode_t, and time_t are hard-coded to
the
intel defaults of unsigned short (unsigned 16 bit) and long
(32
bits) respectively. If your machine uses different types for
these,
then you should look at the code in tapeio.c, specifically,
the
functions tape_read_fi and tape_write_fi and modify it to
convert
the intel types to a type applicable for your machine. If
your
machine uses longer data types than an unsigned 16 bit used
by
intel linux, then, for the time being, you are in strife.
In a
future taper version, I intend to move away from the 16 bit
format
and use my own representation. However this will mean that
existing
archives are incompatible with the new format, so I won't be
doing
this for a while.
o Since version 6.3, taper aligns all structures on addresses
that
are divisable by 4. It does this by padding out the strings
so that
they have a length divisable by 4. Then an offset is added
(which
is the number of bytes that the structure needs to make it a
length
of a multiple of 4). Versions less than 6.3 do not support
this, so
it is unlikely that these archives will work on machines
that
require aligned addresses. However, for machines that do
Maximum length of total pathname is set by MAXNAMLEN. It is
usually
set to 255 bytes long. If you have a very long pathname, then
you may
cause taper to segfault.
If a file grows while taper is trying to back it up, only the
beginning part of the file is backed up (specifically, the num
ber of
bytes the file was when taper started to back up this file) and
the
rest is lost. This is because of inadequate mandatory file
locking. I
know that this is being worked in the kernel and hopefully,
when the
kernel code pertaining to this is stable, I can implement it in
taper.
23. Bugs
Please report bugs to me at the address below. If the bug caus
es a
segment violation or causes the program to crash in any other
way, if
you know how to use gdb, could you re-compile your source using
the
Makefile.debug makefile, and tell me what source line the pro
gram
crashed. Also, please tell me what kernel version, gcc version,
C
library version, ncurses version you are using.
As this is a sideline, I may not be able to respond to e-mail
immediately. However, if you do not receive a reply within one
week,
then repost as I may have forgotten about it!
24. Major bugs in early versions
There was a bug in taper-5.1.1 when creating archives. If you
made an
archive with 5.1.1, unfortunately, you will have to re-backup
if you
find that restore doesn't work since it was a pretty major bug.
However, since taper-5.1.1 was only released a day before 5.1.2
(which
fixed the bug) was released, hopefully, not too many people are
affected.
From taper-5.0 to taper-5.1.3, there was another not so serious
bug.
your
archive and taper will recognize the buggy archive and fudge
your info
file automatically for you.
I don't believe it - the bug crept back into 5.1.5 - it's like
a virus
that won't go away! Anyway, 5.2 corrects it.
In versions 5.6 to 6.1 (inclusive), there was an obscure backup
bug
which causes a problem if there were two buffers queued for
output
when the tape drive was closed. Unfortunately, there is no fix
for
this, however, it is a relatively rare bug, and at most, you
will lose
the last two blocks of the backup. This bug only occurred if
you used
triple buffering.
In versions up to 6.1.3, there was a problem with internal com
pression
of files > 32K. It was a spurious bug.
25. Finally
Wow! You made it through all the 'documentation'! I'm im
pressed.
I would appreciate a short message if you find the program use
ful and
are using it. This will allow me to work out whether people are
using
it and whether to continue development. If you could also state
what
machine you are using, hard drive/floppy controllers, tape
drive type,
SCSI card etc.., then this will help me to maintain a compati
bility
list.
Happy backing up,
Yusuf Nagree
yusuf@e-survey.net.au
02MAR00
Man(1) output converted with
man2html