3. "Using Ftape" related questions !

3.1 How fast is Ftape ?

You can achieve quite respectable backup and restore speeds with Ftape: a Colorado DJ-20 and an Adaptec 1542CF controller, has been measured at 4.25Mbyte/min sustained data transfer rate (no compression) across a 70Mbyte tar archive, while comparing the archive on the tape with data on an IDE disk. The speed of Ftape is mostly dependent on the data transfer rate of your FDC: The AHA1542CF has a ``post-1991 82077'' FDC, and it will push 1Mbit/sec at the tape drive. If you have an FDC which can only deliver 500Kbit/sec data rates, you will see half the transfer rate (well, roughly).

3.2 When I write to some of my tapes, they seem to spend a lot of time "shoe-shining," or repositioning instead of streaming. Is something wrong with my system?

There has been a few reports of "shoeshining". This is when the tape just seems to run back and forth endlessly. This has been seen on a Jumbo 250 (74407.3051@compuserve.com) and on an Iomega 250 Ditto Insider (tom@opus.cais.com). In the latter case it has been narrowed own to using an ELF Linux and running off a SCSI hard disk (connected to an Adaptec 1542cf). Please contact me if you have an update to this problem.

<from the Ftape-Howto>

<answer from Tim Jones>

Presumably other backup utilities could be modified to use a similar technique.

<answer by Michael Hamilton>

GNU tar doesn't use buffering in this way. The commercial backup program "bru" is able to multi-buffer using shared memory; this works only when writing compressed archive with bru (regardless whether you use Ftape's builtin compression)

<answer by Claus Heine>

3.3 Do I have to reboot to the DOS world to format tapes?

Not if you are using the latest version of the Ftape drivers from the Ftape Home Page. To format a QIC-80, TR-1, TR-3, QICWide 3010 or 3020 tape, make sure that the Ftape-format module is loaded and use the special version of the mt utility included with Ftape in the contrib/gnumt directory. Use the ftformat option with an optional argument that is interpreted as follows:

ftmt ftformat 0

keep the existing header information, DON'T erase the tape prior formatting it.

ftmt ftformat 1

keep the existing header information and erase the tape prior formatting it.

ftmt ftformat 2

DON'T keep the existing header information, DON'T erase the tape.

ftmt ftformat 3

DON'T keep the existing header information, but erase the tape prior to formatting it.

"ftformat" without parameter is synonymous to "ftformat 1" NOTE: Do not try to format Ditto 2GB tapes.

<answer from Tim Jones>

NOTE: The formatting ioctl interface as well as the user interface have been completely rewritten for Ftape-4.0. Further information will be included in the manual of Ftape-4.0.

<answer from Claus Heine>

3.4 Ftape detects more bad sectors than DOS on QIC-3020 tapes

If you look at the difference, you will notice that Ftape always detects 2784 sectors more than DOS.

The number that Ftape reports is correct (of course :-). Each correctly formatted QIC-3020 tape has 2784 sectors at fixed positions that are marked in the bad sector map. To quote from the specs:

This gives 12 tracks * 2 * 4 segments * 29 sectors == 2784 sectors.

So Ftape choose to report the real number of sectors that cannot be used on the tape, while DOS gives a more optimistic number giving a better indication of tape quality. (Ftape's behavior might change in the future to detect correct formatting and display the separate numbers. It has rather low priority though).

QIC-3010 are alike QIC-3020 tapes regarding this.

<from the Ftape-Howto>

3.5 Is it ok that I'm not hearing the tape move when I do a fsf or a bsf with mt?

Yes. The driver merely updates an internal counter when those commands are issues. The tape should move to the proper location on the next read or write access to the tape drive.

<from the Ftape-Howto>

3.6 Why does my XYZ backup program complain about "Invalid argument" errors?

You may want to read the section "Tape blocks" of the manual (use its "Concept index" to directly jump to that section)

<answer by Claus Heine>

3.7 Why do I get "/dev/qft0: No such device" errors?

Solution You need to load the zftape.o module as well. With Ftape-3.* the ftape.o module doesn't implement the VFS interface. This is done by zftape.o.

<answer from Claus Heine>

3.8 I get "device busy" when I make multiple backups on a tape using some script.

The "device busy" messages can only occur while the Ftape devices are still held open by some program. As soon as the close() system call has completed the busy flag is cleared. May be "bru" or some other program has still forked off a child that dies delayed?

However, this is not a bug of Ftape. It seems that the parent tar process exits before its child has closed the tape device. I know, however, from hacking the tar code ages ago, that tar properly waits for its parent to die.

However, the busy message simply means that the "busy" variable is still held at 1 (zftape/zftape-init.c). And this simply means that there still is a process hanging around that holds the tape device open.

I think I have it (only for the case of tar 'cause I have the source code.

Now, parent tar only waits for its child to die. gzip surely doesn't wait for the grand child as the gzip is a result of an execlp().

What I don't know is whether the grand child should be implicitly waited for by the parent tar, or if the wait() function also waits for grand childs.

But this seems to be the problem: the parent tar already has exited while its grandchild still is busy closing the archive. One hardly will notice this problem if the close() happens fast (i.e. regular files, block devices, also other tape devices?), but it isn't a bug in Ftape, but either in the backup programs or in the kernel or maybe libc exit code.

Don't know if the considerations above also apply to bru. If there is no grandchild and the parent process properly waits for its childs then there shouldn't be a problem.

<answer from Claus Heine>

3.9 How do I "...." with tar?

If your version of tar is v1.11.1 or earlier, consider upgrading to v1.11.8 - This version can call GNU zip directly (i.e.: it supports the -z option) and has an elaborate help included. Also, it compiles right out of the box on Linux.

<from the Ftape-Howto>

3.10 What block-size should I use with tar ?

When using compression, and in all general, it can be a benefit to specify to tar, that it should block the output into chunks. Since Ftape cuts things into 29Kbyte blocks, saying `-b58' should be optimum.

"Why 29Kbyte?", I hear you cry. Well, the QIC-80 standard specifies that all data should be protected by an Error Correcting Code (ECC) code. The code specified in the QIC-80 standard is known as a Reed-Solomon (R-S) code. The R-S code takes 29 data bytes and generates 3 parity bytes. To increase the performance of the ECC code, the parity bytes are generated across 29 1Kbyte sectors. Thus, Ftape takes 29Kbytes of data, adds 3Kbytes of ECC parity, and writes 32Kbytes to the tape at a time. For this reason, Ftape will always read and write 32K byte blocks to be able to detect (and correct) data errors.

If you are curious, and wish to know more, look in the ecc.c and ecc.h files, for an explanation of the code and a reference to a textbook on Reed-Solomon codes.

<from the Ftape-Howto>

3.11 Where can I find the tar/mt/cpio/dd binaries - sources - manpages?

All of these tools have been developed by the GNU project, and the source (and man page) can be fetched from just-about any ftp site in the world (including ftp.funet.fi, tsx-11.mit.edu, and sunsite.unc.edu). In any case they can be fetched from the official GNU home site: prep.ai.mit.edu [18.71.0.38]:/pub/gnu. The latest versions (as of September 12 1996) are:

They all compile out of the box on Linux v1.0.4 / libc v4.5.19 / gcc v2.5.8.

<from the Ftape-Howto>

3.12 If I use tapers compression, is it a bad idea to use the compression with zftape, or would it be better to not use tapers compression, and let zftapedo it?

It is not bad as such to compress data twice (which would be the case when using tapers compression together with zftape's compression) but it doesn't make any sense. You won't gain much further compression, but only waste CPU cycles.

Tapers compression should be quite safe, as taper compresses single files; in contrast to tar -czf ... which makes the entire data stream a large compressed block of data, which is really a bad thing with serious backups as a single bad byte at the beginning of the archive can make the entire archive unusable, well, it will be at least quite difficult to recover.

<Answer from Claus Heine>

3.13 How does zftape compression compare to say gzip -9?

gzip -9 is better (i.e. one gains higher compression). zftape's compression is comparable with the Un*x compress program, but should be faster, and is faster than gzip.

<Answer from Claus Heine>

3.14 I don't trust compression, but hear that the sftape interface is going away. What should I do?

Use the zftape interface, but don't load the zft-compressor module. The device then becomes /dev/qft0.

<answer from Tim Jones>

3.15 Ftape says "This tape has no 'Linux raw format"

mt -f /dev/nftape erase

<from the Ftape-Howto>

3.16 Can I exchange tapes with someone using DOS?

No. The DOS software conforms to the QIC-80 specs about the layout of the DOS filesystem, and it should(?) be a small problem to write a program that can read/write the DOS format. In fact, I'd bet that creating a nice user interface would be a bigger problem.

<From the Ftape-Howto>

3.17 How does `mt eom' work when you've started overwriting a tape in the middle?

(EOM is "End Of recorded Media", the position right after all data already recorded to the tape)

One cannot use tape "files" like files on an ordinary file system.

In principle, a tape doesn't allow anything but appending new data at EOM. However, if one positiones just in the middle of the already recorded data AND starts writing, then the driver first deletes all following files (thus moving the EOM to the actual position) and then starts writing.

Thus, the new EOM after finishing the write process, is then after the newly recorded data.

One of the consequences of the above is, of course, that writing to the tape in the middle of the already recorded area, is destructive in the sense, that it not only overwrites the "file" the tape is positioned at, but also deletes all following files.

<from the Ftape-Howto> <Answer from Claus Heine>

3.18 When I made backups before using taper, under the 2.0.29 ftape my drive didn'tsupport fsf, under the new zftape it does, why would this be, and whatexactly is fsf ?

cp somefile /dev/my_what_ever_tape

As this is quite inconvenient, somebody invented something which is known under the name file mark or eof mark (eof == End Of File). Those marks don't separate files that have been backed up to the tape device, but only separate blocks of data (whatever data that might be).

Now, the sense of those file marks is, that it is possible to skip between different archives on the tape more quickly than would be possible with reading the data back.

The commands to do that are:

mt fsf

fast skip to the next file mark towards EOT (End Of Tape)

mt bsf

fast skip to the next file marks towards BOT (Begin Of Tape)

mt -f /dev/nqft0 rewind
mt -f /dev/nqft0 fsf
tar -xvf /dev/nqft0 

<Answer from Claus Heine>

3.19 What exactly is the difference between ftape, and zftape?

When Ftape was young there were two versions of the floppy tape driver, one of them was called zftape because of its built-in user-transparent on-the-fly compression. Whether such a thing is a feature or a bug ('cause this needn't be done in kernel space) is another question. However, the ioctl interface and file mark handling provided by zftape was much better and had less bugs. And zftape allows to use floppy tape cartridges with different OS. Well, you can't exchange data, but zftape won't overwrite volumes created by your Windoze program, and vice versa.

Nowadays, Ftape is name of the entire floppy tape driver package AND ftape.o is the file-name of the kernel module that implements the low-level hardware support. zftape has ceased to exist as a separate package, but the new Ftape versions (since ftape-3.00) contain a zftape.o module that needs to be loaded on top of ftape.o (i.e. you need to load BOTH modules to be able to access your floppy tape drive) and implements the file system interface and the advanced (?) features of the previous verions zftape.

<Answer from Claus Heine>

3.20 What is the difference between a rewinding, and non rewinding drive?

Rewinding devices should be used when performing a single backup, non-rewinding devices can be useful when doing multiple backups as one doesn't need to space to EOM (End Of recorded Media) before appending another archive.

<Answer from Claus Heine>

3.21 Can someone tell me how to use mt to rewind my TR-3 drive one record using zftape record, so I can verify it?

Well, it depends. If the tape is still positioned inside the volume just written, "mt bsf 1" (or equivalently "mt bsf") will backspace just to the beginning of that volume (this is how "tar --verify" works). If the tape is already positioned AFTER the filemark that marks the end of the last written volume, then you need to issue "mt bsf 2"

The logic behind this is as follows: "MTBSF #count" backspaces over #count file marks, stops, and then positions on the EOT side of the last skipped file mark. This means, an "mt bsf 2" will position right at the beginning of the previous volume.

<answer form Claus Heine>

3.22 By non-rewinding, they mean that it doesn't automatically rewind, correct? It doesn't mean that under no circumstances it iwill rewind, right? I tried using /dev/zqft0, and it instantly rewinds the tape.

You are right: auto-rewind means, the tape is rewound when the tape device is closed, non-rewinding means, the tape isn't automatically rewound when the tape device is closed (but you can, of course, use the tape motion commands BSF/FSF etc. to position the tape head at every position you like).

<answer form Claus Heine>

3.23 What is the difference between what mt considers a record and what it considers a file?

A record is the minimal amount of bytes that will be accepted by the tape in one read/write operation (except in "variable block size mode" where it just should be the amount of data actually written in a single write operation??).

For zftape every read and write access has to be a multiple of a fixed block size (fixed, but tunable with MTSETBLK). This block size is a "tape record" (as mentioned in the GNU mt man page and defaults to 10kb for zftape.

A "file" (in the sense of the mt man page) is a, well, misleading terminus. What is meant is an area of the tape between two file marks. This is not a file like a file on the file system, in the sense that it could have a name, file access modes, could be moved or copied with cp, mv, rm etc.

Instead, It simply is the area of the tape that was recorded in one backup session, its end is marked by a tape file mark, and its beginning is delimited by either BOT or the file mark of the previous tape "file". That tape "files" are the things that can be skipped with the MTBSF/FSF commands.

<answer form Claus Heine>

3.24 This script implements a simple contents listing for the zftape package using the "MTIOCVOLINFO" ioctl.

Here is as little perl/bash script that lists the contents of a cartridge using the zftape specific "volinfo" ioctl. Hope this shows how to handle this kind of stuff.

What it basically does is the following:

bf/The Perl Script/

#!/usr/bin/perl
#
#     Copyright (C) 1997 Claus-Justus Heine
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; see the file COPYING.  If not, write to
# the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.
#
#   This script implements a simple contents listing for the zftape
#   package using the MTIOCVOLINFO ioctl.
#

$version = <<EOT;
listtape-1.0 -- a perl script to list the contents of a floppy tape cartridge
under Linux using the zftape driver

RCS \$Revision: 1.2 \$
RCS \$Date: 1997/09/11 00:16:28 \$
EOT

$tapedev = "/dev/tape";
$usage = <<EOT;
Usage: listtape [options ...]

Mandatory or optional arguments to long options are mandatory or optional
for short options too.

-f, --file=FILE       Tape device to use. Default is  "/dev/tape".
-h, --help            Print this help.
-?                    Same as '-h'.
--usage           Same as '-h'.
-V, --version         Print version information.

Author: Claus-Justus Heine <claus\@momo.math.rwth-aachen.de>
EOT

while ($ARGV[0] =~ /^-/) { 
$_ = shift; 
if (/--file/) {$_ = shift; $tapedev = $_; next;} 
if (/-f/) {$_ = shift; $tapedev = $_; next;} 
if (/--help/) { print $usage; exit 0; }
if (/-h/) { print $usage; exit 0; }
if (/--usage/) { print $usage; exit 0; }
if (/-\?/) { print $usage; exit 0; }
if (/--version/) { print $version; exit 0; }
if (/-V/) { print $version; exit 0; }
die $usage;
}

&open_tape($tapedev, "status");
while(<FTMT>)
{
$online = 1 if (/.*online.*/);
}

if (! $online) { die "No cartridge present.\n"; }

&mtop($tapedev, "rewind");

printf "%11s%12s%20s%20s\n",
"file number", "block size", "volume size", "tape space";

while (1)
{
&open_tape($tapedev, "volinfo");
while (<FTMT>) {
if (/^file number\s*=\s*([0-9]*)$/) { $filenumber = $1; }
if (/^block size\s*=\s*([0-9]*)$/) { $blocksize = $1; }
if (/^physical space used\s*=\s*([[0-9]*.*)/) { $rawsize = $1; }
if (/^real size of volume\s*=\s*([[0-9]*.*)/) { $size = $1; }
}
close(FTMT);
if (&mtop($tapedev, "fsf 1") != 0) {
&mtop($tapedev,"rewind");
print "\nRemaining space: $rawsize\n";
print "Tape block size: $blocksize\n";
exit 0;
}           
printf "%6d          %5d  %20s%20s\n",
    $filenumber, $blocksize, $size, $rawsize;
}

sub mtop
{
local ($tape, $operation) = @_;
local ($exitval);
system "ftmt -f $tape $operation > /dev/null 2>&1";
}

sub open_tape
{
local ($tape, $operation) = @_;
local ($command);

$command = "ftmt -f " . $tape . " " . $operation . " |";
open(FTMT, $command) || die "Couldn't open $command -- $!\n";
}

The Basch Script

#! /bin/bash
#
#     Copyright (C) 1997 Claus-Justus Heine
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; see the file COPYING.  If not, write to
# the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.
#
#   This script implements a simple contents listing for the zftape
#   package using the MTIOCVOLINFO ioctl.
#

#
# insert better option parsing here
#
TAPEDEV=${1-/dev/tape}

if ! echo $TAPEDEV | grep "/dev/n"
then
TAPEDEV=/dev/n$(basename $TAPEDEV)
fi

if ! [ -c $TAPEDEV ]
then
echo $TAPEDEV is not a character device!  1>&2
exit 1
fi

if ! mt -f $TAPEDEV rewind
then
echo Could not rewind $TAPEDEV - no cartridge present?  1>&2
exit 1
fi

echo -e "\nContents of $TAPEDEV:\n"

printf "%11s%12s%20s%20s\n" "file number" "block size" "volume size" "tape space"

trap "rm -f /tmp/$0.$$" exit

while true
do
if ! foo=$(mt -f $TAPEDEV volinfo |cut -f 2 -d =)
then
echo $TAPEDEV doesn\'t seem to be a floppy tape device 1>&2
exit 1
fi
#
# "echo foo | read foo" will not work as the "read foo" is executed in
# another shell.
#
echo $foo > /tmp/$0.$$
read file blksz used usedunit size sizeunit < /tmp/$0.$$
if ! mt -f $TAPEDEV fsf 1 > /dev/null 2>&1
then
echo -e "\nRemaining space: $used $usedunit"
echo -e "Tape block size: $blksz"
if ! mt -f $TAPEDEV rewind
then
    echo Rewind of $TAPEDEV failed 1>&2
    exit 1
fi
exit 0
fi
printf "%6d          %5d  %20s%20s\n"\
$file $blksz "$size $sizeunit" "$used $usedunit"
done

<answer from Claus Heine>