linux/Documentation/scsi/st.txt
<<
>>
Prefs
   1This file contains brief information about the SCSI tape driver.
   2The driver is currently maintained by Kai Mäkisara (email
   3Kai.Makisara@kolumbus.fi)
   4
   5Last modified: Sun Aug 29 18:25:47 2010 by kai.makisara
   6
   7
   8BASICS
   9
  10The driver is generic, i.e., it does not contain any code tailored
  11to any specific tape drive. The tape parameters can be specified with
  12one of the following three methods:
  13
  141. Each user can specify the tape parameters he/she wants to use
  15directly with ioctls. This is administratively a very simple and
  16flexible method and applicable to single-user workstations. However,
  17in a multiuser environment the next user finds the tape parameters in
  18state the previous user left them.
  19
  202. The system manager (root) can define default values for some tape
  21parameters, like block size and density using the MTSETDRVBUFFER ioctl.
  22These parameters can be programmed to come into effect either when a
  23new tape is loaded into the drive or if writing begins at the
  24beginning of the tape. The second method is applicable if the tape
  25drive performs auto-detection of the tape format well (like some
  26QIC-drives). The result is that any tape can be read, writing can be
  27continued using existing format, and the default format is used if
  28the tape is rewritten from the beginning (or a new tape is written
  29for the first time). The first method is applicable if the drive
  30does not perform auto-detection well enough and there is a single
  31"sensible" mode for the device. An example is a DAT drive that is
  32used only in variable block mode (I don't know if this is sensible
  33or not :-).
  34
  35The user can override the parameters defined by the system
  36manager. The changes persist until the defaults again come into
  37effect.
  38
  393. By default, up to four modes can be defined and selected using the minor
  40number (bits 5 and 6). The number of modes can be changed by changing
  41ST_NBR_MODE_BITS in st.h. Mode 0 corresponds to the defaults discussed
  42above. Additional modes are dormant until they are defined by the
  43system manager (root). When specification of a new mode is started,
  44the configuration of mode 0 is used to provide a starting point for
  45definition of the new mode.
  46
  47Using the modes allows the system manager to give the users choices
  48over some of the buffering parameters not directly accessible to the
  49users (buffered and asynchronous writes). The modes also allow choices
  50between formats in multi-tape operations (the explicitly overridden
  51parameters are reset when a new tape is loaded).
  52
  53If more than one mode is used, all modes should contain definitions
  54for the same set of parameters.
  55
  56Many Unices contain internal tables that associate different modes to
  57supported devices. The Linux SCSI tape driver does not contain such
  58tables (and will not do that in future). Instead of that, a utility
  59program can be made that fetches the inquiry data sent by the device,
  60scans its database, and sets up the modes using the ioctls. Another
  61alternative is to make a small script that uses mt to set the defaults
  62tailored to the system.
  63
  64The driver supports fixed and variable block size (within buffer
  65limits). Both the auto-rewind (minor equals device number) and
  66non-rewind devices (minor is 128 + device number) are implemented.
  67
  68In variable block mode, the byte count in write() determines the size
  69of the physical block on tape. When reading, the drive reads the next
  70tape block and returns to the user the data if the read() byte count
  71is at least the block size. Otherwise, error ENOMEM is returned.
  72
  73In fixed block mode, the data transfer between the drive and the
  74driver is in multiples of the block size. The write() byte count must
  75be a multiple of the block size. This is not required when reading but
  76may be advisable for portability.
  77
  78Support is provided for changing the tape partition and partitioning
  79of the tape with one or two partitions. By default support for
  80partitioned tape is disabled for each driver and it can be enabled
  81with the ioctl MTSETDRVBUFFER.
  82
  83By default the driver writes one filemark when the device is closed after
  84writing and the last operation has been a write. Two filemarks can be
  85optionally written. In both cases end of data is signified by
  86returning zero bytes for two consecutive reads.
  87
  88Writing filemarks without the immediate bit set in the SCSI command block acts
  89as a synchronization point, i.e., all remaining data form the drive buffers is
  90written to tape before the command returns. This makes sure that write errors
  91are caught at that point, but this takes time. In some applications, several
  92consecutive files must be written fast. The MTWEOFI operation can be used to
  93write the filemarks without flushing the drive buffer. Writing filemark at
  94close() is always flushing the drive buffers. However, if the previous
  95operation is MTWEOFI, close() does not write a filemark. This can be used if
  96the program wants to close/open the tape device between files and wants to
  97skip waiting.
  98
  99If rewind, offline, bsf, or seek is done and previous tape operation was
 100write, a filemark is written before moving tape.
 101
 102The compile options are defined in the file linux/drivers/scsi/st_options.h.
 103
 1044. If the open option O_NONBLOCK is used, open succeeds even if the
 105drive is not ready. If O_NONBLOCK is not used, the driver waits for
 106the drive to become ready. If this does not happen in ST_BLOCK_SECONDS
 107seconds, open fails with the errno value EIO. With O_NONBLOCK the
 108device can be opened for writing even if there is a write protected
 109tape in the drive (commands trying to write something return error if
 110attempted).
 111
 112
 113MINOR NUMBERS
 114
 115The tape driver currently supports up to 2^17 drives if 4 modes for
 116each drive are used.
 117
 118The minor numbers consist of the following bit fields:
 119
 120dev_upper non-rew mode dev-lower
 121  20 -  8     7    6 5  4      0
 122The non-rewind bit is always bit 7 (the uppermost bit in the lowermost
 123byte). The bits defining the mode are below the non-rewind bit. The
 124remaining bits define the tape device number. This numbering is
 125backward compatible with the numbering used when the minor number was
 126only 8 bits wide.
 127
 128
 129SYSFS SUPPORT
 130
 131The driver creates the directory /sys/class/scsi_tape and populates it with
 132directories corresponding to the existing tape devices. There are autorewind
 133and non-rewind entries for each mode. The names are stxy and nstxy, where x
 134is the tape number and y a character corresponding to the mode (none, l, m,
 135a). For example, the directories for the first tape device are (assuming four
 136modes): st0  nst0  st0l  nst0l  st0m  nst0m  st0a  nst0a.
 137
 138Each directory contains the entries: default_blksize  default_compression
 139default_density  defined  dev  device  driver. The file 'defined' contains 1
 140if the mode is defined and zero if not defined. The files 'default_*' contain
 141the defaults set by the user. The value -1 means the default is not set. The
 142file 'dev' contains the device numbers corresponding to this device. The links
 143'device' and 'driver' point to the SCSI device and driver entries.
 144
 145Each directory also contains the entry 'options' which shows the currently
 146enabled driver and mode options. The value in the file is a bit mask where the
 147bit definitions are the same as those used with MTSETDRVBUFFER in setting the
 148options.
 149
 150A link named 'tape' is made from the SCSI device directory to the class
 151directory corresponding to the mode 0 auto-rewind device (e.g., st0). 
 152
 153
 154BSD AND SYS V SEMANTICS
 155
 156The user can choose between these two behaviours of the tape driver by
 157defining the value of the symbol ST_SYSV. The semantics differ when a
 158file being read is closed. The BSD semantics leaves the tape where it
 159currently is whereas the SYS V semantics moves the tape past the next
 160filemark unless the filemark has just been crossed.
 161
 162The default is BSD semantics.
 163
 164
 165BUFFERING
 166
 167The driver tries to do transfers directly to/from user space. If this
 168is not possible, a driver buffer allocated at run-time is used. If
 169direct i/o is not possible for the whole transfer, the driver buffer
 170is used (i.e., bounce buffers for individual pages are not
 171used). Direct i/o can be impossible because of several reasons, e.g.:
 172- one or more pages are at addresses not reachable by the HBA
 173- the number of pages in the transfer exceeds the number of
 174  scatter/gather segments permitted by the HBA
 175- one or more pages can't be locked into memory (should not happen in
 176  any reasonable situation)
 177
 178The size of the driver buffers is always at least one tape block. In fixed
 179block mode, the minimum buffer size is defined (in 1024 byte units) by
 180ST_FIXED_BUFFER_BLOCKS. With small block size this allows buffering of
 181several blocks and using one SCSI read or write to transfer all of the
 182blocks. Buffering of data across write calls in fixed block mode is
 183allowed if ST_BUFFER_WRITES is non-zero and direct i/o is not used.
 184Buffer allocation uses chunks of memory having sizes 2^n * (page
 185size). Because of this the actual buffer size may be larger than the
 186minimum allowable buffer size.
 187
 188NOTE that if direct i/o is used, the small writes are not buffered. This may
 189cause a surprise when moving from 2.4. There small writes (e.g., tar without
 190-b option) may have had good throughput but this is not true any more with
 1912.6. Direct i/o can be turned off to solve this problem but a better solution
 192is to use bigger write() byte counts (e.g., tar -b 64).
 193
 194Asynchronous writing. Writing the buffer contents to the tape is
 195started and the write call returns immediately. The status is checked
 196at the next tape operation. Asynchronous writes are not done with
 197direct i/o and not in fixed block mode.
 198
 199Buffered writes and asynchronous writes may in some rare cases cause
 200problems in multivolume operations if there is not enough space on the
 201tape after the early-warning mark to flush the driver buffer.
 202
 203Read ahead for fixed block mode (ST_READ_AHEAD). Filling the buffer is
 204attempted even if the user does not want to get all of the data at
 205this read command. Should be disabled for those drives that don't like
 206a filemark to truncate a read request or that don't like backspacing.
 207
 208Scatter/gather buffers (buffers that consist of chunks non-contiguous
 209in the physical memory) are used if contiguous buffers can't be
 210allocated. To support all SCSI adapters (including those not
 211supporting scatter/gather), buffer allocation is using the following
 212three kinds of chunks:
 2131. The initial segment that is used for all SCSI adapters including
 214those not supporting scatter/gather. The size of this buffer will be
 215(PAGE_SIZE << ST_FIRST_ORDER) bytes if the system can give a chunk of
 216this size (and it is not larger than the buffer size specified by
 217ST_BUFFER_BLOCKS). If this size is not available, the driver halves
 218the size and tries again until the size of one page. The default
 219settings in st_options.h make the driver to try to allocate all of the
 220buffer as one chunk.
 2212. The scatter/gather segments to fill the specified buffer size are
 222allocated so that as many segments as possible are used but the number
 223of segments does not exceed ST_FIRST_SG.
 2243. The remaining segments between ST_MAX_SG (or the module parameter
 225max_sg_segs) and the number of segments used in phases 1 and 2
 226are used to extend the buffer at run-time if this is necessary. The
 227number of scatter/gather segments allowed for the SCSI adapter is not
 228exceeded if it is smaller than the maximum number of scatter/gather
 229segments specified. If the maximum number allowed for the SCSI adapter
 230is smaller than the number of segments used in phases 1 and 2,
 231extending the buffer will always fail.
 232
 233
 234EOM BEHAVIOUR WHEN WRITING
 235
 236When the end of medium early warning is encountered, the current write
 237is finished and the number of bytes is returned. The next write
 238returns -1 and errno is set to ENOSPC. To enable writing a trailer,
 239the next write is allowed to proceed and, if successful, the number of
 240bytes is returned. After this, -1 and the number of bytes are
 241alternately returned until the physical end of medium (or some other
 242error) is encountered.
 243
 244
 245MODULE PARAMETERS
 246
 247The buffer size, write threshold, and the maximum number of allocated buffers
 248are configurable when the driver is loaded as a module. The keywords are:
 249
 250buffer_kbs=xxx             the buffer size for fixed block mode is set
 251                           to xxx kilobytes
 252write_threshold_kbs=xxx    the write threshold in kilobytes set to xxx
 253max_sg_segs=xxx            the maximum number of scatter/gather
 254                           segments
 255try_direct_io=x            try direct transfer between user buffer and
 256                           tape drive if this is non-zero
 257
 258Note that if the buffer size is changed but the write threshold is not
 259set, the write threshold is set to the new buffer size - 2 kB.
 260
 261
 262BOOT TIME CONFIGURATION
 263
 264If the driver is compiled into the kernel, the same parameters can be
 265also set using, e.g., the LILO command line. The preferred syntax is
 266to use the same keyword used when loading as module but prepended
 267with 'st.'. For instance, to set the maximum number of scatter/gather
 268segments, the parameter 'st.max_sg_segs=xx' should be used (xx is the
 269number of scatter/gather segments).
 270
 271For compatibility, the old syntax from early 2.5 and 2.4 kernel
 272versions is supported. The same keywords can be used as when loading
 273the driver as module. If several parameters are set, the keyword-value
 274pairs are separated with a comma (no spaces allowed). A colon can be
 275used instead of the equal mark. The definition is prepended by the
 276string st=. Here is an example:
 277
 278        st=buffer_kbs:64,write_threshold_kbs:60
 279
 280The following syntax used by the old kernel versions is also supported:
 281
 282           st=aa[,bb[,dd]]
 283
 284where
 285  aa is the buffer size for fixed block mode in 1024 byte units
 286  bb is the write threshold in 1024 byte units
 287  dd is the maximum number of scatter/gather segments
 288
 289
 290IOCTLS
 291
 292The tape is positioned and the drive parameters are set with ioctls
 293defined in mtio.h The tape control program 'mt' uses these ioctls. Try
 294to find an mt that supports all of the Linux SCSI tape ioctls and
 295opens the device for writing if the tape contents will be modified
 296(look for a package mt-st* from the Linux ftp sites; the GNU mt does
 297not open for writing for, e.g., erase).
 298
 299The supported ioctls are:
 300
 301The following use the structure mtop:
 302
 303MTFSF   Space forward over count filemarks. Tape positioned after filemark.
 304MTFSFM  As above but tape positioned before filemark.
 305MTBSF   Space backward over count filemarks. Tape positioned before
 306        filemark.
 307MTBSFM  As above but ape positioned after filemark.
 308MTFSR   Space forward over count records.
 309MTBSR   Space backward over count records.
 310MTFSS   Space forward over count setmarks.
 311MTBSS   Space backward over count setmarks.
 312MTWEOF  Write count filemarks.
 313MTWEOFI Write count filemarks with immediate bit set (i.e., does not
 314        wait until data is on tape)
 315MTWSM   Write count setmarks.
 316MTREW   Rewind tape.
 317MTOFFL  Set device off line (often rewind plus eject).
 318MTNOP   Do nothing except flush the buffers.
 319MTRETEN Re-tension tape.
 320MTEOM   Space to end of recorded data.
 321MTERASE Erase tape. If the argument is zero, the short erase command
 322        is used. The long erase command is used with all other values
 323        of the argument.
 324MTSEEK  Seek to tape block count. Uses Tandberg-compatible seek (QFA)
 325        for SCSI-1 drives and SCSI-2 seek for SCSI-2 drives. The file and
 326        block numbers in the status are not valid after a seek.
 327MTSETBLK Set the drive block size. Setting to zero sets the drive into
 328        variable block mode (if applicable).
 329MTSETDENSITY Sets the drive density code to arg. See drive
 330        documentation for available codes.
 331MTLOCK and MTUNLOCK Explicitly lock/unlock the tape drive door.
 332MTLOAD and MTUNLOAD Explicitly load and unload the tape. If the
 333        command argument x is between MT_ST_HPLOADER_OFFSET + 1 and
 334        MT_ST_HPLOADER_OFFSET + 6, the number x is used sent to the
 335        drive with the command and it selects the tape slot to use of
 336        HP C1553A changer.
 337MTCOMPRESSION Sets compressing or uncompressing drive mode using the
 338        SCSI mode page 15. Note that some drives other methods for
 339        control of compression. Some drives (like the Exabytes) use
 340        density codes for compression control. Some drives use another
 341        mode page but this page has not been implemented in the
 342        driver. Some drives without compression capability will accept
 343        any compression mode without error.
 344MTSETPART Moves the tape to the partition given by the argument at the
 345        next tape operation. The block at which the tape is positioned
 346        is the block where the tape was previously positioned in the
 347        new active partition unless the next tape operation is
 348        MTSEEK. In this case the tape is moved directly to the block
 349        specified by MTSEEK. MTSETPART is inactive unless
 350        MT_ST_CAN_PARTITIONS set.
 351MTMKPART Formats the tape with one partition (argument zero) or two
 352        partitions (the argument gives in megabytes the size of
 353        partition 1 that is physically the first partition of the
 354        tape). The drive has to support partitions with size specified
 355        by the initiator. Inactive unless MT_ST_CAN_PARTITIONS set.
 356MTSETDRVBUFFER
 357        Is used for several purposes. The command is obtained from count
 358        with mask MT_SET_OPTIONS, the low order bits are used as argument.
 359        This command is only allowed for the superuser (root). The
 360        subcommands are:
 361        0
 362           The drive buffer option is set to the argument. Zero means
 363           no buffering.
 364        MT_ST_BOOLEANS
 365           Sets the buffering options. The bits are the new states
 366           (enabled/disabled) the following options (in the
 367           parenthesis is specified whether the option is global or
 368           can be specified differently for each mode):
 369             MT_ST_BUFFER_WRITES write buffering (mode)
 370             MT_ST_ASYNC_WRITES asynchronous writes (mode)
 371             MT_ST_READ_AHEAD  read ahead (mode)
 372             MT_ST_TWO_FM writing of two filemarks (global)
 373             MT_ST_FAST_EOM using the SCSI spacing to EOD (global)
 374             MT_ST_AUTO_LOCK automatic locking of the drive door (global)
 375             MT_ST_DEF_WRITES the defaults are meant only for writes (mode)
 376             MT_ST_CAN_BSR backspacing over more than one records can
 377                be used for repositioning the tape (global)
 378             MT_ST_NO_BLKLIMS the driver does not ask the block limits
 379                from the drive (block size can be changed only to
 380                variable) (global)
 381             MT_ST_CAN_PARTITIONS enables support for partitioned
 382                tapes (global)
 383             MT_ST_SCSI2LOGICAL the logical block number is used in
 384                the MTSEEK and MTIOCPOS for SCSI-2 drives instead of
 385                the device dependent address. It is recommended to set
 386                this flag unless there are tapes using the device
 387                dependent (from the old times) (global)
 388             MT_ST_SYSV sets the SYSV semantics (mode)
 389             MT_ST_NOWAIT enables immediate mode (i.e., don't wait for
 390                the command to finish) for some commands (e.g., rewind)
 391             MT_ST_NOWAIT_EOF enables immediate filemark mode (i.e. when
 392                writing a filemark, don't wait for it to complete). Please
 393                see the BASICS note about MTWEOFI with respect to the
 394                possible dangers of writing immediate filemarks.
 395             MT_ST_SILI enables setting the SILI bit in SCSI commands when
 396                reading in variable block mode to enhance performance when
 397                reading blocks shorter than the byte count; set this only
 398                if you are sure that the drive supports SILI and the HBA
 399                correctly returns transfer residuals
 400             MT_ST_DEBUGGING debugging (global; debugging must be
 401                compiled into the driver)
 402        MT_ST_SETBOOLEANS
 403        MT_ST_CLEARBOOLEANS
 404           Sets or clears the option bits.
 405        MT_ST_WRITE_THRESHOLD
 406           Sets the write threshold for this device to kilobytes
 407           specified by the lowest bits.
 408        MT_ST_DEF_BLKSIZE
 409           Defines the default block size set automatically. Value
 410           0xffffff means that the default is not used any more.
 411        MT_ST_DEF_DENSITY
 412        MT_ST_DEF_DRVBUFFER
 413           Used to set or clear the density (8 bits), and drive buffer
 414           state (3 bits). If the value is MT_ST_CLEAR_DEFAULT
 415           (0xfffff) the default will not be used any more. Otherwise
 416           the lowermost bits of the value contain the new value of
 417           the parameter.
 418        MT_ST_DEF_COMPRESSION
 419           The compression default will not be used if the value of
 420           the lowermost byte is 0xff. Otherwise the lowermost bit
 421           contains the new default. If the bits 8-15 are set to a
 422           non-zero number, and this number is not 0xff, the number is
 423           used as the compression algorithm. The value
 424           MT_ST_CLEAR_DEFAULT can be used to clear the compression
 425           default.
 426        MT_ST_SET_TIMEOUT
 427           Set the normal timeout in seconds for this device. The
 428           default is 900 seconds (15 minutes). The timeout should be
 429           long enough for the retries done by the device while
 430           reading/writing.
 431        MT_ST_SET_LONG_TIMEOUT
 432           Set the long timeout that is used for operations that are
 433           known to take a long time. The default is 14000 seconds
 434           (3.9 hours). For erase this value is further multiplied by
 435           eight.
 436        MT_ST_SET_CLN
 437           Set the cleaning request interpretation parameters using
 438           the lowest 24 bits of the argument. The driver can set the
 439           generic status bit GMT_CLN if a cleaning request bit pattern
 440           is found from the extended sense data. Many drives set one or
 441           more bits in the extended sense data when the drive needs
 442           cleaning. The bits are device-dependent. The driver is
 443           given the number of the sense data byte (the lowest eight
 444           bits of the argument; must be >= 18 (values 1 - 17
 445           reserved) and <= the maximum requested sense data sixe), 
 446           a mask to select the relevant bits (the bits 9-16), and the
 447           bit pattern (bits 17-23). If the bit pattern is zero, one
 448           or more bits under the mask indicate cleaning request. If
 449           the pattern is non-zero, the pattern must match the masked
 450           sense data byte.
 451
 452           (The cleaning bit is set if the additional sense code and
 453           qualifier 00h 17h are seen regardless of the setting of
 454           MT_ST_SET_CLN.)
 455
 456The following ioctl uses the structure mtpos:
 457MTIOCPOS Reads the current position from the drive. Uses
 458        Tandberg-compatible QFA for SCSI-1 drives and the SCSI-2
 459        command for the SCSI-2 drives.
 460
 461The following ioctl uses the structure mtget to return the status:
 462MTIOCGET Returns some status information.
 463        The file number and block number within file are returned. The
 464        block is -1 when it can't be determined (e.g., after MTBSF).
 465        The drive type is either MTISSCSI1 or MTISSCSI2.
 466        The number of recovered errors since the previous status call
 467        is stored in the lower word of the field mt_erreg.
 468        The current block size and the density code are stored in the field
 469        mt_dsreg (shifts for the subfields are MT_ST_BLKSIZE_SHIFT and
 470        MT_ST_DENSITY_SHIFT).
 471        The GMT_xxx status bits reflect the drive status. GMT_DR_OPEN
 472        is set if there is no tape in the drive. GMT_EOD means either
 473        end of recorded data or end of tape. GMT_EOT means end of tape.
 474
 475
 476MISCELLANEOUS COMPILE OPTIONS
 477
 478The recovered write errors are considered fatal if ST_RECOVERED_WRITE_FATAL
 479is defined.
 480
 481The maximum number of tape devices is determined by the define
 482ST_MAX_TAPES. If more tapes are detected at driver initialization, the
 483maximum is adjusted accordingly.
 484
 485Immediate return from tape positioning SCSI commands can be enabled by
 486defining ST_NOWAIT. If this is defined, the user should take care that
 487the next tape operation is not started before the previous one has
 488finished. The drives and SCSI adapters should handle this condition
 489gracefully, but some drive/adapter combinations are known to hang the
 490SCSI bus in this case.
 491
 492The MTEOM command is by default implemented as spacing over 32767
 493filemarks. With this method the file number in the status is
 494correct. The user can request using direct spacing to EOD by setting
 495ST_FAST_EOM 1 (or using the MT_ST_OPTIONS ioctl). In this case the file
 496number will be invalid.
 497
 498When using read ahead or buffered writes the position within the file
 499may not be correct after the file is closed (correct position may
 500require backspacing over more than one record). The correct position
 501within file can be obtained if ST_IN_FILE_POS is defined at compile
 502time or the MT_ST_CAN_BSR bit is set for the drive with an ioctl.
 503(The driver always backs over a filemark crossed by read ahead if the
 504user does not request data that far.)
 505
 506
 507DEBUGGING HINTS
 508
 509To enable debugging messages, edit st.c and #define DEBUG 1. As seen
 510above, debugging can be switched off with an ioctl if debugging is
 511compiled into the driver. The debugging output is not voluminous.
 512
 513If the tape seems to hang, I would be very interested to hear where
 514the driver is waiting. With the command 'ps -l' you can see the state
 515of the process using the tape. If the state is D, the process is
 516waiting for something. The field WCHAN tells where the driver is
 517waiting. If you have the current System.map in the correct place (in
 518/boot for the procps I use) or have updated /etc/psdatabase (for kmem
 519ps), ps writes the function name in the WCHAN field. If not, you have
 520to look up the function from System.map.
 521
 522Note also that the timeouts are very long compared to most other
 523drivers. This means that the Linux driver may appear hung although the
 524real reason is that the tape firmware has got confused.
 525
lxr.linux.no kindly hosted by Redpill Linpro AS, provider of Linux consulting and operations services since 1995.