1<!-- Creator : groff version 1.19.2 --> 2<!-- CreationDate: Sun Mar 14 19:50:28 2010 --> 3<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" 4"http://www.w3.org/TR/html4/loose.dtd"> 5<html> 6<head> 7<meta name="generator" content="groff -Thtml, see www.gnu.org"> 8<meta http-equiv="Content-Type" content="text/html; charset=US-ASCII"> 9<meta name="Content-Style" content="text/css"> 10<style type="text/css"> 11 p { margin-top: 0; margin-bottom: 0; } 12 pre { margin-top: 0; margin-bottom: 0; } 13 table { margin-top: 0; margin-bottom: 0; } 14</style> 15<title></title> 16</head> 17<body> 18 19<hr> 20 21 22<p valign="top">libarchive-formats(5) FreeBSD File Formats 23Manual libarchive-formats(5)</p> 24 25<p style="margin-top: 1em" valign="top"><b>NAME</b></p> 26 27<p style="margin-left:8%;"><b>libarchive-formats</b> 28— archive formats supported by the libarchive 29library</p> 30 31 32<p style="margin-top: 1em" valign="top"><b>DESCRIPTION</b></p> 33 34<p style="margin-left:8%;">The libarchive(3) library reads 35and writes a variety of streaming archive formats. Generally 36speaking, all of these archive formats consist of a series 37of ‘‘entries’’. Each entry stores a 38single file system object, such as a file, directory, or 39symbolic link.</p> 40 41<p style="margin-left:8%; margin-top: 1em">The following 42provides a brief description of each format supported by 43libarchive, with some information about recognized 44extensions or limitations of the current library support. 45Note that just because a format is supported by libarchive 46does not imply that a program that uses libarchive will 47support that format. Applications that use libarchive 48specify which formats they wish to support, though many 49programs do use libarchive convenience functions to enable 50all supported formats.</p> 51 52<p style="margin-left:8%; margin-top: 1em"><b>Tar 53Formats</b> <br> 54The libarchive(3) library can read most tar archives. 55However, it only writes POSIX-standard 56‘‘ustar’’ and ‘‘pax 57interchange’’ formats.</p> 58 59<p style="margin-left:8%; margin-top: 1em">All tar formats 60store each entry in one or more 512-byte records. The first 61record is used for file metadata, including filename, 62timestamp, and mode information, and the file data is stored 63in subsequent records. Later variants have extended this by 64either appropriating undefined areas of the header record, 65extending the header to multiple records, or by storing 66special entries that modify the interpretation of subsequent 67entries.</p> 68 69<p style="margin-top: 1em" valign="top"><b>gnutar</b></p> 70 71<p style="margin-left:20%; margin-top: 1em">The 72libarchive(3) library can read GNU-format tar archives. It 73currently supports the most popular GNU extensions, 74including modern long filename and linkname support, as well 75as atime and ctime data. The libarchive library does not 76support multi-volume archives, nor the old GNU long filename 77format. It can read GNU sparse file entries, including the 78new POSIX-based formats, but cannot write GNU sparse file 79entries.</p> 80 81<p style="margin-top: 1em" valign="top"><b>pax</b></p> 82 83<p style="margin-left:20%; margin-top: 1em">The 84libarchive(3) library can read and write POSIX-compliant pax 85interchange format archives. Pax interchange format archives 86are an extension of the older ustar format that adds a 87separate entry with additional attributes stored as 88key/value pairs immediately before each regular entry. The 89presence of these additional entries is the only difference 90between pax interchange format and the older ustar format. 91The extended attributes are of unlimited length and are 92stored as UTF-8 Unicode strings. Keywords defined in the 93standard are in all lowercase; vendors are allowed to define 94custom keys by preceding them with the vendor name in all 95uppercase. When writing pax archives, libarchive uses many 96of the SCHILY keys defined by Joerg Schilling’s 97‘‘star’’ archiver and a few 98LIBARCHIVE keys. The libarchive library can read most of the 99SCHILY keys and most of the GNU keys introduced by GNU tar. 100It silently ignores any keywords that it does not 101understand.</p> 102 103<p style="margin-top: 1em" valign="top"><b>restricted 104pax</b></p> 105 106<p style="margin-left:20%;">The libarchive library can also 107write pax archives in which it attempts to suppress the 108extended attributes entry whenever possible. The result will 109be identical to a ustar archive unless the extended 110attributes entry is required to store a long file name, long 111linkname, extended ACL, file flags, or if any of the 112standard ustar data (user name, group name, UID, GID, etc) 113cannot be fully represented in the ustar header. In all 114cases, the result can be dearchived by any program that can 115read POSIX-compliant pax interchange format archives. 116Programs that correctly read ustar format (see below) will 117also be able to read this format; any extended attributes 118will be extracted as separate files stored in 119<i>PaxHeader</i> directories.</p> 120 121<p style="margin-top: 1em" valign="top"><b>ustar</b></p> 122 123<p style="margin-left:20%; margin-top: 1em">The libarchive 124library can both read and write this format. This format has 125the following limitations:</p> 126 127<p valign="top"><b>•</b></p> 128 129<p style="margin-left:26%;">Device major and minor numbers 130are limited to 21 bits. Nodes with larger numbers will not 131be added to the archive.</p> 132 133<p valign="top"><b>•</b></p> 134 135<p style="margin-left:26%;">Path names in the archive are 136limited to 255 bytes. (Shorter if there is no / character in 137exactly the right place.)</p> 138 139<p valign="top"><b>•</b></p> 140 141<p style="margin-left:26%;">Symbolic links and hard links 142are stored in the archive with the name of the referenced 143file. This name is limited to 100 bytes.</p> 144 145<p valign="top"><b>•</b></p> 146 147<p style="margin-left:26%;">Extended attributes, file 148flags, and other extended security information cannot be 149stored.</p> 150 151<p valign="top"><b>•</b></p> 152 153<p style="margin-left:26%;">Archive entries are limited to 1548 gigabytes in size.</p> 155 156<p style="margin-left:20%;">Note that the pax interchange 157format has none of these restrictions.</p> 158 159<p style="margin-left:8%; margin-top: 1em">The libarchive 160library also reads a variety of commonly-used extensions to 161the basic tar format. These extensions are recognized 162automatically whenever they appear.</p> 163 164<p style="margin-top: 1em" valign="top">Numeric 165extensions.</p> 166 167<p style="margin-left:20%;">The POSIX standards require 168fixed-length numeric fields to be written with some 169character position reserved for terminators. Libarchive 170allows these fields to be written without terminator 171characters. This extends the allowable range; in particular, 172ustar archives with this extension can support entries up to 17364 gigabytes in size. Libarchive also recognizes base-256 174values in most numeric fields. This essentially removes all 175limitations on file size, modification time, and device 176numbers.</p> 177 178<p style="margin-top: 1em" valign="top">Solaris 179extensions</p> 180 181<p style="margin-left:20%;">Libarchive recognizes ACL and 182extended attribute records written by Solaris tar. 183Currently, libarchive only has support for old-style ACLs; 184the newer NFSv4 ACLs are recognized but discarded.</p> 185 186<p style="margin-left:8%; margin-top: 1em">The first tar 187program appeared in Seventh Edition Unix in 1979. The first 188official standard for the tar file format was the 189‘‘ustar’’ (Unix Standard Tar) format 190defined by POSIX in 1988. POSIX.1-2001 extended the ustar 191format to create the ‘‘pax 192interchange’’ format.</p> 193 194<p style="margin-left:8%; margin-top: 1em"><b>Cpio 195Formats</b> <br> 196The libarchive library can read a number of common cpio 197variants and can write ‘‘odc’’ and 198‘‘newc’’ format archives. A cpio 199archive stores each entry as a fixed-size header followed by 200a variable-length filename and variable-length data. Unlike 201the tar format, the cpio format does only minimal padding of 202the header or file data. There are several cpio variants, 203which differ primarily in how they store the initial header: 204some store the values as octal or hexadecimal numbers in 205ASCII, others as binary values of varying byte order and 206length.</p> 207 208<p style="margin-top: 1em" valign="top"><b>binary</b></p> 209 210<p style="margin-left:20%; margin-top: 1em">The libarchive 211library transparently reads both big-endian and 212little-endian variants of the original binary cpio format. 213This format used 32-bit binary values for file size and 214mtime, and 16-bit binary values for the other fields.</p> 215 216<p style="margin-top: 1em" valign="top"><b>odc</b></p> 217 218<p style="margin-left:20%; margin-top: 1em">The libarchive 219library can both read and write this POSIX-standard format, 220which is officially known as the ‘‘cpio 221interchange format’’ or the 222‘‘octet-oriented cpio archive 223format’’ and sometimes unofficially referred to 224as the ‘‘old character format’’. 225This format stores the header contents as octal values in 226ASCII. It is standard, portable, and immune from byte-order 227confusion. File sizes and mtime are limited to 33 bits (8GB 228file size), other fields are limited to 18 bits.</p> 229 230<p style="margin-top: 1em" valign="top"><b>SVR4</b></p> 231 232<p style="margin-left:20%; margin-top: 1em">The libarchive 233library can read both CRC and non-CRC variants of this 234format. The SVR4 format uses eight-digit hexadecimal values 235for all header fields. This limits file size to 4GB, and 236also limits the mtime and other fields to 32 bits. The SVR4 237format can optionally include a CRC of the file contents, 238although libarchive does not currently verify this CRC.</p> 239 240<p style="margin-left:8%; margin-top: 1em">Cpio first 241appeared in PWB/UNIX 1.0, which was released within AT&T 242in 1977. PWB/UNIX 1.0 formed the basis of System III Unix, 243released outside of AT&T in 1981. This makes cpio older 244than tar, although cpio was not included in Version 7 245AT&T Unix. As a result, the tar command became much 246better known in universities and research groups that used 247Version 7. The combination of the <b>find</b> and 248<b>cpio</b> utilities provided very precise control over 249file selection. Unfortunately, the format has many 250limitations that make it unsuitable for widespread use. Only 251the POSIX format permits files over 4GB, and its 18-bit 252limit for most other fields makes it unsuitable for modern 253systems. In addition, cpio formats only store numeric 254UID/GID values (not usernames and group names), which can 255make it very difficult to correctly transfer archives across 256systems with dissimilar user numbering.</p> 257 258<p style="margin-left:8%; margin-top: 1em"><b>Shar 259Formats</b> <br> 260A ‘‘shell archive’’ is a shell 261script that, when executed on a POSIX-compliant system, will 262recreate a collection of file system objects. The libarchive 263library can write two different kinds of shar archives:</p> 264 265<p style="margin-top: 1em" valign="top"><b>shar</b></p> 266 267<p style="margin-left:20%; margin-top: 1em">The traditional 268shar format uses a limited set of POSIX commands, including 269echo(1), mkdir(1), and sed(1). It is suitable for portably 270archiving small collections of plain text files. However, it 271is not generally well-suited for large archives (many 272implementations of sh(1) have limits on the size of a 273script) nor should it be used with non-text files.</p> 274 275 276<p style="margin-top: 1em" valign="top"><b>shardump</b></p> 277 278<p style="margin-left:20%;">This format is similar to shar 279but encodes files using uuencode(1) so that the result will 280be a plain text file regardless of the file contents. It 281also includes additional shell commands that attempt to 282reproduce as many file attributes as possible, including 283owner, mode, and flags. The additional commands used to 284restore file attributes make shardump archives less portable 285than plain shar archives.</p> 286 287<p style="margin-left:8%; margin-top: 1em"><b>ISO9660 288format</b> <br> 289Libarchive can read and extract from files containing 290ISO9660-compliant CDROM images. In many cases, this can 291remove the need to burn a physical CDROM just in order to 292read the files contained in an ISO9660 image. It also avoids 293security and complexity issues that come with virtual mounts 294and loopback devices. Libarchive supports the most common 295Rockridge extensions and has partial support for Joliet 296extensions. If both extensions are present, the Joliet 297extensions will be used and the Rockridge extensions will be 298ignored. In particular, this can create problems with 299hardlinks and symlinks, which are supported by Rockridge but 300not by Joliet.</p> 301 302<p style="margin-left:8%; margin-top: 1em"><b>Zip 303format</b> <br> 304Libarchive can read and write zip format archives that have 305uncompressed entries and entries compressed with the 306‘‘deflate’’ algorithm. Older zip 307compression algorithms are not supported. It can extract jar 308archives, archives that use Zip64 extensions and many 309self-extracting zip archives. Libarchive reads Zip archives 310as they are being streamed, which allows it to read archives 311of arbitrary size. It currently does not use the central 312directory; this limits libarchive’s ability to support 313some self-extracting archives and ones that have been 314modified in certain ways.</p> 315 316<p style="margin-left:8%; margin-top: 1em"><b>Archive 317(library) file format</b> <br> 318The Unix archive format (commonly created by the ar(1) 319archiver) is a general-purpose format which is used almost 320exclusively for object files to be read by the link editor 321ld(1). The ar format has never been standardised. There are 322two common variants: the GNU format derived from SVR4, and 323the BSD format, which first appeared in 4.4BSD. The two 324differ primarily in their handling of filenames longer than 32515 characters: the GNU/SVR4 variant writes a filename table 326at the beginning of the archive; the BSD format stores each 327long filename in an extension area adjacent to the entry. 328Libarchive can read both extensions, including archives that 329may include both types of long filenames. Programs using 330libarchive can write GNU/SVR4 format if they provide a 331filename table to be written into the archive before any of 332the entries. Any entries whose names are not in the filename 333table will be written using BSD-style long filenames. This 334can cause problems for programs such as GNU ld that do not 335support the BSD-style long filenames.</p> 336 337<p style="margin-left:8%; margin-top: 1em"><b>mtree</b> 338<br> 339Libarchive can read and write files in mtree(5) format. This 340format is not a true archive format, but rather a textual 341description of a file hierarchy in which each line specifies 342the name of a file and provides specific metadata about that 343file. Libarchive can read all of the keywords supported by 344both the NetBSD and FreeBSD versions of mtree(1), although 345many of the keywords cannot currently be stored in an 346archive_entry object. When writing, libarchive supports use 347of the archive_write_set_options(3) interface to specify 348which keywords should be included in the output. If 349libarchive was compiled with access to suitable 350cryptographic libraries (such as the OpenSSL libraries), it 351can compute hash entries such as <b>sha512</b> or <b>md5</b> 352from file data being written to the mtree writer.</p> 353 354<p style="margin-left:8%; margin-top: 1em">When reading an 355mtree file, libarchive will locate the corresponding files 356on disk using the <b>contents</b> keyword if present or the 357regular filename. If it can locate and open the file on 358disk, it will use that to fill in any metadata that is 359missing from the mtree file and will read the file contents 360and return those to the program using libarchive. If it 361cannot locate and open the file on disk, libarchive will 362return an error for any attempt to read the entry body.</p> 363 364<p style="margin-top: 1em" valign="top"><b>SEE ALSO</b></p> 365 366<p style="margin-left:8%;">ar(1), cpio(1), mkisofs(1), 367shar(1), tar(1), zip(1), zlib(3), cpio(5), mtree(5), 368tar(5)</p> 369 370 371<p style="margin-left:8%; margin-top: 1em">FreeBSD 9.0 372December 27, 2009 FreeBSD 9.0</p> 373<hr> 374</body> 375</html> 376