bsdcat,cpio,tar: derive version string from archive.h

(cherry picked from commit 2ff66a915526b6509ca57f2046bc7b20b0de8a61)

Git Hash: 7e3d37d73d382c6d36e4dcb36de79ac3f233be3e
Git Author: delphij@FreeBSD.org

MFC r368207,368607:

MFC r368207:
Update libarchive to 3.5.0

Relevant vendor changes:
Issue #1258: add archive_read_support_filter_by_code()
PR #1347: mtree digest reader support
Issue #1381: skip hardlinks pointing to itself on extraction
PR #1387: fix writing of cpio archives with hardlinks without file type
PR #1388: fix rdev field in cpio format for device nodes
PR #1389: completed support for UTF-8 encoding conversion
PR #1405: more formats in archive_read_support_format_by_code()
PR #1408: fix uninitialized size in rar5_read_data
PR #1409: system extended attribute support
PR #1435: support for decompression of symbolic links in zipx archives
Issue #1456: memory leak after unsuccessful archive_write_open_filename

MFC r368607:
Sync libarchive with vendor.

Vendor changes:
Issue #1461: Unbreak build without lzma
Issue #1462: warc reader: Fix build with gcc11
Issue #1463: Fix code compatibility in test_archive_read_support.c
Issue #1464: Use built-in strnlen on platforms where not available
Issue #1465: warc reader: fix undefined behaviour in deconst() function

MFC r361294:
Update libarchive to 3.4.3

Relevant vendor changes:
PR #1352: support negative zstd compression levels
PR #1359: improve zstd version checking
PR #1348: support RHT.security.selinux from GNU tar
PR #1357: support for archives compressed with pzstd
PR #1367: fix issues in acl tests
PR #1372: child handling cleanup
PR #1378: fix memory leak from passphrase callback

Relnotes: yes

MFC r356212,r356366,r356416,r357785
Update libarchive to version 3.4.2

Relevant vendor changes (r356212):
Issue #351: Refactor and implement private state logic for write filters
PR #1252: RAR5 reader - verify window size for solid files (OSS-Fuzz 15482)
PR #1255: zip writer - don't append unused NUL for directories
PR #1260: Fix sparse file offset overflow on 32-bit systems
PR #1263: UNICODE filename support for reading lha/lzh format
Issue #1276: Bugfix and optimize archive_wstring_append_from_mbs()
PR #1288: Add the "xattrhdr" option to pax write options
PR #1295: 7z reader - fix reading archives with digests in PackInfo
PR #1296: RAR5 reader - verify window size for multivolume archives
PR #1297: ZIP reader - support LZMA_STREAM_END marker in 'lzma alone' files
Issue #1298: Fix a heap-buffer-overflow in archive_string_append_from_wcs()
OSS-Fuzz 19360, 19362: LHA reader - plug two memory leaks on error
Fix possible off-by-one when dealing with readlink(2)

Relevant vendor changes (r356366):
Issue #1302: Plug memory leak on failure of archive_write_client_open()

Relevant vendor changes (r356416):
Issue #1302: Re-do fix for archive_write_client_open()

Relevant vendor changes (r357785):
PR #1289: atomic extraction support (bsdtar -x --safe-writes)
PR #1308: big endian fix for UTF16 support in LHA reader
PR #1326: reject RAR5 files that declare invalid header flags
Issue #987: fix support 7z archive entries with Delta filter
Issue #1317: fix compression output buffer handling in XAR writer
Issue #1319: fix uname or gname longer than 32 characters in pax writer
Issue #1325: fix use after free when archiving hardlinks in ISO9660 or XAR
Use localtime_r() and gmtime_r() instead of localtime() and gmtime()

MFC r348993,349135:
Sync libarchive with vendor including security fixes

r348993:
- version bumped to 3.4.0
- check_symlinks_fsobj() without chdir() and fchdir()
- bsdtar.1 manpage fixes
- patches from OpenBSD to libarchive_fe/passphrase.c

r349135:
PR #1212: RAR5 reader - window_mask was not updated correctly
(OSS-Fuzz 15278)
OSS-Fuzz 15120: RAR reader - extend use after free bugfix

MFC r338600:
Update libarchive to 3.3.3

As all important changes have already been merged from libarchive git
this is just a version number bump, documentation update and some
polishing for cpio tests. Other source code changes are not relevant to
FreeBSD.

Relnotes: yes

MFC r320927,320931,320932:
Bump libarchive to 3.3.2

Vendor changes:
PR #901: don't depend on stdin in a testcase

Relnotes: yes

MFC r315170:
r315170 (by imp):

Adopt SRCTOP in usr.bin

Prefer ${SRCTOP}/foo over ${.CURDIR}/../../foo and ${SRCTOP}/usr.bin/foo
over ${.CURDIR}/../foo for paths in Makefiles.

Silence on: arch@ (twice)

MFC r314571:
Update libarchive to version 3.3.1 (and sync with latest vendor dist)

Notable vendor changes:
PR #501: improvements in ACL path handling
PR #724: fix hang when reading malformed cpio files
PR #864: fix out of bounds read with malformed GNU tar archives
Documentation, style, test suite improvements and typo fixes.

New options to bsdtar that enable or disable reading and/or writing of:
Access Control Lists (--acls, --no-acls)
Extended file flags (--fflags, --no-fflags)
Extended attributes (--xattrs, --no-xattrs)
Mac OS X metadata (Mac OS X only) (--mac-metadata, --no-mac-metadata)

MFC r307861:
Update libarchive to 3.2.2

Most of the post-3.2.1 fixes have already been merged. This update contains
just the version bump and some fixes to the test framework.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFV r302003,r302037,r302038,r302056:

Update libarchive to 3.2.1 (bugfix and security fix release)

List of vendor fixes:
- fix exploitable heap overflow vulnerability in Rar decompression
(vendor issue 719, CVE-2016-4302, TALOS-2016-0154)
- fix exploitable stack based buffer overflow vulnebarility in mtree
parse_device functionality (vendor PR 715, CVE-2016-4301, TALOS-2016-0153)
- fix exploitable heap overflow vulnerability in 7-zip read_SubStreamsInfo
(vendor issue 718, CVE-2016-4300, TALOS-2016-152)
- fix integer overflow when computing location of volume descriptor
(vendor issue 717)
- fix buffer overflow when reading a crafred rar archive (vendor issue 521)
- fix possible buffer overflow when reading ISO9660 archives on machines
where sizeof(int) < sizeof(size_t) (vendor issue 711)
- tar and cpio should fail if an input file named on the command line is
missing (vendor issue 708)
- fix incorrect writing of gnutar filenames that are exactly 512 bytes
long (vendor issue 682)
- allow tests to be run from paths that are equal or longer than 128
characters (vendor issue 657)
- add memory allocation errors in archive_entry_xattr.c (vendor PR 603)
- remove dead code in archive_entry_xattr_add_entry() (vendor PR 716)
- fix broken decryption of ZIP files (vendor issue 553)
- manpage style, typo and description fixes

Post-3.2.1 vendor fixes:
- fix typo in cpio version reporting (Vendor PR 725, 726)
- fix argument range of ctype functions in libarchive_fe/passphrase.c
- fix ctype use and avoid empty loop bodies in WARC reader

MFC after: 1 week
Security: CVE-2016-4300, CVE-2016-4301, CVE-2016-4302
Approved by: re (kib)

MFV r299425:

Update libarchive to 3.2.0

New features:
- new bsdcat command-line utility
- LZ4 compression (in src only via external utility from ports)
- Warc format support
- 'Raw' format writer
- Zip: Support archives >4GB, entries >4GB
- Zip: Support encrypting and decrypting entries
- Zip: Support experimental streaming extension
- Identify encrypted entries in several formats
- New --clear-nochange-flags option to bsdtar tries to remove noschg and
similar flags before deleting files
- New --ignore-zeros option to bsdtar to handle concatenated tar archives
- Use multi-threaded LZMA decompression if liblzma supports it
- Expose version info for libraries used by libarchive

Patched files (fixed compiler warnings):

contrib/libarchive/cat/bsdcat.c (vendor PR #702)
contrib/libarchive/cat/bsdcat.h (vendor PR #702)
contrib/libarchive/libarchive/archive_read_support_format_mtree.c (PR #701)
contrib/libarchive/libarchive_fe/err.c (vendor PR #703)

MFC after: 1 month
Relnotes: yes

Don't override LIB*DIR variables from src.libnames.mk.

In some cases switch to the LIB*SRCDIR value.

These recently were defined in r291327 and r291619.

Sponsored by: EMC / Isilon Storage Division

Integrate the tests from lib/libarchive, usr.bin/cpio, and usr.bin/tar in to
the FreeBSD test suite

functional_test.sh was ported from bin/sh/tests/functional_test.sh, as a
small wrapper around libarchive_test, bsdcpio_test, and bsdtar_test provided
by upstream.

A handful of testcases in lib/libarchive/tests have been disabled as they
were failing when run with kyua test (see BROKEN_TESTS in
lib/libarchive/tests/Makefile)

As a sidenote: this removes the check/test targets from the Makefiles as they
don't match the pattern used in the rest of the FreeBSD test suite.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Remove stray DEBUG_FLAGS=-g that snuck in with r184761.

Convert to usr.bin/ to LIBADD
Reduce overlinking

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

MFV r248590,248594:
Update libarchive to 3.1.2

Some of new features:
- support for lrzip and grzip compression
- support for writing tar v7 format
- b64encode and uuencode filters
- support for __MACOSX directory in Zip archives
- support for lzop compresion (external utility)

Update libarchive to 3.0.4

Update libarchive to 3.0.3

Some of new features:
- New readers: RAR, LHA/LZH, CAB reader, 7-Zip
- New writers: ISO9660, XAR
- Improvements to many formats, especially including ISO9660 and Zip
- Stackable write filters to write, e.g., tar.gz.uu in a single pass
- Exploit seekable input; new "seekable" Zip reader can exploit the Zip
Central Directory when it's available; the old "streamable" Zip reader
is still fully supported for cases where seeking is not possible.

Full release notes available at:
https://github.com/libarchive/libarchive/wiki/ReleaseNotes

Use contrib sources for building libarchive, tar and cpio.
Make "make test" fully operational.

MFC after: 2 weeks

Update libarchive, tar and cpio to version 2.8.5

The following additional vendor revisions are applied:

Revision 3740:
Use archive_clear_error() to clear the error markers.

Obtained from: http://code.google.com/p/libarchive
MFC after: 2 weeks

Correctly link bsdcpio and bsdtar against libmd and libcrpyto
by applying the change from r221472 (libarchive).

Reviewed by: kientzle
Approved by: re (kib)
MFC after: 3 days

Update bsdtar to 2.8.4
Use common code from lib/libarchive/libarchive_fe

Approved by: kientzle
MFC after: 2 weeks

Enable liblzma support in libarchive
Adjust dependencies for programs using libarchive
Add xz and linkage against liblzma to rescue system

Approved by: kientzle, delphij (mentor)
MFC after: 2 weeks

FreeBSD is now using bsdtar 2.8.3.

Fixed dependencies (make checkdpadd).

Merge a bunch of refactoring from Joerg Sonnenberger to
isolate common code used by tar and cpio (and useful to other
libarchive clients). The functions here are prefixed with
"lafe" (libarchive front-end) to indicate their use.

Refactor the siginfo/sigusr1 handling. The read/write reporting
is sufficiently different that it was simpler to just put separate
reporting functions into read.c and write.c rather than try to have
a single all-purpose reporting function.
Switch to a custom function for converting int64_t to a string; in
the portable version, this saves a lot of configuration headaches
trying to decipher the platform printf().

Reformat the Makefile slightly.

Diff reduction compared to portable bsdtar 2.8: Move the
program name into a global, which eliminates an extra
argument from a lot of places.

Build usr.bin/ with WARNS=6 by default.

Also add some missing $FreeBSD$ to keep svn happy.

Make sure to use up-to-date libarchive header files from source tree when
compiling tar and not the ones from /usr/include/.

Reviewed by: kientzle
Approved by: kientzle

Make -lcrypto usage dependent on whether or not we're building with OpenSSL.

Merge remaining changes from libarchive.googlecode.com:
* Add xz and lzma compression options
* Rename --format-options to simply --options
* Add --same-owner for GNU tar compat
* Add -lmd and -lcrypto to fix link
* Documentation

Update version to 2.6.901a to indicate this is synced up with r745 of
libarchive.googlecode.com (except for the lzma/xz support).

Merge r687-689,691,693-701,720 from libarchive.googlecode.com:
Translate getdate.y into C for portability. Make the get_date()
function easier to test as well:
* Have it accept a time_t "now" to use as a reference so that test
code can verify relative time specifications against known starting
points.
* Set up default date after parsing the string so that we
can use the specified timezone (if any) instead of the local
default. Otherwise, local DST makes it almost impossible to
reliably test time specifications such as "sunday UTC"

bsdtar is synced up with 2.5.903a

New command-line parser for bsdtar.

This replaces the getopt()/getopt_long() wrapper, the old-style
argument rewriter and the associated configuration glue with a more
straightforward custom command parser. In particular, this ensures
that bsdtar will have consistent option parsing on every platform,
regardless of whether the platform supports getopt_long().

MFC after: 30 days

Don't clean before running tests.

FreeBSD-CURRENT bsdtar is synched up with libarchive-portable 2.5.5.

MFp4: bsdtar 2.5.4b

In addition to a number of bug fixes and minor changes:
* --numeric-owner (ignore user/group names on create and extract)
* -S (sparsify files on extraction)
* -s (regex filename substitutions)
* Use new libarchive 'linkify' to get correct hardlink handling for
both old and new cpio formats
* Rework 'copy' test to be insensitive to readdir() filename ordering

Most of the credit for this work goes to Joerg Sonnenberger, who
has been duplicating features from NetBSD's 'pax' program.

Add SIGINFO (and for portability to SIGINFO-lacking systems, SIGUSR1)
handling to bsdtar. When writing archives (including copying via the
@archive directive) a line is output to stderr indicating what is being
done (adding or copying), the path, and how far through the file we are;
extracting currently does not report progress within each file, but
this is likely to happen eventually.

Discussed with: kientzle
Obtained from: tarsnap

Fix --fast-read by decrementing the remaining patterns to be matched,
instead of incrementing.

Pointy hat: me
Thanks to: Tomasz Przygoda
MFC after: 3 days

A couple of miscellaneous fixes:
* prototypes for optarg/optind on platforms that don't already have them
* Disambiguate version number macros
* Remove unnecessary PACKAGE_NAME macro
* Hook for forthcoming bsdtar test suite
* Sync version number up with the portable distribution

Don't exit immediately on libarchive warnings, just
set the delayed return value and keep going.

Approved by: re (hrs)
MFC after: 7 days

bsdtar 2.2.3:
* Implement --use-compress-program using new libarchive feature.
* Minor portability improvement by adjusting casts used to
print out uids, gids, and device numbers.

Thanks to: Joerg Sonnenberger for the --use-compress-program implementation.
MFC after: 15 days

Make it easier to support more platforms.
Thanks to: Joerg Sonnenberger for pointing out the need and the technique.

Issue a warning if there's a non-zero exit value.

bsdtar 2.0.23:
* New test scripts exercise some basic functionality
* Most header inclusions are now protected (portability)
* read.c now relies on security checks in libarchive instead
of trying to do its own (optimization)
* -p now enabled by default for root, add --no-same-permissions
to disable it
* Comments, minor style fixes.

If you see a broken symlink, don't try to follow it,
just archive it as a symlink, even if -h was specified.

Thanks to: Jin Guojun
PR: bin/95175
MFC After: 1 week

Extended attribute support on write for Linux; FreeBSD hooks are
forthcoming. This commit also has a number of style(9) fixes and
minor corrections so the code works better with the build system being
used for non-FreeBSD builds.

Many thanks to: Jaakko Heinonen, who proposed a mechanism for extended
attribute support and implemented both the machine-independent portion
and the Linux-specific portion.

Remove configure.ac.in and the 'distfile' target from
the FreeBSD makefile. This is a step towards cleaning
out the non-FreeBSD bits from the FreeBSD source tree.

Clarify some error messages.

bsdtar now uses the "tree" package instead of "fts" for walking
directory heirarchies.

Correct autoconf/automake invocation to work with devel/gnu-auto* from
ports.

Thanks to: Dag-Erling Sm?rgrav (des@)

Update "make distfile" to use newest automake/autoconf from ports.

Thanks to: Juergen Lock

Support path-rewriting options (including --strip-components) for both
extraction and creation. While I'm here, fix a bug reported by Garrett
Wollman: when stripping the leading '/' from the path "/", don't produce
an entry with an empty name; produce "." instead.

Deorbit gtar, step 1: Remove WITH_GTAR knob.
"tar" is now always a synonym for "bsdtar".
People who need gtar can access it as "gtar."

Approved by: No objections on arch@

Re-unbreak the distfile target.

Also, reduce the WARNS level to 5 since different build environments
end up using different Yacc skeletons. The BSD one does not
predeclare yyparse, the FSF one does, so it's not really possible to
consistently enforce both -Wmissing-prototypes and -Wredundant-decls.

"make depend" with .y files is trickier than it looks. <sigh>

Add --newer-ctime, --newer-mtime, --newer-ctime-than, and --newer-mtime-than
switches to support selecting files by time of modification.

Special thanks to: Steven M. Bellovin, Rich $alz, and Jim Berets,
authors of the public-domain getdate.y date-parsing code.

Add --strip-components option, per bin/77666.

Thanks to: Sangwoo Shim

Portability: the included fts.c should now
compile on many non-FreeBSD systems, including
Linux, Solaris, Darwin, etc.

Thanks to: many people who pointed out the same portability problem

Use a relative symlinking so that "tar" also works in /stand.

Correct error handling on failed writes
to the archive.

gtar uses -F for another purpose, so bsdtar no longer treats it as a
synonym for --format. Update the man page to reflect this. While
I'm here, change the man page to document "tar" rather than "bsdtar,"
update some comments about -l compatibility and fix a few grammar nits.

Add --totals option. Unlike gtar, this reports the bytes actually
written to the archive, not the ones written to the compressor.
For uncompressed archives, these numbers are the same, of course.

Add "make distfile" capabilities to bsdtar, including informational
COPYING file and some conditional compilation cleanups.

Add a --version option to bsdtar that prints the versions of
both bsdtar and libarchive. Of course, this requires that bsdtar
have a version number. Let's call this 1.00, shall we? ;-)

Make bsdtar the default system tar. This makes /usr/bin/tar a symlink
pointing to /usr/bin/bsdtar by default. To make it point to /usr/bin/gtar,
you can define WITH_GTAR.

Oops. Forced commit to document that the previous commit
also included a Makefile change: use a symlink
/usr/bin/tar -> /usr/bin/bsdtar rather than a hardlink so that
people can more easily tell which tar they're currently using.

Pointy hat: sticky bit 't' or 'T' is shown over IXOTH bit, not IXGRP. <sigh>

Allow brave souls to install bsdtar as the
default "tar" command by defining WITH_BSDTAR.

Note: "gtar" and "bsdtar" commands both exist regardless.

Style fixes

Pointed out by: bde

Add DPADD to Makefile (thanks to Andrey for pointing this out).

Generally clean up Makefile, remove dmalloc references from source
(there are better ways to do this than pollute the FreeBSD src tree).

Initial commit for bsdtar.