libarchive: import changes from upstream

Libarchive 3.5.2

New features:
PR #1502: Support for PWB and v7 binary cpio formats
PR #1509: Support of deflate algorithm in symbolic link decompression
for ZIP archives

Important bugfixes:
IS #1044: fix extraction of hardlinks to symlinks
PR #1480: Fix truncation of size values during 7zip archive
extraction on 32bit architectures
PR #1504: fix rar header skiming
PR #1514: ZIP excessive disk read - fix location of central directory
PR #1520: fix double-free in CAB reader
PR #1521: Fixed leak of rar before ending with error
PR #1530: Handle short writes from archive_write_callback
PR #1532: 7zip: Use compression settings from file also for file header
IS #1566: do not follow symlinks when processing the fixup list

Relnotes: yes

(cherry picked from commit ddce862ad8594542e1fa1af9ffae7264e12ffd27)

Git Hash: fd9825481d30a1a083219d610ecaaa152fceb21b
Git Author: mm@FreeBSD.org

MFC r358533:
Sync libarchive with vendor.

Relevant vendor changes:
Issue #1257: Add testcase for ZIPX files with LZMA_STREAM_END marker
PR #1331: cpio.5: fix hard link description
Issue #1335: archive_read.c: fix UBSan warning about undefined behavior
Issue #1338: XAR reader: fix UBSan warning about undefined behavior
Issue #1339: bsdcpio_test: fix datatype in from_hex()
Issue #1341: Safe writes: delete temporary file if rename fails.
Issue #1341: Safe writes: improve error handling

MFC r356212,r356366,r356416,r357785
Update libarchive to version 3.4.2

Relevant vendor changes (r356212):
Issue #351: Refactor and implement private state logic for write filters
PR #1252: RAR5 reader - verify window size for solid files (OSS-Fuzz 15482)
PR #1255: zip writer - don't append unused NUL for directories
PR #1260: Fix sparse file offset overflow on 32-bit systems
PR #1263: UNICODE filename support for reading lha/lzh format
Issue #1276: Bugfix and optimize archive_wstring_append_from_mbs()
PR #1288: Add the "xattrhdr" option to pax write options
PR #1295: 7z reader - fix reading archives with digests in PackInfo
PR #1296: RAR5 reader - verify window size for multivolume archives
PR #1297: ZIP reader - support LZMA_STREAM_END marker in 'lzma alone' files
Issue #1298: Fix a heap-buffer-overflow in archive_string_append_from_wcs()
OSS-Fuzz 19360, 19362: LHA reader - plug two memory leaks on error
Fix possible off-by-one when dealing with readlink(2)

Relevant vendor changes (r356366):
Issue #1302: Plug memory leak on failure of archive_write_client_open()

Relevant vendor changes (r356416):
Issue #1302: Re-do fix for archive_write_client_open()

Relevant vendor changes (r357785):
PR #1289: atomic extraction support (bsdtar -x --safe-writes)
PR #1308: big endian fix for UTF16 support in LHA reader
PR #1326: reject RAR5 files that declare invalid header flags
Issue #987: fix support 7z archive entries with Delta filter
Issue #1317: fix compression output buffer handling in XAR writer
Issue #1319: fix uname or gname longer than 32 characters in pax writer
Issue #1325: fix use after free when archiving hardlinks in ISO9660 or XAR
Use localtime_r() and gmtime_r() instead of localtime() and gmtime()

MFC r339746,339751,339794,340866,340939,342042:
Sync libarchive with vendor.

Relevant vendor changes:
PR #1013: Add missing h_base offset when performing absolute seeks in
xar decompression
PR #1023: Support extracting extattrs as non-root on non-user-writeable
files
PR #1061: Add support for extraction of RAR v5 archives
PR #1066: Fix out of bounds read on empty string filename for gnutar, pax
and v7tar
PR #1067: Fix temporary file path buffer overflow in tests
IS #1068: Correctly process and verify integer arguments passed to
bsdcpio and bsdtar
PR #1070: Don't default XAR entry atime/mtime to the current time
PR #1080: Spelling fixes
PR #1084: RAR5 reader bugfixes
PR #1091: fix use-after-free in delayed newc link processing
PR #1092: Fix a few obvious resource leaks and strcpy() misuses
IS #1096: Support extracting ACLs with in-entry comments (GNU tar)
PR #1102: RAR5 reader - fix big-endian problems
PR #1105: Fix various crash, memory corruption and infinite loop conditions

RAR5 reader: FreeBSD build platform fixes for powerpc(64), mips(64),
sparc64 and riscv64
RAR5 reader: more maybe-uninitialized size_t fixes for riscv64
FreeBSD build

MFH r336801,r336854:

MFH r336801 (cem):
Cherry-pick upstream 2c8c83b9

Relevant vendor changes:
Fix issue #948: out-of-bounds read in lha_read_data_none()

MFH r336854:
Sync libarchive with vendor.

Important vendor changes:
PR #993: Chdir to -C directory for metalog processing
OSS-Fuzz #4969: Check size of the extended time field in zip archives
PR #973: Record informational compression level in gzip header

amdbugs: 877
Security: CVE-2017-14503

MFH r324148:
Sync libarchive with vendor.

Relevant vendor changes:
PR #905: Support for Zstandard read and write filters
PR #922: Avoid overflow when reading corrupt cpio archive
Issue #935: heap-based buffer overflow in xml_data (CVE-2017-14166)
OSS-Fuzz 2936: Place a limit on the mtree line length
OSS-Fuzz 2394: Ensure that the ZIP AES extension header is large enough
OSS-Fuzz 573: Read off-by-one error in RAR archives (CVE-2017-14502)

Security: CVE-2017-14166, CVE-2017-14502

MFC r317782,318181:

MFC r317782 (mm):
Sync libarchive with vendor

Vendor changes (FreeBSD-related):
PR 897: add test for ZIP archives with invalid EOCD headers
PR 901: fix invalid renaming of sparse files
OSS-Fuzz issue 497: remove fallback tree in LZX decoder
OSS-Fuzz issue 527: rewrite expressions in lz4 filter
OSS-Fuzz issue 577: fix integer overflow in cpio reader
OSS-Fuzz issue 862: fix numerc parsing in mtree reader
OSS-Fuzz issue 1097: fix undefined shift in rar reader
cpio: various optimizations and memory leak fixes

MFC r318181 (ngie) (2):
cpio/tests/test_option_lz4: fix a use after free in the failure case

Reported by: Coverity (2)
Sponsored by: Dell EMC Isilon (2)

MFC r315636,315876,316095:
Sync libarchive with vendor

Vendor changes/bugfixes (FreeBSD-related):
r315636:
PR 867 (bsdcpio): show numeric uid/gid when names are not found
PR 870 (seekable zip): accept files with valid ZIP64 EOCD headers
PR 880 (pax): Fix handling of "size" pax header keyword
PR 887 (crypto): Discard 3072 bytes instead of 1024 of first keystream
OSS-Fuzz issue 806 (mtree): rework mtree_atol10 integer parser
Break ACL read/write code into platform-specific source files

r315876:
Store extended attributes with extattr_set_link() if no fd is provided
Add extended attribute tests to libarchive and bsdtar
Fix tar's test_option_acls
Support the UF_HIDDEN file flag

r316095:
Constify variables in several places
Unify platform ACL code in a single source file
Fix unused variable if compiling on FreeBSD without NFSv4 ACL support

MFC r314571:
Update libarchive to version 3.3.1 (and sync with latest vendor dist)

Notable vendor changes:
PR #501: improvements in ACL path handling
PR #724: fix hang when reading malformed cpio files
PR #864: fix out of bounds read with malformed GNU tar archives
Documentation, style, test suite improvements and typo fixes.

New options to bsdtar that enable or disable reading and/or writing of:
Access Control Lists (--acls, --no-acls)
Extended file flags (--fflags, --no-fflags)
Extended attributes (--xattrs, --no-xattrs)
Mac OS X metadata (Mac OS X only) (--mac-metadata, --no-mac-metadata)

MFC r310866,310868,310870,311903,313074:
Sync libarchive with vendor.

MFC r310866:
PR #771: Add NFSv4 ACL support to pax and restricted pax

NFSv4 ACL information may now be stored to and restored from tar archives.
ACL must be non-trivial and supported by the underlying filesystem, e.g.
natively by ZFS or by UFS with the NFSv4 ACL enable flag set.

MFC r310868:
PR #843: Fix memory leak of struct archive_entry in cpio/cpio.c
PR #851: Spelling fixes
Fix two protoypes in manual page archive_read_disk.3

MFC r310870:
Use __LA_DEPRECATED macro with functions deprecated in 379867e

MFC r311903:
#691: Support for SCHILY.xattr extended attributes
#854: Spelling fixes

Multiple fixes in ACL code:
- prefer acl_set_fd_np() to acl_set_fd()
- if acl_set_fd_np() fails, do no fallback to acl_set_file()
- do not warn if trying to write ACLs to a filesystem without ACL support
- fix id handling in archive_acl_(from_to)_text*() for NFSv4 ACLs

MFC r313074:
- support extracting NFSv4 ACLs from Solaris tar archives
- bugfixes and optimizations in the ACL code
- multiple fixes in the test suite
- typo and other small bugfixes

Security fixes:
- cab reader: endless loop when parsing MSZIP signature (OSS-Fuzz 335)
- LHA reader: heap-buffer-overflow in lha_read_file_header_1()
(CVE-2017-5601)
- LZ4 reader: null-pointer dereference in lz4_filter_read_legacy_stream()
(OSS-Fuzz 453)
- mtree reader: heap-buffer-overflow in detect_form() (OSS-Fuzz 421, 443)
- WARC reader: heap-buffer-overflow in xstrpisotime() (OSS-Fuzz 382, 458)

Memory leak fixes:
- ACL support: free memory allocated by acl_get_qualifier()
- disk writer: missing free in create_filesystem_object()
- file reader: fd leak (Coverity 1016755)
- gnutar writer: fix free in archive_write_gnutar_header()
(Coverity 101675)
- iso 9660 reader: missing free in parse_file_info()
(partial Coverity 1016754)
- program reader: missing free in __archive_read_program()
- program writer: missing free in __archive_write_program_free()
- xar reader: missing free in xar_cleanup()
- xar reader: missing frees in expat_xmlattr_setup()
(Coverity 1229979-1229981)
- xar writer: missing free in file_free()
- zip reader: missing free in zip_read_local_file_header()

List of all libarchive issues at OSS-Fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=libarchive

Security: CVE-2017-5601

MFC r309300,r309363,r309405,r309523,r309590,r310185,r310623:

Sync libarchive with vendor.

Fixed vendor issues (relevant to FreeBSD)
#825, #832: Add sanity check of tar "uid, "gid" and "mtime" fields
#830, #831, #833, #846: Spelling fixes
#850: Fix issues with reading certain jar files

Fixed issues found by Google OSS-Fuzz:
OSS-Fuzz #15: Fix heap-buffer-overflow in archive_le16dec()
OSS-Fuzz #16: Fix possible hang in uudecode_filter_read()
OSS-Fuzz #139, #145, #152: Fix heap-buffer-overflow in uudecode_bidder_bid()
OSS-Fuzz #220: Reject an 'ar' filename table larger than 1GB or a filename
larger than 1MB
OSS-Fuzz #227, #230, #239: Fix possible memory leak in archive_read_free()
OSS-Fuzz #237: Fix heap buffer overflow when reading invalid ar archives
OSS-Fuzz #286: Bugfix in archive_strncat_l()

More information:
https://github.com/libarchive/libarchive/issues/[libarchive_issue_number]
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=[oss_fuzz_issue_number]

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFV r302003,r302037,r302038,r302056:

Update libarchive to 3.2.1 (bugfix and security fix release)

List of vendor fixes:
- fix exploitable heap overflow vulnerability in Rar decompression
(vendor issue 719, CVE-2016-4302, TALOS-2016-0154)
- fix exploitable stack based buffer overflow vulnebarility in mtree
parse_device functionality (vendor PR 715, CVE-2016-4301, TALOS-2016-0153)
- fix exploitable heap overflow vulnerability in 7-zip read_SubStreamsInfo
(vendor issue 718, CVE-2016-4300, TALOS-2016-152)
- fix integer overflow when computing location of volume descriptor
(vendor issue 717)
- fix buffer overflow when reading a crafred rar archive (vendor issue 521)
- fix possible buffer overflow when reading ISO9660 archives on machines
where sizeof(int) < sizeof(size_t) (vendor issue 711)
- tar and cpio should fail if an input file named on the command line is
missing (vendor issue 708)
- fix incorrect writing of gnutar filenames that are exactly 512 bytes
long (vendor issue 682)
- allow tests to be run from paths that are equal or longer than 128
characters (vendor issue 657)
- add memory allocation errors in archive_entry_xattr.c (vendor PR 603)
- remove dead code in archive_entry_xattr_add_entry() (vendor PR 716)
- fix broken decryption of ZIP files (vendor issue 553)
- manpage style, typo and description fixes

Post-3.2.1 vendor fixes:
- fix typo in cpio version reporting (Vendor PR 725, 726)
- fix argument range of ctype functions in libarchive_fe/passphrase.c
- fix ctype use and avoid empty loop bodies in WARC reader

MFC after: 1 week
Security: CVE-2016-4300, CVE-2016-4301, CVE-2016-4302
Approved by: re (kib)

Revert r299576 and MFV r299895:

Revert r299576:
Fix broken cpio behavior.

MFV r299895:
Update to vendor git commit 860ec63.

MFC after: 3 weeks (together with libarchive 3.2.0)
Fix broken cpio behavior in pass-through mode with vendor code.
> Description of fields to fill in above: 76 columns --|
> PR: If and which Problem Report is related.
> Submitted by: If someone else sent in the change.
> Reported by: If someone else reported the issue.
> Reviewed by: If someone else reviewed your modification.
> Approved by: If you needed approval for this commit.
> Obtained from: If the change is from a third party.
> MFC after: N [day[s]|week[s]|month[s]]. Request a reminder email.
> MFH: Ports tree branch name. Request approval for merge.
> Relnotes: Set to 'yes' for mention in release notes.
> Security: Vulnerability reference (one per line) or description.
> Sponsored by: If the change was sponsored by an organization.
> Differential Revision: https://reviews.freebsd.org/D### (*full* phabric URL needed).
> Empty fields above will be automatically removed.

_M libarchive
_M libarchive/cpio
M libarchive/cpio/bsdcpio.1
M libarchive/cpio/cpio.c
_M libarchive/libarchive
M libarchive/libarchive/archive_read_support_format_cpio.c

Fix broken cpio behavior.
Suggested upstream as PR #704.

MFC: 1 month (together with libarchive 3.2.0)

MFV r299425:

Update libarchive to 3.2.0

New features:
- new bsdcat command-line utility
- LZ4 compression (in src only via external utility from ports)
- Warc format support
- 'Raw' format writer
- Zip: Support archives >4GB, entries >4GB
- Zip: Support encrypting and decrypting entries
- Zip: Support experimental streaming extension
- Identify encrypted entries in several formats
- New --clear-nochange-flags option to bsdtar tries to remove noschg and
similar flags before deleting files
- New --ignore-zeros option to bsdtar to handle concatenated tar archives
- Use multi-threaded LZMA decompression if liblzma supports it
- Expose version info for libraries used by libarchive

Patched files (fixed compiler warnings):

contrib/libarchive/cat/bsdcat.c (vendor PR #702)
contrib/libarchive/cat/bsdcat.h (vendor PR #702)
contrib/libarchive/libarchive/archive_read_support_format_mtree.c (PR #701)
contrib/libarchive/libarchive_fe/err.c (vendor PR #703)

MFC after: 1 month
Relnotes: yes

Fix the following -Werror warning from clang 3.5.0, while building
usr.bin/cpio on amd64 (or any arch with 64-bit time_t):

contrib/libarchive/cpio/cpio.c:1143:6: error: absolute value function 'abs' given an argument of type 'long' but has parameter of type 'int' which may cause truncation of value [-Werror,-Wabsolute-value]
if (abs(mtime - now) > (365/2)*86400)
^
contrib/libarchive/cpio/cpio.c:1143:6: note: use function 'labs' instead
if (abs(mtime - now) > (365/2)*86400)
^~~
labs
1 error generated.

This is because time_t is a long on amd64. To avoid the warning, just
copy the equivalent test from a few lines before, which is used in the
Windows case, and which is type safe.

Reviewed by: emaste
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D1198

MFV r248590,248594:
Update libarchive to 3.1.2

Some of new features:
- support for lrzip and grzip compression
- support for writing tar v7 format
- b64encode and uuencode filters
- support for __MACOSX directory in Zip archives
- support for lzop compresion (external utility)

Update libarchive to 3.0.4

Update libarchive to 3.0.3

Some of new features:
- New readers: RAR, LHA/LZH, CAB reader, 7-Zip
- New writers: ISO9660, XAR
- Improvements to many formats, especially including ISO9660 and Zip
- Stackable write filters to write, e.g., tar.gz.uu in a single pass
- Exploit seekable input; new "seekable" Zip reader can exploit the Zip
Central Directory when it's available; the old "streamable" Zip reader
is still fully supported for cases where seeking is not possible.

Full release notes available at:
https://github.com/libarchive/libarchive/wiki/ReleaseNotes

Merge FreeBSD changes from usr.bin/cpio to contrib/libarchive/cpio:

r204111 (uqs):
Fix common misspelling of hierarchy

r211054 (kientzle);
Fix -R when used with -p. Previously, the
uname and gname weren't overwritten, so the
disk restore would use those to lookup the
original uid/gid again. Clearing the uname
and gname prevents this.

r212263 (gjb):
Fix typo in bsdcpio manual:
s/libarchive_formats/libarchive-formats

MFC after: 2 weeks

Set svn:keywords to FreeBSD=%H for contrib/libarchive

MFC after: 2 weeks

Copy libarchive from vendor branch to contrib

MFC after: 2 weeks

Set svn:eol-style property to native for all text files in vendor/libarchive

Vendor import of libarchive (release/2.8, r3824)

Obtained from: http://libarchive.googlecode.com/svn/release/2.8