MFC r362452, r362478: liblzma: Make liblzma use libmd implementation of SHA256.

Fix build.

MFC r359201: MFV r359197: xz 5.2.5.

MFC r333783: MFV r333779: xz 5.2.4.

MFC r312452-r312512:

r312452-r312512:

- Use SRCTOP-relative paths to other directories instead of .CURDIR-relative ones

This simplifies pathing in make/displayed output

- Use .CURDIR:H instead of .CURDIR to simplify pathing in output, etc

MFC r311504: MFV r311477: xz 5.2.3.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Merge the projects/release-pkg branch to head.

This allows packaging the base system with pkg(8), including
but not limited to providing the ability to provide upstream
binary update possibilities for non-tier-1 architectures.

This merge is a requirement of the 11.0-RELEASE, and as such,
thank you to everyone that has tested the project branch.

Documentation in build(7) etc. is still somewhat sparse, but
updates to those parts will follow.

Sponsored by: The FreeBSD Foundation

META MODE: Fix 'make bootstrap-tools'.

The main problem was bitrot after elftoolchain being swapped in for the
GNU toolchain.

This also reworks how the list of 'host allowed' libraries is determined
to only allow INTERNALLIBs, which is needed for libelftc to come in.

For usr.bin/readelf use the same hack, as libelf and libdward, to bring in
the needed sys/ headers for host builds. This has not yet been a problem due
to readelf not being built as a host tool in buildworld. This is possible
in the meta build though when building the toolchain.

Sponsored by: EMC / Isilon Storage Division

META MODE: Prefer INSTALL=tools/install.sh to lessen the need for xinstall.host.

This both avoids some dependencies on xinstall.host and allows
bootstrapping on older releases to work due to lack of at least 'install -l'
support.

Sponsored by: EMC / Isilon Storage Division

MFV r291123:

xz 5.2.2.

MFC after: 1 month
Relnotes: yes

Replace beforeinstall: handling with FILES.

This actually fixes some cases to respect LIBRARIES_ONLY.

Sponsored by: EMC / Isilon Storage Division

only enable immintrin when clang is used. The base gcc does not support it.

Reviewed by: delphij

new depends

Add META_MODE support.

Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision: D2796
Reviewed by: brooks imp

Revert r282775 for now. The added dependency would cause problems for
e.g. ports-mgmt/pkg and we would like to do an exp-build.

Requested by: bdrewery

Revert r281372, it's no longer needed after r282726 (fix for PR 199119) as the
symbol conflict between libmd and libcrypto have been solved.

For reasons that I am not understand yet, using the libmd version of
SHA256 would make libarchive based application to give:

archive_read_extract(): Lzma library error: Corrupted input data

(Internally this is LZMA_DATA_ERR)

However, the libmd version of SHA256 is supposed to give same result
as the bundled version do, and xz(1) is not affected.

As a precaution, revert the SHA256 portion of r281316 for now until
we figure out why it breaks libarchive interoperability as we can't
yet rule out this change have introduced an ABI breakage.

MFV r281278:

- Update xz to 5.2.1, where the most visible change is that it
fixed a compression-ratio regression in fast mode LZMA1 and
LZMA2 and used cpuset_getaffinity() for CPU cores detection.
- Make liblzma use the base system SHA256 implementation instead of
the bundled one.
- Additional annotation in config.h for FreeBSD specific tweaks.
- Refresh symbols in XZprivate_1.0 to reflect reality.

Relnotes: yes
MFC after: 1 month (TBD)

compress,gzip,xz: Preserve timestamps with nanosecond precision.

Merge xz 5.2.0.

This brings support for multi-threaded compression. This brings close
N times faster compression where N is the number of CPU cores.
Because of this, liblzma now depends on libthr.

Soon libarchive will be modified to use the new lzma API.

Thanks to antoine@ for the exp-run.

Differential Revision: https://reviews.freebsd.org/D1786
Reviewed by: bapt

Do not regenerate and install liblzma.pc when only build libraries
aka do not regenerate while generating 32bits libs

Reported by: antoine

Install the liblzma pkg-config file

MFC after: 1 week

Update xz to git snapshot of v5.0 branch as of 20110711 (post-5.0.3)

For architectures with __NO_STRICT_ALIGNMENT
define TUKLIB_FAST_UNALIGNED_ACCESS

MFC after: 2 weeks

Fix some leftover binaries and shared libraries in the system that still
have an executable stack, due to linking in hand-assembled .S or .s
files, that have no .GNU-stack sections:

RWX --- --- /lib/libcrypto.so.6
RWX --- --- /lib/libmd.so.5
RWX --- --- /lib/libz.so.6
RWX --- --- /lib/libzpool.so.2
RWX --- --- /usr/lib/liblzma.so.5

These were found using scanelf, from the sysutils/pax-utils port.

Reviewed by: kib

Reintroduce bugfix from r210103 and fix xz on strong-aligned architectures.
This fix was accidentially reverted with the 5.0.0 update in r215187.

PR: bin/154310
Submitted by: Michael Moll <kvedulv@kvedulv.de>
MFC after: 3 days

Remove obsoleted private symbol.

Approved by: delphij (mentor)
MFC after: together with 5.0.0 update

Update xz to release 5.0.0

Approved by: delphij (mentor)
MFC after: 1 week

Upgrade xz to git snapshot as of 20101010

Approved by: delphij (mentor)
MFC after: 1 month

Unbreak xz (liblzma) on strong-aligned architectures (and without
emulation of misaligned memory accesses). We cannot map the unaligned
memory access functions to the ones used for aligned accesses, so do
not define TUKLIB_FAST_UNALIGNED_ACCESS.

Detect bit endianness through machine/endian.h. This fixes xz on big-endian
systems.

Tested on: sparc64 (kindly provided by linimon), amd64

- Add versioned symbols to liblzma
- Use default SHLIB_MAJOR.

Approved by: mm
X-MFC with: lzma library MFC

Add two public headers missing in r207842
Adjust CFLAGS for lzmainfo, xz, xzdec

Approved by: delphij (mentor)
MFC after: 2 weeks

Import of liblzma, xz, xzdec, lzmainfo from vendor branch
Add support for xz and lzma to lesspipe.sh (xzless, lzless)
Bump __FreeBSD_version

Approved by: delphij (mentor)
MFC after: 2 weeks