Copy stable/11@r303970 to releng/11.0 as part of the 11.0-RELEASE
cycle.

Prune svn:mergeinfo from the new branch, and rename it to RC1.

Update __FreeBSD_version.

Use the quarterly branch for the default FreeBSD.conf pkg(8) repo and
the dvd1.iso packages population.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division

Merge the projects/release-pkg branch to head.

This allows packaging the base system with pkg(8), including
but not limited to providing the ability to provide upstream
binary update possibilities for non-tier-1 architectures.

This merge is a requirement of the 11.0-RELEASE, and as such,
thank you to everyone that has tested the project branch.

Documentation in build(7) etc. is still somewhat sparse, but
updates to those parts will follow.

Sponsored by: The FreeBSD Foundation

DIRDEPS_BUILD: Connect MK_TESTS.

Sponsored by: EMC / Isilon Storage Division

Update NetBSD RCS IDs to reflect the changes being upstreamed.

MFC after: 13 days
X-MFC-With: r290024

In gunzip(1), treat trailing garbage as a warning and not an error. This
allows scripts to distinguish it between real fatal errors, for instance a
CRC mismatch.

Update manual page for the behavior change.

PR: bin/203873
Submitted by: Eugene Grosbein <eugen grosbein net>
MFC after: 2 weeks

Refactor the test/ Makefiles after recent changes to bsd.test.mk (r289158) and
netbsd-tests.test.mk (r289151)

- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison

Add META_MODE support.

Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision: D2796
Reviewed by: brooks imp

r281540 was upstreamed as NetBSD r1.108 of gzip.c, note it as merged.

When reading in the original file name from gzip header, we read
in PATH_MAX + 1 bytes from the file. In r281500, strrchr() is
used to strip possible path portion of the file name to mitigate
a possible attack. Unfortunately, strrchr() expects a buffer
that is NUL-terminated, and since we are processing potentially
untrusted data, we can not assert that be always true.

Solve this by reading in one less byte (now PATH_MAX) and
explicitly terminate the buffer after the read size with NUL.

Reported by: Coverity
CID: 1264915
X-MFC-with: 281500
MFC after: 13 days

Sync with NetBSD:

- Mention xz(1) in gzip(1).
- Strip away path from header name when decompressing.

MFC after: 2 weeks

compress,gzip,xz: Preserve timestamps with nanosecond precision.

Convert to usr.bin/ to LIBADD
Reduce overlinking

Sync with NetBSD.

MFC after: 2 weeks

Integrate usr.bin/gzip/tests from NetBSD into atf/kyua

Sponsored by: EMC / Isilon Storage Division

Correct buffer size.

Submitted by: Sascha Wildner <swildner dragonflybsd org>
MFC after: 2 weeks

use .Mt to mark up email addresses consistently (part3)

PR: 191174
Submitted by: Franco Fichtner <franco at lastsummer.de>

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

More -Wmissing-variable-declarations fixes.

In addition to adding `static' where possible:

- bin/date: Move `retval' into extern.h to make it visible to date.c.
- bin/ed: Move globally used variables into ed.h.
- sbin/camcontrol: Move `verbose' into camcontrol.h and fix shadow warnings.
- usr.bin/calendar: Remove unneeded variables.
- usr.bin/chat: Make `line' local instead of global.
- usr.bin/elfdump: Comment out unneeded function.
- usr.bin/rlogin: Use _Noreturn instead of __dead2.
- usr.bin/tset: Pull `Ospeed' into extern.h.
- usr.sbin/mfiutil: Put global variables in mfiutil.h.
- usr.sbin/pkg: Remove unused `os_corres'.
- usr.sbin/quotaon, usr.sbin/repquota: Remove unused `qfname'.

Fix style nit: don't use leading zero for dates in .Dd

Prompted by: brueffer
Approved by: brueffer
MFC after: 3 days

According to the NetBSD foundation [1]:

Third parties are encouraged to change the license on any files which have
a 4-clause license contributed to the NetBSD Foundation to a 2-clause
license. We would also encourage you to inform us about these files, so
that we can continue to track the many places in which NetBSD is used.

http://www.netbsd.org/about/redistribution.html#why2clause [1]

Requested by: joel@

Incorporate recent changes from NetBSD. Most notable change is the addition
of support of decompressing xz files.

Obtained from: NetBSD

Sync RCS id with NetBSD, this was intentionally omitted from the advisory
in order to reduce patchset size.

MFC after: 3 days

Fix handling of corrupt compress(1)ed data. [11:04]

Add missing length checks on unix socket addresses. [11:05]

Approved by: so (cperciva)
Approved by: re (kensmith)
Security: FreeBSD-SA-11:04.compress
Security: CVE-2011-2895 [11:04]
Security: FreeBSD-SA-11:05.unix

Fix a regression introduced with previous changeset: if output is stdout,
do not check for symbolic link.

Match symbolic link handling behavior with GNU gzip, bzip2 and xz:

When we are operating on a symbolic link pointing to an existing
file, bail out by default, but go ahead if -f is specified.

Submitted by: arundel
MFC after: 2 weeks

Diff reduction against NetBSD. The most notable change is to zdiff(1) to
handle more file formats including bzip2 and xz.

MFC after: 2 weeks

Update manpage to remove CRT reference.

PR: 154441
Submitted by: Ben Haga (bhaga of absoludicrous com)
Suggestion by: pluknet
Approved by: keramida (mentor)
MFC after: 3 days

Correct some typos in comments, no functional changes.

In the past gunzip(1) write()'s after each inflate return. This is
not optimal from a performance standpoint since the write buffer is
not necessarily be filled up when the inflate rountine reached the
end of input buffer and it's not the end of file.

This problem gets uncovered by trying to pipe gunzip -c output to
a GEOM device directly, which enforces the writes be multiple of
sector size.

Sponsored by: iXsystems, Inc.
Reported by: jpaetzel
MFC after: 2 weeks

Check return value of dup(), it could be -1 when the system is running
out of file descriptors for instance.

Found with: Coverity Prevent(tm)
CID: 6084
MFC after: 1 month

Several style fixes as prompted by bde@.

While I'm there, loosen the st_nlink check and fix grammar for 1 extra
links.

Follow up revision 208888 with purely ident changes.

MFC after: 1 week

Correct a bug in gzip(1): make sure that initialize isb with fstat() on
input file before using it.

PR: bin/147275
Submitted by: thomas
MFC after: 1 week

Use _exit(2) system call directly instead of using exit(3) in signal
handler, as the latter is not guaranteed to be signal safe, and we
do not really care about flushing the stream during SIGINT.

Suggested by: Maxim Konovalov <maxim.konovalov gmail com>
MFC after: 13 days

Language improvements to make the BUGS section easier to read.

Reviewed by: Alexander Best <alexbestms wwu de>
MFC after: 13 days

Add a signal handler for SIGINT which removes output file when necessary.

While I'm there, move unlink_input() slightly down to after closing the
output file, in uncompression path.

MFC after: 2 weeks

Diff reduction against NetBSD and add myself to AUTHORS section of the
manual page as I wrote the unpack functionality. No actual executable
code change verified with md5(1).

Change all our own code to use st_*tim instead of st_*timespec.

Also remove some local patches to diff(1) which are now unneeded.

Build usr.bin/ with WARNS=6 by default.

Also add some missing $FreeBSD$ to keep svn happy.

We should distinguish between a real truncated case and EOF after
BZ_STREAM_END triggered re-init. Do it by introducing a new flag
to represent the 'cold' case after bzip2 state is reinitialized.

This fixes regression reported on -current@ as well as another one
I found during twiddling with gzip.

Reported by: swell.k gmail.com
MFC after: 1 week

Apply a NetBSD fix (revision 1.12) to handle multi-session bzip2 files
as created by pbzip2.

Submitted by: mrg (NetBSD.org)
MFC after: 1 week

Correct a stack underflow in gzip:

- Limit suffix to be no more than 30 bytes long. This matches GNU
behavior.
- Correct usage of memcpy().

Note that this commit only corrects the stack underflow issue, we
still need some other fixes to cover other edges. [1]

Reported by: Ron Jude <ronj wytheville org>
Discussed with: Matthew Green (original NetBSD gzip author),
Eygene Ryabinkin <rea-fbsd codelabs ru> [1]
Approved by: re (kib)

Sync with NetBSD:
- gzip -n does not store timestamp; [1]
- Reduce diff against NetBSD by moving some casts in our local
versions.

PR: bin/134955
Obtained from: NetBSD
MFC after: 1 month

Add support for uncompressing pack(1)'ed files. Pack(1) is a program found
in some commercial Unix systems, which utilizes Huffman minimum redundancy
code tree to compress files. This implementation supports the "new" pack
format only, just like GNU gzip did.

Thanks for oliver@'s archive set which I can test against, and Mingyan Guo
for providing helpful review of my code.

PR: bin/109567
MFC after: 1 month

Two fixes for SMALL case when compiling with WARNS=6:

- Reduce scope where return value can be referenced.
- Add a dummy access to timestamp to silence warning.

Submitted by: Mingyan Guo <guomingyan gmail com>

Mark functions as __dead2 in order to help the LLVM static checker
understand which code paths aren't possible.

Reported by: edwin

Sync with NetBSD's license changes.

Reflect the fact that we actually have NetBSD revision 1.3.

Remove clause 3 and 4 of NetBSD license.

Obtained from: NetBSD

Make it clear that gzcat expects each argument to be separate files.

PR: docs/123010
Submitted by: Andrew Wright <andrewhw ieee org>
MFC after: 2 weeks

Teach gunzip that .tbz and .tbz2 uncompress to .tar, in a manner similar to
its existing understanding that .tgz uncompresses to .tar.

MFC after: 3 days
PR: 121519
Submitted by: Ighighi <ighighi at gmail.com>

When gzipping to a target file, only set the source file flags on the
target file after the timestamp has been set; otherwise setting the
timestamp will fail if the flags don't permit it (i.e., uchg).

MFC after: 1 week
PR: 120208
Submitted by: Ighighi <ighighi at gmail.com>

Fix a few obvious errors in the znew(1) script.

PR: bin/120249
Submitted by: Jaakko Heinonen <see the PR for email>

- Simulate GNU gzip(1) behavior where full filename is
being output in verbose mode when doing recursive[1].
- Use better representation of S:

PR: bin/114470
Submitted by: Ighighi <ighighi gmail com> [1]
Approved by: re (hrs)

Mention 'k' option of gunzip.

Pointed out by: ighighi gmail.com

Add a '-k' flag which is similar with its bzip2(1) counterpart,
meaning that the user wants the input file to be left intact.

Feature request: Ighighi <ighighi gmail.com>
PR: bin/103006
MFC after: 1 month

Mention a limitation that was inherted from RFC1952, making
it impossible to obtain correct file size from a file that
is larger than 4GB before compression.

PR: bin/110329
MFC after: 1 week

Do not install zgrep, or links to zgrep, since zgrep is provided by
src/gnu/usr.bin/grep.

Reviewed by: delphij

Fix typo.

Obtained from: DragonFly

Replace the GNU gzip with a slightly modified NetBSD gzip. The
NetBSD version is a feature-to-feature re-implementation of GNU
gzip using the freely-redistributable zlib and this version is
expected to be mostly bug-to-bug compatible with the GNU
implementation.

- Because this is a piece of mature code and we want to make
changes so it is added directly rather than importing to
src/contrib.
- Connect newly added code to src/usr.bin/ and rescue/rescue
build.
- Disconnect the GNU gzip code from build for now, they will
be eventually removed completely.
- Provide two new src.conf(5) knobs, WITHOUT_BZIP2_SUPPORT and
WITHOUT_BZIP2.

Tested by: kris (full exp-7 pointyhat build)
Approved by: core (importing a 4-clause BSD licensed file)
Approved by: re (adding new utility during -HEAD code slush)