Copy stable/11@r303970 to releng/11.0 as part of the 11.0-RELEASE
cycle.

Prune svn:mergeinfo from the new branch, and rename it to RC1.

Update __FreeBSD_version.

Use the quarterly branch for the default FreeBSD.conf pkg(8) repo and
the dvd1.iso packages population.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
namespacing is kept with FILES appropriately, and that this shouldn't need
to be repeated if the namespace changes -- only the definition of PACKAGE
needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
`tests`. In the event we get to the point where things can be split up
enough in the base system, it would make more sense to group the tests
with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division

Merge the projects/release-pkg branch to head.

This allows packaging the base system with pkg(8), including
but not limited to providing the ability to provide upstream
binary update possibilities for non-tier-1 architectures.

This merge is a requirement of the 11.0-RELEASE, and as such,
thank you to everyone that has tested the project branch.

Documentation in build(7) etc. is still somewhat sparse, but
updates to those parts will follow.

Sponsored by: The FreeBSD Foundation

DIRDEPS_BUILD: Connect MK_TESTS.

Sponsored by: EMC / Isilon Storage Division

sh: Don't create bad parse result when postponing a bad substitution error.

An invalid substitution like ${var@} does not cause a parse error but is
stored in the intermediate representation, to be written as part of the
error message. If there is a CTL* byte in the stored part, this confuses
some code such as the code to skip an unused alternative such as in
${var-alternative}.

To keep things simple, do not store CTL* bytes.

Found with afl-fuzz.

MFC after: 1 week

sh: Don't hardcode relative paths in the tests stderr files.

These paths have had to be adjusted to changes in the testsuite runner
several times, so modify the tests to remove the need for such adjustment.

A cp in functional_test.sh is now unneeded, but this matters little in
performance.

Convert bin/sh/tests to ATF

The new code uses a "test discovery mechanism" to determine
what tests are available for execution

The test shell can be specified via:

kyua test -v test_suites.FreeBSD.bin.sh.test_shell=/path/to/test/sh

Sponsored by: EMC / Isilon Storage Division
Approved by: jmmv (mentor)
Reviewed by: jilles (maintainer)

Migrate tools/regression/bin/ tests to the new layout.

This change is a proof of concept on how to easily integrate existing
tests from the tools/regression/ hierarchy into the /usr/tests/ test
suite and on how to adapt them to the new layout for src.

To achieve these goals, this change:

- Moves tests from tools/regression/bin/<tool>/ to bin/<tool>/tests/.
- Renames the previous regress.sh files to legacy_test.sh.
- Adds Makefiles to build and install the tests and all their supporting
data files into /usr/tests/bin/.
- Plugs the legacy_test test programs into the test suite using the new
TAP backend for Kyua (appearing in 0.8) so that the code of the test
programs does not have to change.
- Registers the new directories in the BSD.test.dist mtree file.

Reviewed by: freebsd-testing
Approved by: rpaulo (mentor)

sh: Fix a Subversion property.

Test scripts should have keywords; only pure output that must match exactly
should have keywords disabled.

sh: Detect and flag write errors on stdout in builtins.

If there is a write error on stdout, a message will be printed (to stderr)
and the exit status will be changed to 2 if it would have been 0 or 1.

PR: bin/158206

sh: Detect an error for ${#var<GARBAGE>}.

In particular, this makes things like ${#foo[0]} and ${#foo[@]} errors
rather than silent equivalents of ${#foo}.

PR: bin/151720
Submitted by: Mark Johnston
Exp-run done by: pav (with some other sh(1) changes)

sh: Do not try to execute binary files as scripts.

If execve() returns an [ENOEXEC] error, check if the file is binary before
trying to execute it using sh. A file is considered binary if at least one
of the first 256 bytes is '\0'.

In particular, trying to execute ELF binaries for the wrong architecture now
fails with an "Exec format error" message instead of syntax errors and
potentially strange results.

sh: Check readonly status for assignments on regular builtins.

An error message is written, the builtin is not executed, nonzero exit
status is returned but the shell does not abort.

This was already checked for special builtins and external commands, with
the same consequences except that the shell aborts for special builtins.

Obtained from: NetBSD

sh: Check if dup2 for redirection from/to a file succeeds.

A failure (e.g. caused by ulimit -n being set very low) is a redirection
error.

Example:
ulimit -n 9; exec 9<.

sh: Fix some tests that used sh instead of ${SH}
so they tested the wrong sh.

This was caused because these tests were committed after the sh -> ${SH}
change but were created before.

sh: Detect various additional errors in the parser.

Apart from detecting breakage earlier or at all, this also fixes a segfault
in the testsuite. The "handling" of the breakage left an invalid internal
representation in some cases.

Examples:
echo a; do echo b
echo `) echo a`
echo `date; do do do`

Exp-run done by: pav (with some other sh(1) changes)

Allow one to regression test 'sh' changes without having to install
a potentially bad /bin/sh first.

sh: Fix bug in assignment error test.

The test failed if the command returned nonzero exit status, and it really
should return that.

sh: Do not abort on a redirection error on a compound command.

Redirection errors on subshells already did not abort the shell because
the redirection is executed in the subshell.

Other shells seem to agree that these redirection errors should not abort
the shell.

Also ensure that the redirections will be cleaned up properly in cases like
command eval '{ shift x; } 2>/dev/null'

Example:
{ echo bad; } </var/empty/x; echo good

sh: Add test for redirection error on subshell (should not abort).

sh: Do not abort on a redirection error if there is no command word.

Although simple commands without a command word (only assignments and/or
redirections) are much like special builtins, POSIX and most shells seem to
agree that redirection errors should not abort the shell in this case. Of
course, the assignments persist and assignment errors are fatal.

To get the old behaviour portably, use the ':' special builtin.
To get the new behaviour portably, given that there are no assignments, use
the 'true' regular builtin.

sh: Add test for assignment errors (e.g. trying to change a readonly var).
We currently ignore readonly status for assignments before regular builtins
and external programs (these assignments are not persistent anyway), so just
check that the readonly variable really is not changed.
The test depends on the command builtin changes for 'command :'.

sh: Improve the command builtin:
* avoid unnecessary fork
* allow executing builtins via command
* executing a special builtin via command removes its special properties

Obtained from: NetBSD (parts)

Disable job control when running 'sh -i' in the testsuite.

Job control tty manipulations sometimes cause the tests to stop (SIGTTOU
and the like) when run from the 'prove' tool.

sh: Disallow mismatched quotes in backticks (`...`).

Due to the amount of code removed by this, it seems that allowing unmatched
quotes was a deliberate imitation of System V sh and real ksh. Most other
shells do not allow unmatched quotes (e.g. bash, zsh, pdksh, NetBSD /bin/sh,
dash).

PR: bin/137657

Add tests for r194406 and r194516.

Approved by: ed (mentor)

Add a regression test for r181017.

Submitted by: simon

Set magic fbsd:nokeywords property that allows files to bypass
keyword expansion. (file-specific replacement for CVSROOT/exclude)

Add regression tests for PR 105078.

Add test cases that check utility syntax errors and redirection errors. For
special built-in utilities they must terminate the shell, for other
utilities only a error message shall be written. We currently fail both
tests.