Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

sh: Detect an error for ${#var<GARBAGE>}.

In particular, this makes things like ${#foo[0]} and ${#foo[@]} errors
rather than silent equivalents of ${#foo}.

PR: bin/151720
Submitted by: Mark Johnston
Exp-run done by: pav (with some other sh(1) changes)

sh: Do not try to execute binary files as scripts.

If execve() returns an [ENOEXEC] error, check if the file is binary before
trying to execute it using sh. A file is considered binary if at least one
of the first 256 bytes is '\0'.

In particular, trying to execute ELF binaries for the wrong architecture now
fails with an "Exec format error" message instead of syntax errors and
potentially strange results.

sh: Check readonly status for assignments on regular builtins.

An error message is written, the builtin is not executed, nonzero exit
status is returned but the shell does not abort.

This was already checked for special builtins and external commands, with
the same consequences except that the shell aborts for special builtins.

Obtained from: NetBSD

sh: Check if dup2 for redirection from/to a file succeeds.

A failure (e.g. caused by ulimit -n being set very low) is a redirection
error.

Example:
ulimit -n 9; exec 9<.

sh: Fix some tests that used sh instead of ${SH}
so they tested the wrong sh.

This was caused because these tests were committed after the sh -> ${SH}
change but were created before.

sh: Detect various additional errors in the parser.

Apart from detecting breakage earlier or at all, this also fixes a segfault
in the testsuite. The "handling" of the breakage left an invalid internal
representation in some cases.

Examples:
echo a; do echo b
echo `) echo a`
echo `date; do do do`

Exp-run done by: pav (with some other sh(1) changes)

Allow one to regression test 'sh' changes without having to install
a potentially bad /bin/sh first.

sh: Fix bug in assignment error test.

The test failed if the command returned nonzero exit status, and it really
should return that.

sh: Do not abort on a redirection error on a compound command.

Redirection errors on subshells already did not abort the shell because
the redirection is executed in the subshell.

Other shells seem to agree that these redirection errors should not abort
the shell.

Also ensure that the redirections will be cleaned up properly in cases like
command eval '{ shift x; } 2>/dev/null'

Example:
{ echo bad; } </var/empty/x; echo good

sh: Add test for redirection error on subshell (should not abort).

sh: Do not abort on a redirection error if there is no command word.

Although simple commands without a command word (only assignments and/or
redirections) are much like special builtins, POSIX and most shells seem to
agree that redirection errors should not abort the shell in this case. Of
course, the assignments persist and assignment errors are fatal.

To get the old behaviour portably, use the ':' special builtin.
To get the new behaviour portably, given that there are no assignments, use
the 'true' regular builtin.

sh: Add test for assignment errors (e.g. trying to change a readonly var).
We currently ignore readonly status for assignments before regular builtins
and external programs (these assignments are not persistent anyway), so just
check that the readonly variable really is not changed.
The test depends on the command builtin changes for 'command :'.

sh: Improve the command builtin:
* avoid unnecessary fork
* allow executing builtins via command
* executing a special builtin via command removes its special properties

Obtained from: NetBSD (parts)

Disable job control when running 'sh -i' in the testsuite.

Job control tty manipulations sometimes cause the tests to stop (SIGTTOU
and the like) when run from the 'prove' tool.

sh: Disallow mismatched quotes in backticks (`...`).

Due to the amount of code removed by this, it seems that allowing unmatched
quotes was a deliberate imitation of System V sh and real ksh. Most other
shells do not allow unmatched quotes (e.g. bash, zsh, pdksh, NetBSD /bin/sh,
dash).

PR: bin/137657

Add tests for r194406 and r194516.

Approved by: ed (mentor)

Add a regression test for r181017.

Submitted by: simon

Set magic fbsd:nokeywords property that allows files to bypass
keyword expansion. (file-specific replacement for CVSROOT/exclude)

Add regression tests for PR 105078.

Add test cases that check utility syntax errors and redirection errors. For
special built-in utilities they must terminate the shell, for other
utilities only a error message shall be written. We currently fail both
tests.