sh(1): Fix typo

Remove extra 'ing' from 'bookinging.'

Event: Advanced UNIX Programming Course (Fall’23) at NTHU.
Reviewed by: imp, Mina Galić
Pull Request: https://github.com/freebsd/freebsd-src/pull/934

bin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

bin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

src/bin/sh: Fix spelling errors

Pull Request: https://github.com/freebsd/freebsd-src/pull/544
Signed-off-by: Elyes HAOUAS <ehaouas@noos.fr>

sh: Unify EXERROR and EXEXEC

The difference between EXERROR and EXEXEC was that EXEXEC passed along
exitstatus and EXERROR set exitstatus to 2 in the handling code.

By changing the places that raised EXERROR to set exitstatus to 2, the
handling of EXERROR and EXEXEC becomes the same.

sh: Use exitstatus instead of exerrno to pass EXEXEC status

No functional change is intended.

sh: Don't leak wait* implementation details from jobs.c

Allow defining nofork builtins from builtins.def and move always-safe ones there.

The generated code remains the same.

Reviewed by: jilles
Differential Revision: https://reviews.freebsd.org/D11042

sh: Fix INTOFF leak when a redirection on a compound command fails.

Reported by: bdrewery

sh: Ensure memout.bufsize matches allocated buffer, if it exists.

sh: Simplify output buffering.

Similarly to how STPUTC was changed, change struct output to store the
pointer just past the end of the available space instead of the size of the
available space, so after writing a character it is only necessary to
increment a pointer and not to decrement a counter.

sh: Update TOUR and comments for some code changes, some of them old.

Also, improve some terminology in TOUR and comments.

Renumber copyright clause 4

Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96

sh: Update associated state when restoring locals while leaving a function.

Some variables like PATH call a function when modified. Make sure to call
this also when leaving a function where such a variable was made local.

Make sure to restore local variables before shellparam, so getopts state is
not clobbered.

sh: Ensure OPTIND=1 in subshell without forking does not affect outer env.

Command substitutions containing a single simple command and here-document
expansion are performed in a subshell environment, but may not fork. Any
modified state of the shell environment should be restored afterward.

The state that OPTIND=1 had been done was not saved and restored here.

Note that the other parts of shellparam need not be saved and restored,
since they are not modified in these situations (a fork is done before such
modifications).

sh: Avoid copying argv for simple commands.

Add dummy entries before and after so arglist's array is directly usable as
argv.

sh: Make struct arglist an array instead of a linked list.

This simplifies the code (e.g. allowing use of qsort(3) instead of a
hand-rolled mergesort) and should have better cache properties.

The waste of unused args arrays after resizes is approximately the same as
the savings from getting rid of the next pointers.

At the same time, remove a piece of global state and move some duplicated
code into a function.

sh: Fix more compiler warnings.

sh: Prefer "" to nullstr where possible.

sh: Remove EXP_REDIR.

EXP_REDIR was supposed to generate pathnames in redirection if exactly one
file matches, as permitted but not required by POSIX in interactive mode. It
is unlikely this will be implemented.

No functional change is intended.

MFC after: 1 week

sh: Remove special case for '=' in set -x; always quote it in outqstr().

I plan to make set -x output always printable using $'...'; avoiding quoting
words containing '=' is not worth the extra complexity.

sh: Make getopts memory-safe if with changing arguments.

POSIX does not permit to continuing a getopts loop with different
arguments. For parsing the positional parameters, we handle this case by
resetting the getopts state when the positional parameters are changed in
any way (and the getopts state is local to a function). However, in the
syntax getopts <optstring> <var> <arg...>, changes could lead to invalid
memory access.

In the syntax getopts <optstring> <var> <arg...>, store a copy of the
arguments and continue to use them until getopts is reset.

sh: Fix break/continue/return in multiline eval.

Example:
eval $'return\necho bad'

sh: Eliminate some gotos.

sh: Allow arbitrarily large numbers in break and continue.

The argument is capped to loopnest, so strtol()'s [ERANGE] can be ignored.

sh: Split set -x output into a separate function.

sh: Make return return from the closest function or dot script.

Formerly, return always returned from a function if it was called from a
function, even if there was a closer dot script. This was for compatibility
with the Bourne shell which only allowed returning from functions.

Other modern shells and POSIX return from the function or the dot script,
whichever is closest.

Git 1.8.4's rebase --continue depends on the POSIX behaviour.

Reported by: Christoph Mallon, avg

sh: Remove unnecessary reset functions.

These are already handled by exception handlers.

sh: Remove #define MKINIT.

MKINIT only served for the removed mkinit. Many variables can be static now.

sh: Remove mkinit.

Replace the RESET blocks with regular functions and a reset() function that
calls them all.

This code generation tool is unusual and does not appear to provide much
benefit. I do not think isolating the knowledge about which modules need to
be reset is worth an almost 500-line build tool and wider scope for
variables used by the reset functions.

Also, relying on reset functions is often wrong: the cleanup should be done
in exception handlers so that no stale state remains after 'command eval'
and the like.

sh: Do not close(-1) if pipe() fails.

sh: Don't modify exit status when break/continue/return passes !.

This matches what would happen if ! P were to be replaced with
if P; then false; else true; fi.

Example:
f() { ! return 0; }; f

sh: If a SIGINT or SIGQUIT interrupts "wait", return status 128+sig.

sh: Fix a crash with the stackmark code.

If a stack mark is set while the current stack block is empty, the stack
block may move later on (because of realloc()) and the stack mark needs to
be updated. This updating does not happen after popstackmark() has been
called; therefore, call setstackmark() again if the stack mark is still
being used.

For some reason, this only affects a few users. I cannot reproduce it. The
situation seems quite rare as well because an empty stack block would
usually be freed (by popstackmark()) before execution reaches a
setstackmark() call.

PR: 175922
Tested by: KT Sin

sh: Expand here documents in the current process.

Expand here documents at the same point other redirections are expanded but
use a non-fork subshell environment (like simple command substitutions) for
compatibility. Substitition errors result in an empty here document like
before.

As a result, a fork is avoided for short (<4K) expanded here documents.

Unexpanded here documents (with quoted end marker after <<) are not affected
by this change. They already only forked when >4K.

Side effects:
* Order of expansion is slightly different.
* Slow expansions are not executed in parallel with the redirected command.
* A non-fork subshell environment is subtly different from a forked process.

sh: Move some stackmarks to fix high memory usage in some loops.

If a loop contained certain commands (such as redirected compound commands),
the temporary memory for the redirection was not freed between iterations of
the loop but only after the loop.

Put a stackmark in evaltree(), freeing memory whenever a node has been
evaluated. Some other stackmarks are then redundant; remove them.

Example:
while :; do { :; } </dev/null; done

sh: Pass $? to command substitution containing compound/multiple commands.

Example:
false; echo $(echo $?; :)

sh: Detect and flag write errors on stdout in builtins.

If there is a write error on stdout, a message will be printed (to stderr)
and the exit status will be changed to 2 if it would have been 0 or 1.

PR: bin/158206

sh: Prefer internal nextopt() to libc getopt().

This reduces code duplication and code size.

/usr/bin/printf is not affected.

Side effect: different error messages when certain builtins are passed
invalid options.

sh: Expand assignment-like words specially for export/readonly/local.

Examples:
export x=~
now expands the tilde
local y=$1
is now safe, even if $1 contains IFS characters or metacharacters.

For a word to "look like an assignment", it must start with a name followed
by an equals sign, none of which may be quoted.

The special treatment applies when the first word (potentially after
"command") is "export", "readonly" or "local". There may be quoting
characters but no expansions. If "local" is overridden with a function there
is no special treatment ("export" and "readonly" cannot be overridden with a
function).

If things like
local arr=(1 2 3)
are ever allowed in the future, they cannot call a "local" function. This
would either be a run-time error or it would call the builtin.

This matches Austin Group bug #351, planned for the next issue of POSIX.1.

PR: bin/166771

sh: Use vfork in a few common cases.

This uses vfork() for simple commands and command substitutions containing a
single simple command, invoking an external program under certain conditions
(no redirections or variable assignments, non-interactive shell, no job
control). These restrictions limit the amount of code executed in a vforked
child.

There is a large speedup (for example 35%) in microbenchmarks. The
difference in buildkernel is smaller (for example 0.5%) but still
statistically significant. See
http://lists.freebsd.org/pipermail/freebsd-hackers/2012-January/037581.html
for some numbers.

The use of vfork() can be disabled by setting a variable named
SH_DISABLE_VFORK.

sh: Fix $? in the first command of a 'for'.

In the first command of a 'for', $? should be the exit status of the last
pipeline (command substitution in the word list or command before 'for'),
not always 0.

sh: Fix execution of multiple statements in a trap when evalskip is set

Before this fix, only the first statement of the trap was executed if
evalskip was set. This is for example the case when:
o "-e" is set for this shell
o a trap is set on EXIT
o a function returns 1 and causes the script to abort

Reviewed by: jilles
MFC after: 2 weeks

sh: Fix some bugs with exit status from case containing ;&.

Also, rework evalcase() to not evaluate any tree. Instead, return the
NCLISTFALLTHRU node and handle it in evaltree().

Fixed bugs:

* If a ;& list with non-zero exit status is followed by an empty ;; or final
list, the exit status of the case command should be equal to the exit
status of the ;& list, not 0.

* An empty ;& case should not reset $?.

sh: Fix two bugs with case and exit status:

* If no pattern is matched, POSIX says the exit status shall be 0 (even if
there are command substitutions).
* If a pattern is matched and there are no command substitutions, the first
command should see the $? from before the case command, not always 0.

sh: Do not force special builtins non-special in optimized command subst.

This is not necessary: errors are already caught in evalbackcmd() and
forcelocal handles changes to variables.

Note that this depends on r223024.

MFC after: 4 weeks

sh: Remove impossible evalskip check in 'for'.

sh: Reduce one level of evaltree() recursion when executing 'case'.
Free expanded case text before executing commands.
Remove impossible evalskip checks (expanding an argument cannot set
evalskip anymore since $(break) and the like are properly executed in a
subshell environment).

sh: Remove special support for background simple commands.

It expands the arguments in the parent shell process, which is incorrect.

sh: Add case statement fallthrough (with ';&' instead of ';;').

Replacing ;; with the new control operator ;& will cause the next list to be
executed as well without checking its pattern, continuing until a list ends
with ;; or until the end of the case statement. This is like omitting
"break" in a C "switch" statement.

The sequence ;& was formerly invalid.

This feature is proposed for the next POSIX issue in Austin Group issue
#449.

sh: Reduce unnecessary forks with eval.

The eval special builtin now runs the code with EV_EXIT if it was run
with EV_EXIT itself.

In particular, this eliminates one fork when a command substitution contains
an eval command that ends with an external program or a subshell.

This is similar to what r220978 did for functions.

sh: Save/restore changed variables in optimized command substitution.

In optimized command substitution, save and restore any variables changed by
expansions (${var=value} and $((var=assigned))), instead of trying to
determine if an expansion may cause such changes.

If $! is referenced in optimized command substitution, do not cause jobs to
be remembered longer.

This fixes $(jobs $!) again, simplifies the man page and shortens the code.

sh: Do parameter expansion before printing PS4 (set -x).

The function name expandstr() and the general idea of doing this kind of
expansion by treating the text as a here document without end marker is from
dash.

All variants of parameter expansion and arithmetic expansion also work (the
latter is not required by POSIX but it does not take extra code and many
other shells also allow it).

Command substitution is prevented because I think it causes too much code to
be re-entered (for example creating an unbounded recursion of trace lines).

Unfortunately, our LINENO is somewhat crude, otherwise PS4='$LINENO+ ' would
be quite useful.

sh: Fix $? in heredocs on simple commands.

PR: bin/41410

sh: Honour -n while processing -c string.

sh: Expand aliases after assignments and redirections.

sh: Avoid close(-1) when evaluating a multi-command pipeline.

Valgrind complains about this.

sh: Set $? to 0 for background commands.

For backgrounded pipelines and subshells, the previous value of $? was being
preserved, which is incorrect.

For backgrounded simple commands containing a command substitution, the
status of the last command substitution was returned instead of 0.

If fork() fails, this is an error.

sh: Allow EV_EXIT through function calls, make {...} <redir more consistent.

If EV_EXIT causes an exit, use the exception mechanism to unwind
redirections and local variables. This way, if the final command is a
redirected command, an EXIT trap now executes without the redirections.

Because of these changes, EV_EXIT can now be inherited by the body of a
function, so do so. This means that a function no longer prevents a fork
before an exec being skipped, such as in
f() { head -1 /etc/passwd; }; echo $(f)

Wrapping a single builtin in a function may still cause an otherwise
unnecessary fork with command substitution, however.

An exit command or -e failure still invokes the EXIT trap with the
original redirections and local variables in place.

Note: this depends on SHELLPROC being gone. A SHELLPROC depended on
keeping the redirections and local variables and only cleaning up the
state to restore them.

sh: Remove clearcmdentry()'s now unused argument.

sh: Remove special code for shell scripts without magic number.

These are called "shell procedures" in the source.

If execve() failed with [ENOEXEC], the shell would reinitialize itself
and execute the program as a script. This requires a fair amount of code
which is not frequently used (most scripts have a #! magic number).
Therefore just execute a new instance of sh (_PATH_BSHELL) to run the
script.

sh: Do not call exitshell() from evalcommand() unless evalcommand() forked
itself.

This ensures that certain traps caused by builtins are executed.

sh: Check readonly status for assignments on regular builtins.

An error message is written, the builtin is not executed, nonzero exit
status is returned but the shell does not abort.

This was already checked for special builtins and external commands, with
the same consequences except that the shell aborts for special builtins.

Obtained from: NetBSD

sh: Avoid side effects from builtins in optimized command substitution.

Change the criterion for builtins to be safe to execute in the same process
in optimized command substitution from a blacklist of only cd, . and eval to
a whitelist.

This avoids clobbering the main shell environment such as by $(exit 4) and
$(set -x).

The builtins jobid, jobs, times and trap can still show information not
available in a child process; this is deliberately permitted. (Changing
traps is not.)

For some builtins, whether they are safe depends on the arguments passed to
them. Some of these are always considered unsafe to keep things simple; this
only harms efficiency a little in the rare case they are used alone in a
command substitution.

sh: Don't do optimized command substitution if expansions have side effects.

Before considering to execute a command substitution in the same process,
check if any of the expansions may have a side effect; if so, execute it in
a new process just like happens if it is not a single simple command.

Although the check happens at run time, it is a static check that does not
depend on current state. It is triggered by:
- expanding $! (which may cause the job to be remembered)
- ${var=value} default value assignment
- assignment operators in arithmetic
- parameter substitutions in arithmetic except ${#param}, $$, $# and $?
- command substitutions in arithmetic

This means that $((v+1)) does not prevent optimized command substitution,
whereas $(($v+1)) does, because $v might expand to something containing
assignment operators.

Scripts should not depend on these exact details for correctness. It is also
imaginable to have the shell fork if and when a side effect is encountered
or to create a new temporary namespace for variables.

Due to the $! change, the construct $(jobs $!) no longer works. The value of
$! should be stored in a variable outside command substitution first.

sh: Make expansion errors in optimized command substitution non-fatal.
Command substitutions consisting of a single simple command are executed in
the main shell process but this should be invisible apart from performance
and very few exceptions such as $(trap).

sh: Code size optimizations to "stack string" memory allocation:
* Prefer one CHECKSTRSPACE with multiple USTPUTC to multiple STPUTC.
* Add STPUTS macro (based on function) and use it instead of loops that add
nul-terminated strings to the stack string.

No functional change is intended, but code size is about 1K less on i386.

sh: Code size optimizations to buffered output.

This is mainly less use of the outc macro.

No functional change is intended, but code size is about 2K less on i386.

sh: Reindent evaltree().

sh: Use iteration instead of recursion to evaluate semicolon lists.
This reduces CPU and memory usage when executing long lists (such
as long functions).

sh: Tweak some string constants to reduce code size.

* Reduce some needless differences.
* Shorten some error messages that should not happen.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

In the spirit of r90111, depend on c89 and remove the "STATIC" macro
and its usage.

Consistently use "STATIC" for all functions in order to be able to set
breakpoints with in a debugger. And use naked "static" for variables.

Noticed by: bde

sh: Fix exit status if return is used within a loop condition.

sh: Get rid of some magic numbers.

MFC after: 1 week

sh: Add a forgotten const.

sh: Return 0 from eval if no command was given.

This makes a difference if there is a command substitution.

To make this work, evalstring() has been changed to set exitstatus to 0 if
no command was executed (the string contained only whitespace).

Example:
eval $(false); echo $?
should print 0.

sh: Recognize "--" in . and exec.

Although "--" historically has not been required to be recognized for
certain special builtins that do not take options in POSIX, some other
implementations recognize options for them, requiring scripts to use "--" or
avoid operands starting with "-".

Operands starting with "-" can be avoided with eval by prepending a space,
and cannot occur with break, continue, exit, return and shift as they only
take numbers, nor with times as it does not take operands. With . and exec,
avoiding "-" is not so easy as it may require reimplementing the PATH
search; therefore the current proposal for POSIX is to require recognition
of "--" for them.

We continue to accept other strings starting with "-" as operands to . and
exec, and also "--" if it is alone to . (which would otherwise be invalid
anyway).

sh: Have only one copy of _PATH_STDPATH in the binary.

sh: Apply locale vars on builtins, recognize LC_MESSAGES as a locale var.

This allows doing things like LC_ALL=C some_builtin to run a builtin under a
different locale, just like is possible with external programs. The
immediate reason is that this allows making printf(1) a builtin without
breaking things like LC_NUMERIC=C printf '%f\n' 1.2

This change also affects special builtins, as even though the assignment is
persistent, the export is only to the builtin (unless the variable was
already exported).

Note: for this to work for builtins that also exist as external programs
such as /bin/test, the setlocale() call must be under #ifndef SHELL. The
shell will do the setlocale() calls which may not agree with the environment
variables.

sh: Do not abort on a redirection error on a compound command.

Redirection errors on subshells already did not abort the shell because
the redirection is executed in the subshell.

Other shells seem to agree that these redirection errors should not abort
the shell.

Also ensure that the redirections will be cleaned up properly in cases like
command eval '{ shift x; } 2>/dev/null'

Example:
{ echo bad; } </var/empty/x; echo good

sh: Do not abort on a redirection error if there is no command word.

Although simple commands without a command word (only assignments and/or
redirections) are much like special builtins, POSIX and most shells seem to
agree that redirection errors should not abort the shell in this case. Of
course, the assignments persist and assignment errors are fatal.

To get the old behaviour portably, use the ':' special builtin.
To get the new behaviour portably, given that there are no assignments, use
the 'true' regular builtin.

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

sh: Make sure to popredir() even if a function caused an error.

sh: Make sure to popredir() even if a special builtin caused an error.

sh: Improve the command builtin:
* avoid unnecessary fork
* allow executing builtins via command
* executing a special builtin via command removes its special properties

Obtained from: NetBSD (parts)

sh: Send the "not found" message for builtin <cmd> to redirected fd 2.

sh: Fix some bugs with backquoted builtins:
- correctly handle error output in $(builtin 2>&1), clarify out1/out2 vs
output/errout in the code
- treat all builtins as regular builtins so errors do not abort the shell
and variable assignments do not persist
- respect the caller's INTOFF

Some bugs still exist:
- expansion errors may still abort the shell
- some side effects of expansions and builtins persist

sh: Allow command -pv and command -pV (lookup using _PATH_STDPATH).

sh: Ensure funcnest is decremented if there was an error in the function.

This will be important when things like 'command eval f' will be possible.
Currently, the funcnest = 0 assignment in RESET (called when returning to
the top level after an error in interactive mode) is really sufficient.

sh: Various warning fixes (from WARNS=6 NO_WERROR=1):
- const
- initializations to silence -Wuninitialized (it was safe anyway)
- remove nested extern declarations
- rename "index" locals to "idx"

Fix some cases where file descriptors from redirections leak to programs.

- Redirecting fds that were not open before kept two copies of the
redirected file.
sh -c '{ :; } 7>/dev/null; fstat -p $$; true'
(both fd 7 and 10 remained open)
- File descriptors used to restore things after redirection were not
set close-on-exec, instead they were explicitly closed before executing
a program normally and before executing a shell procedure. The latter
must remain but the former is replaced by close-on-exec.
sh -c 'exec 7</; { exec fstat -p $$; } 7>/dev/null; true'
(fd 10 remained open)

The examples above are simpler than the testsuite because I do not want to
use fstat or procstat in the testsuite.

Fix various things about SIGINT handling:
* exception handlers are now run with interrupts disabled, which avoids
many race conditions
* fix some cases where SIGINT only aborts one command and continues the
script, in particular if a SIGINT causes an EINTR error which trumped the
interrupt.

Example:
sh -c 'echo < /some/fifo; echo This should not be printed'
The fifo should not have writers. When pressing ctrl+c to abort the open,
the shell used to continue with the next command.

Example:
sh -c '/bin/echo < /some/fifo; echo This should not be printed'
Similar. Note, however, that this particular case did not and does not work
in interactive mode with job control enabled.

sh: Ensure the same command input file is on top after executing a builtin.

This avoids weirdness when 'fc -e vi' or the like is done and there is a
syntax error in the file. Formerly an interactive shell tried to execute
stuff after the syntax error and exited.

This should also avoid similar issues with 'command eval' and 'command .'
when 'command' is implemented properly as in NetBSD sh.

Special builtins did not have this problem since errors in them cause the
shell to exit or to reset various state such as the current command input
file.

MFC r196483,r196634:
sh: Fix crash when undefining or redefining a currently executing function

Add a reference count to function definitions.
Memory may leak if a SIGINT arrives in interactive mode at exactly the wrong
time, this will be fixed later by changing SIGINT handling.

PR: bin/137640
Approved by: re (kib)

sh: Send the "xyz: not found" message to redirected fd 2.
This also fixes that trying to execute a non-regular file with a command
name without '/' returns 127 instead of 126.
The fix is rather simplistic: treat CMDUNKNOWN as if the command were found
as an external program. The resulting fork is a bit wasteful but executing
unknown commands should not be very frequent.

PR: bin/137659

sh: Fix crash with empty functions (f() { }) introduced in r196483

Empty pairs of braces are represented by a NULL node pointer, just like
empty lines at the top level.

Support for empty pairs of braces may be removed later. They make the code
more complex, have inconsistent behaviour (may or may not change $?), are
not specified by POSIX and are not allowed by some other shells like bash,
dash and ksh93.

Reported by: kan

sh: Fix crash when undefining or redefining a currently executing function.

Add a reference count to function definitions.
Memory may leak if multiple SIGINTs arrive in interactive mode,
this will be fixed later by changing SIGINT handling.

PR: bin/137640

Quote -x tracing output so it is unambiguous.

It is usually but not always suitable for re-input to the shell.

Approved by: ed (mentor) (implicit)

Do not fork for a subshell if it is the last thing this shell is doing
(EV_EXIT). The fork is still done as normal if any traps are active.

In many cases, the fork can be avoided even without this change by using {}
instead of (), but in practice many scripts use (), likely because the
syntax is simpler.

Example:
sh -c '(/bin/sleep 10)& sleep 1;ps -p $! -o comm='
Now prints "sleep" instead of "sh". $! is more useful this way.
Most shells (dash, bash, pdksh, ksh93, zsh) seem to print "sleep" for this.

Example:
sh -c '( ( ( (ps jT))))'
Now shows no waiting shell processes instead of four.
Most shells (dash, bash, pdksh, ksh93, zsh) seem to show zero or one.

PR: bin/74404
Approved by: ed (mentor) (implicit)

sh: Improve handling of setjmp/longjmp volatile:
- remove ineffective and unnecessary (void) &var; [1]
- remove some unnecessary volatile keywords
- add a necessary volatile keyword
- save the old handler before doing something that could use the saved
value

Submitted by: Christoph Mallon [1]
Approved by: ed (mentor)

Avoid leaving unnecessary waiting shells in many forms of sh -c COMMAND.

This change only affects strings passed to -c, when the -s
option is not used.

The approach is to check if there may be additional data
in the string after parsing each command. If there is none,
use the EV_EXIT flag so that a fork may be omitted in
specific cases.

If there are empty lines after the command, the check will
not see the end and forks will not be omitted. The same
thing seems to happen in bash.

Example:
sh -c 'ps lT'
No longer shows a shell process waiting for ps to finish.

PR: bin/113860
Reviewed by: stefanf
Approved by: ed (mentor)

Don't skip forking for an external command if any traps are active.

Example:
sh -c '(trap "echo trapped" EXIT; sleep 3)'
now correctly prints "trapped".

With this check, it is no longer necessary to check for -T
explicitly in that case.

This is a useful bugfix by itself and also important because I plan to
skip forking more often.

PR: bin/113860 (part of)
PR: bin/74404 (part of)
Reviewed by: stefanf
Approved by: ed (mentor)

correctly test for __GNUC__ macro (non-GCC compilers do not have it defined at all)

Fix the eval command in combination with set -e. Before this change the shell
would always terminate if eval returned with a non-zero exit status regardless
if the status was actually tested. Unfortunately a new file-scope variable
is needed, the alternative would only be to add a new parameter to all
built-ins.

PR: 134881

Don't let trailing empty lines overwrite the result of the last command with 0.
This affects the built-ins eval, fc, and trap and also the string passed to sh
with the -c option.

Submitted by: Jilles Tjoelker

Fix $? at the first command of a function. The previous exit status was saved
twice and thus lost.

- Import the HEAD csup code which is the basis for the cvsmode work.

Pass the correct flags to expandarg() for NFROMFD and NTOFD. This fixes a
segmentation fault when the argument expands to an empty string.

Reported by: simon
MFC after: 3 weeks

The exit status of a case statement where none of the patterns is matched
is supposed to be 0, not the status of the previous command.

Reported by: Eygene Ryabinkin
PR: 116559
Approved by: re (gnn)

Merge NetBSD's revision 1.86: Don't crash on "<cmd> | { }".

Implement the PS4 variable which is defined by the POSIX User Portability
Utilities option. Its value is printed at the beginning of the line if tracing
(-x) is active. PS4 defaults to the string "+ " which is compatible with the
old behaviour to always print "+ ".

We still need to expand variables in PS1, PS2 and PS4.

PR: 46441 (part of)
Submitted by: schweikh
Obtained from: NetBSD

Implement some of the differences between special built-ins and other builtins
demanded by POSIX.
- A redirection error is only fatal (meaning the execution of a shell script is
terminated) for special built-ins. Previously it was fatal for all shell
builtins, causing problems like the one reported in PR 88845.
- Variable assignments remain in effect for special built-ins.
- Option or operand errors are only fatal for special built-ins.
This change also makes errors from 'fc' non-fatal (I could not find any reasons
for this behaviour).

Somewhat independently from the above down-grade the error handling in the
shift built-in if the operand is bigger than $# from an error() call (which is
now fatal) to a return 1. I'm not sure if this should be considered a POSIX
"operand error", however this change is needed for now as we trigger that error
while building libncurses. Comparing with other shells, zsh does the same as
our sh before this change (write a diagnostic, return 1), bash behaves as our
sh after this commit (no diagnostic, return 1) and ksh93 and NetBSD's sh treat
it as a fatal error.

Remove some white space at EOL.

Add the times builtin. It reports the user and system time for the shell
itself and its children. Instead of calling times() (as implied by POSIX) this
implementation directly calls getrusage() to get the times because this is more
convenient.

Add the POSIX options -v and -V to the 'command' builtin. Both describe the
type of their argument, if it is a shell function, an alias, a builtin, etc.
-V is more verbose than -v.

PR: 77259, 84539

Pass the EV_TESTED flag to evalloop() and evalfor(). This fixes unwanted
termination with set -e if a command fails in a loop body inside a function
with an explicitely tested exit status, eg

f() {
for i in 1 2 3; do
false
done
}
f || true

Briefly reviewed by: cracauer

Pass the EV_TESTED flag to the left operand of NSEMI nodes. This fixes
two cases of unwanted termination with set -e:

* if-commands containing several commands separated by semicolons, eg

if false; false; then [...]

* functions with an explicitely tested exit status that contain a failing
command which is not the last one, eg

f() {
false
false
}
f || true

PR: 77067, 85267
Briefly reviewed by: cracauer

Exit the shell if a pipeline that is not preceded by ! fails and set -e is
active. Use a separate flag to avoid adding another condition to the
if-statement at the end of evaltree().

Briefly reviewed by: cracauer

Print pointers with %p rather than casting them to long.

Remove clause 3 from the UCB licenses.

OK'ed by: imp, core

The PR diff I committed recently had one bug noticed by
Joe Marcus Clarke <marcus@FreeBSD.ORG>, subshells could lose a
non-zero exit status.

This commit is Joe's proposed patch. Thanks!

I verified that the problem Joe found is fixed and I ran a full world
with this patch.

I don't plan to ever commit language patches to /bin/sh again. It is
a minefield too big to navigate without a full-time committment, which
I am not willing to do on our /bin/sh.

Under normal circumstances I would recommend using NetBSD's sh which
has a lot of language fixes (like the ones what these patches were
about) but unfortunately they had implemented broken signal behaviour
for shellscript containing interactive programs. Similar issues apply
to pdksh which is OpenBSD's sh.

From my perspective bash2 is the only really working bourne sh out
there and that one is GPLed. Oh well.

Replace home-grown dup2() implementation with actual dup2() calls. This
should slightly reduce the number of system calls in critical portions of
the shell, and select a more efficient path through the fdalloc code.

Reviewed by: bde

o Spell 'diretories' correctly.

PR: misc/60730
Submitted by: Eugene Grosbein
MFC after: 3 days

PR 28852

sh -e behaviour was incorrect when && and || statements where used in
"if" clauses.

This is the patch submitted by MORI Kouji <mori@tri.asanuma.co.jp>.

It fixes the issue at hand, but sh fixes like this are super-hard to
verify that they don't break anything else. I ran some of my old test
cases and a few big GNU configure scripts that detected mistakes
before, with the previous sh, patched sh and bash. No differences in
behaviour found. MFC recommended after longer than usual time.

Compiles on i386 and sledge.

Make this compile with DEBUG defined now that WARNS=0 has been removed
from the Makefile:
- Print pointers with %p instead of %x.
- Include missing headers to get prototypes.

Noticed by: benno

Add <stdlib.h> to get a prototype for exit().

Reviewed by: tjr

Set opterr to zero to avoid duplicate warnings from getopt(3) for unknown
options.

Implement the P1003.2 `command' builtin command, which is used to suppress
shell function and alias lookup. The -p option has been implemented, the
UPE -v and -V options have not. The old `command' command has been renamed
to `builtin'.

Consistently use FBSDID

Implement the -C (-o noclobber) option, which prevents existing regular
files from being overwritten by shell redirection.

o __P has been reoved
o Old-style K&R declarations have been converted to new C89 style
o register has been removed
o prototype for main() has been removed (gcc3 makes it an error)
o int main(int argc, char *argv[]) is the preferred main definition.
o Attempt to not break style(9) conformance for declarations more than
they already are.
o Change
int
foo() {
...
to
int
foo(void)
{
...

Fix how /bin/sh handles 'for' and 'case' statements when it is called to do
errexit (-e) processing. This solves a problem where 'make clean' would
fail with an unspecified error in certain automake-generated makefiles.

Reviewed by: no objections from -hackers...
MFC after: 2 weeks

Implement the <> redirection operator.

Fix environment passung to eval'ed commands.

PR: bin/6577
Submitted by: Anatoly Vorobey <mellon@pobox.com>
Approved by: silence amoung other sh experts

Fix command hash handling on
PATH=... command

Noted by and fix works for Marcel Moolenaar <marcel@scc.nl>

Include strerror(errno) in error messages after failed system calls.
Fix a warning.

The pipleline problem introduced in version 1.22 wasn't fully fixed in
1.23. This revision should work for long pipes both in
here-documents/backquote and in normal cases.

Fix works for jmz, bde.

Correct logic from previous commit - middle processes in long pipes
don't have their stdin closed indisciminantly.

Reviewed by: markm

When a backquote command inside a here-document had a pipe with more
than two processes (got that? :-), the stdin fd of the middle
processes that has just been set up was accidetially closed. Don't do
this.

PR: bin/14527

Fix ';' command when used with -e flag.

PR: bin/14509

$Id$ -> $FreeBSD$

Fix handling of the cd command inside evaluations. It was changing PWD,
which it should not do.

PR: 12578
Reported by: Gerald Pfeifer <pfeifer@dbai.tuwien.ac.at>
Submitted by: Niall Smart <niall@pobox.com>

Various spelling/formatting changes.

Submitted by: Philippe Charnier <charnier@xp11.frmug.org>

Next approach to make loops in interactive interruptable.

PR: bin/9173

In interactive shells, break loops to the topmost level when a child
is killed by a signal.

(In non-interactive shells - that means a shellscript - the shell just
exits, this was already working)

PR: bin/9173

The immediate execution of traps I introduced in September 1998 (to
make /etc/rc interruptible in cases when programs hang with blocked
signals) isn't standard enough.

It is now switched off by default and a new switch -T enables it.

You should update /etc/rc to the version I'm about to commit in a few
minutes to keep it interruptible.

Add rcsid. Spelling.

When calling a shell function, remember whether exit status is tested.
This is needed for the '-e' option. See the PR for more details.
PR: 6047
Reviewed by: PR submitter, silence on review request.

Use the __unused attribute where warranted.

Nuke register keyword usage and #if -> #ifdef.

Obtained from: NetBSD

Revert $FreeBSD$ to $Id$

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

Merge in NetBSD mods and -Wall cleaning.

Obtained from: NetBSD, me

Back out rev 1.7 which was to fix PR#1206 (to be reopened).

This patch causes too many side effects, one of which bites hard is
when interrupting a 'make fetch' in the ports tree (PR#1990).
This whole area is a real can of worms....

This most definately should go into 2.2

Reviewed by: steve, bde

Close PR# 1206. sh(1) now ignores SIGINT and SIGQUIT
when a child is forked and the -c commandline switch
has been specified.

Reviewed by: joerg

Fix a problem with sh that conflicts with the -e
flag text in the man page. Now this will work
correctly:

/bin/sh -ec 'false && true; echo hi'

Obtained from: VaX#n8 <vax@linkdead.paranoia.com>

Merge of 4.4-Lite2 sh source, plus some gcc -Wall cleaning. This is a
merge of parallel duplicate work by Steve Price and myself. :-]

There are some changes to the build that are my fault... mkinit.c was
trying (poorly) to duplicate some of the work that make(1) is designed to
do. The Makefile hackery is my fault too, the depend list was incomplete
because of some explicit OBJS+= entries, so mkdep wasn't picking up their
source file #includes.

This closes a pile of /bin/sh PR's, but not all of them..

Submitted by: Steve Price <steve@bonsai.hiwaay.net>, peter

Import the 4.4BSD-Lite2 /bin/sh sources

Requested by: joerg

(Note, this is mostly going to be conflicts, which is expected. Our entire
sh source has a mainline, so this should not change anything except for
a few new files appearing. I dont think they are a problem)

Don't dereference a NULL pointer in the case of a null pipe.
e.g.:
ls |> foo.out

sh now behaves the same as it does under SunOS 4.x for this case.

Remove trailing whitespace.

Reviewed by: phk

Added $Id$

BSD 4.4 Lite bin Sources