Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r245525:

Add a conditional sleep 1 in case we add any IPv6 addresses to interfaces.
Do this per jail started, not per address. This will allow DAD to complete
and services to properly start. Before we have seen problems with services
trying to start before the IPv6 address was available to use and thus
erroring and failing to start.

MFC r243080:
Only pass ip[46].addr when _addrl contains a value

Approved by: cperciva (implicit)

MFC: r240336:
Simply things so that "#REQUIRE: FILESYSTEMS" means the file
systems are fully "ready to go".

'FILESYSTEMS' states: "This is a dummy dependency, for services which
require file systems to be mounted before starting." However, we have
'var' which is was run after 'FILESYSTEMS' and can mount /var if it
already isn't mounted. Furthermore, several scripts cannot use /var
until 'cleanvar' has done its thing. Thus "FILESYSTEMS" hasn't really
meant all critical file systems are fully usable.

MFC r239382:

- Allow to pass extra parameters for each jails.
- To achieve above, convert jail(8) invocation to use new style
command line "-c" flag.

MFH (r238102, r238605): automatically name jails.

MFC r230099:

Change rcvar= assignments to the literal values set_rcvar
would have returned. This will slightly reduce boot time,
and help in diff reduction to HEAD.

MFC r230403.

When creating the jails /dev/log symlink, do it by full path to avoid
creating stray "log" symlinks if the mount fails. That apparently
happens in some ezjail configs.

PR: conf/143084
Submitted by: Dirk Engling <erdgeist at erdgeist.org>

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Replace ${SYSCTL_W} with ${SYSCTL} in rc.d scripts, as they are identical.
This is a further clean up after r202988.

SYSCTL_W is still initialized in rc.subr as some ports may still use it.

Implement the idea of parallel-only-at-start-time in a cleaner, more
rc.d'ish way.

Not objected to by: netchild

Redirect stdin from /dev/null when starting a jail:
At least in RELENG_7 this fixes some start problems for some programs
from the ports. It is also more correct, as a jail shall not expect
input (interactivity) from the jail-host.

Revert the current behavior of starting jails in the background and
make it optional only for the start of jails (jail_parallell_start=YES
in rc.conf):
- The stop can not be done in the background, the system needs to wait
until everything is stopped correctly before it can reboot or power
down.
- The start should not be done in parallel by default, this not only
breaks POLA for people comming from RELENG_x, it may also break a
dependency chain with other scripts in the jail-host, which need to
do some stuff after the jails are up and running (e.g. hardlinking
a mysql socket from one jail into another one).

Discussed on: freebsd-jails@

Execute the start/stop process of a jail in the background.
This will prevent that the script hangs during startup, which
could cause annoying effects after rebooting for example.

PR: kern/139422
Submitted by: Andrey Groshev <greenx at yartv dot ru>
Approved by: imp (mentor, implicit)
MFC after: 3 days
Facilitated by: Snow B.V.

Small cleanups to the jail script:

- Remove redundant debugging of consolelog.
- Use `while :', instead of `while [ true ]'. This is done in other
places as well.

Submitted by: Jille Timmermans <jille quis cx> (not jilles)
Reviewed by: jilles

Added (pre|post)(start|stop) jail hooks. These can be used to run
arbitrary commands (outside the jail) associated with said events,
e.g. to bring up/down CARP interfaces representing services run in
jails.

Reviewed by: simon

Update jail startup script for multi-IPv4/v6/no-IP jails.

Note: this is only really necessary because of the ifconfig
logic to add/remove the jail IPs upon start/stop.
Consensus among simon and I is that the logic should
really be factored out from the startup script and put
into a proper management solution.

- We now support starting of no-IP jails.
- Remove the global jail_<jname>_netmask option as it is only
helpful to set netmasks/prefixes for the right address
family and per address.
- Implement jail_<jname>_ip options to support both
address familes with regard to ifconfig logic.
- Implement _multi<n> support suffix to the jail_<jname>_ip
option to configure additional addresses to avoid overlong,
unreadbale jail_<jname>_ip lines with lots of addresses.

Submitted by: initial work from Ruben van Staveren
Discussed on: freebsd-jail in Nov 2008.
Reviewed by: simon, ru (partial, older version)
MFC after: 1 week

Allow a jail's IP alias to be created with an arbitrary netmask.

MFC after: 3 days

Allow a jail to be started with a specific route fib.

Reviewed by: secteam (simon)
Reviewed by: brooks, bz

As previously discussed, add the svn:executable property to all scripts

Add warning about this script dealing with untrusted data.

MFC after: 1 week

Fix indentation.

Fix jail rc.d script privilege escalation via symlink attack against
/var/log/console.log and mount points.

Security: FreeBSD-SA-07:01.jail

Jail_ip and jail_interface local variables were renamed to _ip and _interface
in a previous commit to avoid namespace collisions, unfortunately I missed two
of them. This leads to the ip alias being incorrectly removed in some cases
when using the stop command.

Reported by: Philipp Wuensche <cryx-freebsd@h3q.com>

Add jail_<jname>_exec_afterstart<N> rc.conf variable, where <N> is
1,2 and so on.
It specifies the command to be run as Nth after jail startup.

sh(1)-fu by: Dario Freni
PR: conf/97697
MFC after: 2 weeks
Reviewed by: ru@ (man page)

style(9)

- Change the "jail_" prefix for internal script variables. This fixes an
issue where some global jail_* variables were overriden in the script. [1]
- Change "jid" to "jname" in rc.conf(5), since it's more a jail name than a
jail id. [1]
- Update examples and comments in defaults/rc.conf to advertise new
variables and the fact that some of the jail-specific variables may be made
jail-global. [2]

Reported by: pjd [1], clsung [2]
Approved by: cperciva
X-MFC after: i got sufficient testing from people using rc.d/jail

if we fail to start a jail and jail_foobar_*fs_enable or jail_foobar_mount_enable were set, umount those filesystem before exiting. If we set up an alias for jail's IP, remove that alias before exiting.

MFC after: 2 weeks

if a jail fails to start, don't add its jid to /var/run and print a message with the error.

PR: conf/97024
MFC after: 1 week

- Fix quoting.

Reported by: Dirk Engling <erdgeist@erdgeist.org>
Pointyhat to: self

- Check for some mandatory variables.

Approved by: cperciva (mentor)
MFC after: 1 week

- Add following global jail options, used if no jail-specific options are
set:
* jail_mount_enable
* jail_devfs_ruleset
* jail_devfs_enable
* jail_fdescfs_enable
* jail_procfs_enable
* jail_fstab
* jail_flags
- Add a jail_interface / jail_<jid>_interface option. An ip alias will be
created (jail_<jid>_ip) on jail_interface or jail_<jid>_interface if set.
This is not a mandatory option.
- Document all missing jail_* options in rc.conf(5).

Approved by: cperciva (mentor)
MFC after: 2 weeks

Fix overriding jail_list from command line.

MFC after: 3 days

Skip jails which are already running and inform why.
We're checking for /var/run/jail_<name>.id file and if it exists, we don't
start the jail. It should be also safe in case of reboot(8), because
rc.d/cleanvar script is going to remove /var/run/jail_* files.

It helps to avoid potential mess when the same jail is started twice,
because of an administrator mistake (been there, done that).

MFC after: 1 week

Allow to give more than one jail's name, eg.:

# /etc/rc.d/jail start www mail

MFC after: 3 days

Introduce new per-jail variable jail_<name>_flags, which allows to specify
jail(8) flags (before the change we had hardcoded "-l -U root").

Submitted by: Frank Behrens <frank@pinky.sax.de>
PR: conf/80244
Approved by: re (scottl)
MFC after: 1 week

Do not unconditionally mount devfs to ${jail_devdir}/dev. First check
to see if a prior devfs has been mounted. If no devfs is mounted on
${jail_devdir}/dev then proceed. This will prevent the stack up of
multiple devfs mounts on the same mount point.

Discussed with: pjd
MFC after: 1 week

"REQUIRE: cleanvar" for all RC's writing into /var/run.

Improve the RC framework for the clean booting/shutdown of Jails:

1. Feature: for flexibility reasons and as a prerequisite to clean
shutdowns, allow the configuration of a stop/shutdown command
via rc.conf variable "jail_<name>_exec_stop" in addition to the
start/boot command (rc.conf variable "jail_<name>_exec_start"). For
backward compatibility reasons, rc.conf variable "jail_<name>_exec"
is still supported, too.

2. Debug: Add the used boot/shutdown commands to the debug output of
the /etc/rc.d/jail script, too.

3. Security: Run the Jail start/boot command in a cleaned environment
to not leak information from the host to the Jail during startup.

4. Feature: Run the Jail stop/shutdown command "jail_<name>_exec_stop" on
"/etc/rc.d/jail stop <name>" to allow a graceful shutdown of the Jail
before its processes are just killed.

5. Bugfix: When killing the remaining Jail processes give the processes
time to actually perform their termination sequence. Without this the
subsequent umount(8) operations usually fail because the resources
are still in use. Additionally, if after trying to TERM-inate the
processes there are still processes hanging around, finally just KILL
them.

6. Bugfix: In rc.shutdown, if running inside a Jail, skip the /etc/rc.d/*
scripts which are flagged with the KEYWORD "nojail" to allow the
correct operation of rc.shutdown under jail_<name>_exec_stop="/bin/sh
/etc/rc.shutdown". This is analogous to what /etc/rc does inside a Jail.

Now the following typical host-configuration for two Jails works as
expected and correctly boots and shutdowns the Jails:

-----------------------------------------------------------
# /etc/rc.conf:
jail_enable="YES"
jail_list="foo bar"
jail_foo_rootdir="/j/foo"
jail_foo_hostname="foo.example.com"
jail_foo_ip="192.168.0.1"
jail_foo_devfs_enable="YES"
jail_foo_mount_enable="YES"
jail_foo_exec_start="/bin/sh /etc/rc"
jail_foo_exec_stop="/bin/sh /etc/rc.shutdown"
jail_bar_rootdir="/j/bar"
jail_bar_hostname="bar.example.com"
jail_bar_ip="192.168.0.2"
jail_bar_devfs_enable="YES"
jail_bar_mount_enable="YES"
jail_bar_exec_start="/path/to/kjailer -v"
jail_bar_exec_stop="/bin/sh -c 'killall kjailer && sleep 60'"
-----------------------------------------------------------
# /etc/fstab.foo
/v/foo /j/foo/v/foo nullfs rw 0 0
-----------------------------------------------------------
# /etc/fstab.bar
/v/bar /j/bar/v/bar nullfs rw 0 0
-----------------------------------------------------------

Reviewed by: freebsd-hackers
MFC after: 2 weeks

Fix a typo in an error message.

Spotted by: ceri

Implement per-jail fstab(5) files. Here's a rc.conf sample using
this feature for a jail named foo :

jail_foo_mount_enable="YES"
jail_foo_fstab="/etc/fstab.foo"

The second line is actually useless, since the code defaults to
using "/etc/fstab.$jailname" as the fstab file if none is specified.

MFC after: 3 days
Submitted by: Jeremie Le Hen <jeremie@le-hen.org>

Remove the requirement for the FreeBSD keyword as it no longer
makes any sense.

Discussed with: dougb, brooks
MFC after: 3 days

Always quote variables in tests, to ensure correct evaluation even when
they are empty or undefined.

MFC after: 3 days

Create temporary files safely.

Submitted by: Jon Passki <cykyc@yahoo.com>

Mark scripts as not usable inside a jail by adding keyword 'nojail'.

Some suggestions from: rwatson, Ruben de Groot <mail25@bzerk.org>

o Unbreak the individual jail starting patch that I broke when
I committed it. Apologies to Juergen Unger <j.unger@addict.de>.
o When stopping jails output the hostname of the jails that
were stopped.
o Refactor
o Remove extraneous empty line
o Correct spelling error

Configure a jail sysctl value only if it is different than
what the rc.conf(5) knob specifies. Also, correct a minor
capitalization error.

Support starting/stoping of jails individually.

This commit also removes the support for the sysutils/jailer port. This
is inline with the general policy to keep ports related knobs out
of the base system's configuration mechanism.

Submitted by: Juergen Unger <j.unger@addict.de>

o Rename devfs_link() to make_symlink() and turn it into a generic
symlinking routine.
o Modify rc.d/jail to create its own symlink relative to the jail's
filesystem

If fdescfs(5) is mounted in a jail unmount it before devfs(5).

Submitted by: "Scot W. Hetzel" <hetzelsw@westbend.net>

Redirect rc(8) output to a file in the jail's /var/log directory.
Previously it was redirected to /dev/null.

Submitted by: phk

Remove trailing whitespace

Enhance the jail start/stop script.
o The following additional configuration attributes of a jail can be
controlled from rc.conf:
- mounting devfs(5)
- mounting fdescfs(5)
- mounting procfs(5)
- custom devfs(8) ruleset
If no ruleset is specified, the default jail ruleset is used.

o The output of executing /etc/rc in the jail is now redirected
to /dev/null. Instead, the hostname of the jail is echoed if
the jail(8) command exited successfully. If the output is wanted
it can probably be redirected to a file (/var/run/$jail maybe)
instead of /dev/null.

Submitted by: Scot W. Hetzel <hetzels@westbend.net>
with modifications by Jens Rehsack <rehsack@liwing.de>
and me.

Quote $jail_exec with double-quote to avoid "[: unexpected operator".

Move securelevel further back in the boot order.

Approved by: markm (mentor)(implicit)
Reviewed by: dougb

Spell ${_jail} correctly

Submitted by: Shizuka Kudo <shizukakudo_99@yahoo.com>
Approved by: markm (mentor) (implicit)

Fix typo (and make it more explicit/self-documenting).

Noticed by: gshapiro

o Add a script to start jails on boot.
o Hook it up to the build

Approved by: markm (mentor)
Submitted by: Clement Laforet <sheepkiller@cultdeadsheep.org> (modified)
Prodded by: mike