jail: add the -C flag to clean up after a partially removed jail

Differential Revision: https://reviews.freebsd.org/D42670

Add a missing space in a warning.

jail(8): add support for ZFS datasets

Add zfs.dataset to jail(8) to add a list of ZFS datasets.
Bump FreeBSD version for jail managers to switch to native
dataset support.

Datasets are attached to the jail after the jail creation and
before the execution of any start command. Unlike current
implementations in jail managers which attach datasets after
the start command, this allows the zfs rc.d script to mount
the datasets on start.

Discussed with: jamie

Unbreak build from ed31b3f4a146 (misapplied diff).

Differential Revision: <https://reviews.freebsd.org/D28150

jail: Don't allow jail_set(2) to resurrect dying jails.

Currently, a prison in "dying" state (removed but still holding
resources) can be brought back to alive state via "jail -d", or
the JAIL_DYING flag to jail_set(2). This seemed like a good idea
at the time.

Its main use was to improve support for specifying the jid when
creating a jail, which also seemed like a good idea at the time.
But resurrecting a jail that was partway through thr process of
shutting down is trouble waiting to happen.

This patch deprecates that flag, leaving it as a no-op for creating
jails (but still useful for looking at dying jails). It sill allows
creating a new jail with the same jid as a dying one, but will renumber
the old one in that case. That's imperfect, but allows for current
behavior.

Reviewed by: bz
Differential Revision: https://reviews.freebsd.org/D28150

usr.sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

Move "jail -e" out of the rest of jail(8) flow.

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Mark usage function as __dead2 in programs where it does not return

In most cases, usage does not return, so mark them as __dead2. For the
cases where they do return, they have not been marked __dead2.

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/735

jail: Use re-entrant versions of lex & yacc, and lex's yylineno

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

jail: handle jailsys parameters in modification permission test

Avoid a null dereference when a value-less jailsys parameter is passed
to "jail -m". There was already code to handle boolean parameters,
but in reality any parameter could be passed without a value.

jail: fix jail(8) synposis and usage message to match reality.

Reported by: yuri
PR: 254741
MFC after: 5 days

jail: Add exec.prepare and exec.release command hooks

This change introduces new jail command hooks that run before and after any
other actions.

The exec.prepare hook can be used for example to invoke a script that checks
if the jail's root exists, creating it if it does not. Since arbitrary
variables in jail.conf can be passed to the command, it can be pretty useful
for templating jails.

An example use case for exec.release would be to remove the filesystem of an
ephemeral jail.

The names "prepare" and "release" are borrowed from the names of similar hooks
in libvirt.

Reviewed by: jamie, manpages, mmacy
Approved by: mmacy (mentor)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D24829

jail(8): improve manual and usage information with more clear
description for "jail -e" mode to show that it does not take
additional jail name argument.

Reported by: David Marec <david.marec@davenulle.org>
MFC after: 3 days

jail(8): introduce new command option -e to exhibit
a list of configured non-wildcard jails with their parameters,
no matter running or not.

The option -e takes separator argument that is used
to separate printed parameters. It will be used with following
additions to system periodic scripts to differentiate parts
of directory tree belonging jails as opposed to host's.

MFC after: 1 month

security.jail.enforce_statfs is handled by jail_set(2), so handling it in
userspace jail(8) is redundant.

Differential Revision: D14791

Don't let clobber jailparam values when checking for modification of
init-only parameters.

Compare string parameter values with strncmp, not memcmp.

PR: 230487
Reported by: Jason Mader
MFC after: 3 days

- Add exec hook "exec.created". This is called when the jail is
created and before exec.start is called. [1]
- Bump __FreeBSD_version.

This allows to attach ZFS datasets and various other things to be
done before any command/service/rc-script is started in the new
jail.

PR: 228066 [1]
Reviewed by: jamie [1]
Submitted by: Stefan Grönke <stefan@gronke.net> [1]
Differential Revision: https://reviews.freebsd.org/D15330 [1]

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

Strip EOL whitespace in usr.sbin/{jail,jexec}

Don't assign rtjp twice.

Reported by: clang static analyzer
MFC after: 2 weeks

Add mount.procfs jail parameter, so procfs can be mounted when a prison's
root is in its fstab.

Also fix a typo while I'm at it.

PR: 197237 197066
MFC after: 3 days

In preparation for using clang's -Wcast-qual:

Use __DECONST (instead of my own attempted re-invention) for the iov
parameters to jail_get/set(2). Similarly remove the decost-ish hack
from execvp's argv, except the __DECONST is only added at very end.

While I'm at it, remove an unused variable and fix a comment typo.

- Add mount.fdescfs parameter to jail(8). This is similar to
mount.devfs but mounts fdescfs. The mount happens just after
mount.devfs.

- rc.d/jail now displays whole error message from jail(8) when a jail
fails to start.

Approved by: re (gjb)

- Update rc.d/jail to use a jail(8) configuration file instead of
command line options. The "jail_<jname>_*" rc.conf(5) variables for
per-jail configuration are automatically converted to
/var/run/jail.<jname>.conf before the jail(8) utility is invoked.
This is transparently backward compatible.

- Fix a minor bug in jail(8) which prevented it from returning false
when jail -r failed.

Approved by: re (glebius)

Pre-separate IP addresses passed on the command line, so they can be
properly parsed for interface prefixes and netmask suffixes. This was
already done for the old-style (fixed) command line, but missed for
the new-style.

MFC after: 1 week

o Restore -u <username> getopt(3) flag somehow killed in r234712.

PR: bin/169490
Submitted by: amdmi3
MFC after: 2 weeks

When writing the jid via the -i flag, do it right when the jail is created,
before any commands run. /etc/rc.d/jail depends on this.

Add a meta-parameter IP__NULL to enum intparam, instead of mixing
enum values and zeroes. This keeps clang happy (and is just good form).

Submitted by: dim

Improvements in error messages:

Some errors printed the jail name for unnamed (command line) jails.

Attempting to create an already-existing jail from the command line
returned with no error (even for non-root) due to bad logic in
start_state.

Ignore kvm_proc errors, which are typically caused by permission
problems. Instead, stop ignoring permission errors when removing
a jail (but continue to silently ignore other errors, i.e. the
jail no longer existing). This makes non-root attempts at removing
a jail give a clearer error message.

Allow relative pathnames for jails generated on the command line
(but continue to flag when from a config file).

Try resolving jail path with realpath(3).

jail(8) does a chdir(2) to the given path argument. Kernel evaluates the
jail path from the new cwd and not from the original cwd, which leads to
undesired behavior if given a relative path.

Reviewed by: jamie
MFC after: 2 weeks

Advance to the next command before running anything, so errors found in
finish_command can be processed properly.
Call failed() once in next_command() instead of multiple times in
run_command().
Continue processing commands when a no-wait operation (IP__OP or background
command) succeeds.

Following r222465:

Check for IPv4 or IPv6 to be available by the kernel to not
provoke errors trying to query options not available.
Make it possible to compile out INET or INET6 only parts.

Move the actual create/remove (IP__OP) handling into run_command,
and the cost of an ugly single-use global variable.

Update copyright dates and other whitespacey stuff.

Split run_command up into an outer function (next_command) that chooses
a single command string to run, and an inner function (run_command) that
runs that single string.
Move the list of start/stop commands to run from a switch statement into
an array, with a new placeholder parameter IP__OP for actually creating
or removing the jail.
When jail creation fails, revert all non-exec commands in reverse order.

Change cfstrings from an STAILQ into a TAILQ to allow commands to be
traversed in reverse order.

Check for IPv4 or IPv6 to be available by the kernel to not
provoke errors trying to query options not available.
Make it possible to compile out INET or INET6 only parts.

Reviewed by: jamie
Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems
MFC after: 10 days

run_command (mostly) cleanup:

Make the parallelism limit a global instead of always passing it
to run_command and finish_command.
In the case of an empty command string, try to run any other strings
the command may have.
Replace JF_BACKGROUND with its sort-of opposite JF_SLEEPQ.
Change j->comstring earlier to render JF_RUNQ unncessary.
Change the if-else series to a more readable switch statement.
Treat IP_STOP_TIMEOUT like a command, calling run_command which then
calls term_procs.
When the IP_STOP_TIMEOUT "command" finishes, it shouldn't mess with
the parallelism limit.
Make sufficient checks in finish_command and run_command so that
the nonintuitive j->comstring null check isn't necessary to run them.
Rename the "waiting" queue to "depend", because the "sleeping" and
"runnable" queues are also used to wait for something.

Check paths for security:
path must be absolute.
mount paths must exist and have no symlinks beyond the jail's path itself.
consolelog must exist (apart from the final component) and have no
symlinks beyond the jail's path itself.

Reads the mount.fstab file, and put its lines separately into the
IP__MOUNT_FROM_FSTAB internal parameter.

Combine check_intparams() and ip_params(), JF_CHECKINT and JF_IPPARAMS.

Don't assume either jid or name is set - they may not be from the
command line.

Keep all internal/known parameter names in one place, and use
enum constants everywhere else.

Initial work on the new jail(8). There are more features to add, and some
cleaning up to do on existing features, but this is pretty much what the
final product will look like.

Change the current working directory to be inside the jail created by
the jail(8) command. [10:04]

Fix a one-NUL-byte buffer overflow in libopie. [10:05]

Correctly sanity-check a buffer length in nfs mount. [10:06]

Approved by: so (cperciva)
Approved by: re (kensmith)
Security: FreeBSD-SA-10:04.jail
Security: FreeBSD-SA-10:05.opie
Security: FreeBSD-SA-10:06.nfsclient

Change the current working directory to be inside the jail created by
the jail(8) command. [10:04]

Fix a one-NUL-byte buffer overflow in libopie. [10:05]

Correctly sanity-check a buffer length in nfs mount. [10:06]

Approved by: so (cperciva)
Approved by: re (kensmith)
Security: FreeBSD-SA-10:04.jail
Security: FreeBSD-SA-10:05.opie
Security: FreeBSD-SA-10:06.nfsclient

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

Handle kernels that don't have IPv6 by not sending an "ip6.addr"
parameter unless a (numeric) IPv6 address is given. Even the default
binaries built with -DINET6 will work with IPv6-less kernels. With an
eye to the future, similarly handle the possibility of an IPv4-less kernel.

Approved by: re (kib), bz (mentor)

Add libjail, a (somewhat) simpler interface to the jail_set and jail_get
system calls and the security.jail.param sysctls.

Approved by: bz (mentor)

Remove obsolete comment describing how the command line is
no longer parsed.

Approved by: bz (mentor)

In preparation for raising NGROUPS and NGROUPS_MAX, change base
system callers of getgroups(), getgrouplist(), and setgroups() to
allocate buffers dynamically. Specifically, allocate a buffer of size
sysconf(_SC_NGROUPS_MAX)+1 (+2 in a few cases to allow for overflow).

This (or similar gymnastics) is required for the code to actually follow
the POSIX.1-2008 specification where {NGROUPS_MAX} may differ at runtime
and where getgroups may return {NGROUPS_MAX}+1 results on systems like
FreeBSD which include the primary group.

In id(1), don't pointlessly add the primary group to the list of all
groups, it is always the first result from getgroups(). In principle
the old code was more portable, but this was only done in one of the two
places where getgroups() was called to the overall effect was pointless.

Document the actual POSIX requirements in the getgroups(2) and
setgroups(2) manpages. We do not yet support a dynamic NGROUPS, but we
may in the future.

MFC after: 2 weeks

In the old-style jail command line, explicitly set parameters from the
security.jail.* sysctls since jail_set(2) doesn't do it implicitly.

Approved by: bz (mentor)

Add support for the arbitrary named jail parameters used by jail_set(2)
and jail_get(2). Jail(8) can now create jails using a "name=value"
format instead of just specifying a limited set of fixed parameters; it
can also modify parameters of existing jails. Jls(8) can display all
parameters of jails, or a specified set of parameters. The available
parameters are gathered from the kernel, and not hard-coded into these
programs.

Small patches on killall(1) and jexec(8) to support jail names with
jail_get(2).

Approved by: bz (mentor)

With the permission of phk@ change the license on remaining jail code
to a 2 clause BSD license.

Approved by: phk
Approved by: bz (mentor)

MFp4:
Bring in updated jail support from bz_jail branch.

This enhances the current jail implementation to permit multiple
addresses per jail. In addtion to IPv4, IPv6 is supported as well.
Due to updated checks it is even possible to have jails without
an IP address at all, which basically gives one a chroot with
restricted process view, no networking,..

SCTP support was updated and supports IPv6 in jails as well.

Cpuset support permits jails to be bound to specific processor
sets after creation.

Jails can have an unrestricted (no duplicate protection, etc.) name
in addition to the hostname. The jail name cannot be changed from
within a jail and is considered to be used for management purposes
or as audit-token in the future.

DDB 'show jails' command was added to aid debugging.

Proper compat support permits 32bit jail binaries to be used on 64bit
systems to manage jails. Also backward compatibility was preserved where
possible: for jail v1 syscalls, as well as with user space management
utilities.

Both jail as well as prison version were updated for the new features.
A gap was intentionally left as the intermediate versions had been
used by various patches floating around the last years.

Bump __FreeBSD_version for the afore mentioned and in kernel changes.

Special thanks to:
- Pawel Jakub Dawidek (pjd) for his multi-IPv4 patches
and Olivier Houchard (cognet) for initial single-IPv6 patches.
- Jeff Roberson (jeff) and Randall Stewart (rrs) for their
help, ideas and review on cpuset and SCTP support.
- Robert Watson (rwatson) for lots and lots of help, discussions,
suggestions and review of most of the patch at various stages.
- John Baldwin (jhb) for his help.
- Simon L. Nielsen (simon) as early adopter testing changes
on cluster machines as well as all the testers and people
who provided feedback the last months on freebsd-jail and
other channels.
- My employer, CK Software GmbH, for the support so I could work on this.

Reviewed by: (see above)
MFC after: 3 months (this is just so that I get the mail)
X-MFC Before: 7.2-RELEASE if possible

- Import the HEAD csup code which is the basis for the cvsmode work.

correct strtol(3) usage and style(9)

Reviewed by: maxim
MFC after: 2 weeks

o Style(9) the previous commit a bit.

Add the -s option to set jail's securelevel. This is useful for jails run with non-root privileges.

PR: bin/80242
MFC after: 2 weeks

o Do not mangle current session user login name with jail -u|-U.

PR: bin/94730
Submitted by: Frank Behrens
MFC after: 1 month

Add [-J jid_file] option to write out a JidFile, similar to a PidFile,
containing the jailid, path, hostname, ip and the command used to start
the jail.

PR: misc/89883
Submitted by: L. Jason Godsey <lannygodsey -at- yahoo.com>
Reviewed by: phk
MFC after: 1 week

Initialize lcap and pwd to NULL. This allows a WARNS=6 clean build,
hence bump it to 6.

Note that the last commit message was not quite accurate. While the
assumption exists in the code, it's not possible to have an
uninitialized p there because if lflag is set when username is NULL
then execution would be terminated earlier.

The code path in main() dealing with lflag assumes that p was
initialized with NULL, while it is not. So let's initialize
it.

Pass an array of gid_t rather than an array of int to getgroups().

PR: 56646

o Add -l option to jail(8) similar to su(1): before running jail'ed
program under specific user's credentials, clean the environment and
set only a few variables.

PR: bin/70024
Submitted by: demon
MFC after: 1 month

Prepare jail(8) utility for new functionality which will limit
seeing status of mounted file system for jailed processes.
Pass full path of jail's root directory to the kernel. mount(8) utility is
doing the same thing already.

o Implement -U flag: run command as user which exists only in jail.
o getpwnam(3) returns NULL and does not set errno when the user does
not exist. Bail out with "no such user" instead of "Unknown error: 0".

PR: bin/67262
Submitted by: demon (-U flag)
MFC after: 3 weeks

add FBSDID

Force output of jail ID (if necessary) before excuting the command,
otherwise redirection of stdout to a file using block buffering will
not complete in time.

o Add jls(8) for listing active jails.
o Add jexec(8) to execute a command in an existing jail.
o Add -j option for killall(1) to kill all processes in a specified
jail.
o Add -i option to jail(8) to output jail ID of newly created jail.

Free login_cap(3) resources after usage.

Submitted by: demon

o Fix error messages formatting, style.

Prodded by: bde
Reviewed by: bde

o Add -u <username> flag to jail(8): set user context before exec.

PR: bin/44320
Submitted by: Mike Matsnev <mike@po.cs.msu.su>
Reviewed by: -current
MFC after: 6 weeks

Usage style sweep: spell "usage" with a small 'u'.
Also change one case of blatant __progname abuse (several more remain)
This commit does not touch anything in src/{contrib,crypto,gnu}/.

Add missing includes and sort includes.

Include missing header files which define functions for which gcc has
builtints (e.g., exit, strcmp).

Add a version number field to the jail(2) argument so that future changes
can be handled intelligently.

WARNING: you will need to reinstall #includes and recompile jail(8).

$Id$ -> $FreeBSD$

Various cosmetics.

Submitted by: Rudolf Cejka <cejkar@dcse.fee.vutbr.cz>
Reviewed by: phk

Fix various bogons.

Submitted by: Rudolf Cejka <cejkar@dcse.fee.vutbr.cz>
Reviewed by: phk

This Implements the mumbled about "Jail" feature.

This is a seriously beefed up chroot kind of thing. The process
is jailed along the same lines as a chroot does it, but with
additional tough restrictions imposed on what the superuser can do.

For all I know, it is safe to hand over the root bit inside a
prison to the customer living in that prison, this is what
it was developed for in fact: "real virtual servers".

Each prison has an ip number associated with it, which all IP
communications will be coerced to use and each prison has its own
hostname.

Needless to say, you need more RAM this way, but the advantage is
that each customer can run their own particular version of apache
and not stomp on the toes of their neighbors.

It generally does what one would expect, but setting up a jail
still takes a little knowledge.

A few notes:

I have no scripts for setting up a jail, don't ask me for them.

The IP number should be an alias on one of the interfaces.

mount a /proc in each jail, it will make ps more useable.

/proc/<pid>/status tells the hostname of the prison for
jailed processes.

Quotas are only sensible if you have a mountpoint per prison.

There are no privisions for stopping resource-hogging.

Some "#ifdef INET" and similar may be missing (send patches!)

If somebody wants to take it from here and develop it into
more of a "virtual machine" they should be most welcome!

Tools, comments, patches & documentation most welcome.

Have fun...

Sponsored by: http://www.rndassociates.com/
Run for almost a year by: http://www.servetheweb.com/