Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

- Introduce the Hhook (Helper Hook) KPI. The KPI is closely modelled on pfil(9),
and in many respects can be thought of as a more generic superset of pfil.
Hhook provides a way for kernel subsystems to export hook points that Khelp
modules can hook to provide enhanced or new functionality to the kernel. The
KPI has been designed to ensure hook points pose no noticeable overhead when
no hook functions are registered.

- Introduce the Khelp (Kernel Helpers) KPI. Khelp provides a framework for
managing Khelp modules, which indirectly use the Hhook KPI to register their
hook functions with hook points of interest within the kernel. Khelp modules
aim to provide a structured way to dynamically extend the kernel at runtime in
an ABI preserving manner. Depending on the subsystem providing hook points, a
Khelp module may be able to associate per-object data for maintaining relevant
state between hook calls.

- pjd's Object Specific Data (OSD) KPI is used to manage the per-object data
allocated to Khelp modules. Create a new "OSD_KHELP" OSD type for use by the
Khelp framework.

- Bump __FreeBSD_version to 900028 to mark the introduction of the new KPIs.

In collaboration with: David Hayes <dahayes at swin edu au> and
Grenville Armitage <garmitage at swin edu au>
Sponsored by: FreeBSD Foundation
Reviewed by: bz, others along the way
MFC after: 3 months

Introduce the extensible jail framework, using the same "name=value"
interface as nmount(2). Three new system calls are added:
* jail_set, to create jails and change the parameters of existing jails.
This replaces jail(2).
* jail_get, to read the parameters of existing jails. This replaces the
security.jail.list sysctl.
* jail_remove to kill off a jail's processes and remove the jail.
Most jail parameters may now be changed after creation, and jails may be
set to exist without any attached processes. The current jail(2) system
call still exists, though it is now a stub to jail_set(2).

Approved by: bz (mentor)

Add support for methods to the OSD subsystem. Each object type has a
predefined set of methods, which are set in osd_register() and called
via osd_call(). Currently, no methods are defined, though prison
objects will have some in the future.

Expand the locking from a single per-type mutex to three different kinds
of locks (four if you include the requirement that the container
(e.g. prison) be locked when getting/setting data). This clears up one
existing issue, as well as others added by the method support.

Approved by: bz (mentor)

Update ZFS from version 6 to 13 and bring some FreeBSD-specific changes.

This bring huge amount of changes, I'll enumerate only user-visible changes:

- Delegated Administration

Allows regular users to perform ZFS operations, like file system
creation, snapshot creation, etc.

- L2ARC

Level 2 cache for ZFS - allows to use additional disks for cache.
Huge performance improvements mostly for random read of mostly
static content.

- slog

Allow to use additional disks for ZFS Intent Log to speed up
operations like fsync(2).

- vfs.zfs.super_owner

Allows regular users to perform privileged operations on files stored
on ZFS file systems owned by him. Very careful with this one.

- chflags(2)

Not all the flags are supported. This still needs work.

- ZFSBoot

Support to boot off of ZFS pool. Not finished, AFAIK.

Submitted by: dfr

- Snapshot properties

- New failure modes

Before if write requested failed, system paniced. Now one
can select from one of three failure modes:
- panic - panic on write error
- wait - wait for disk to reappear
- continue - serve read requests if possible, block write requests

- Refquota, refreservation properties

Just quota and reservation properties, but don't count space consumed
by children file systems, clones and snapshots.

- Sparse volumes

ZVOLs that don't reserve space in the pool.

- External attributes

Compatible with extattr(2).

- NFSv4-ACLs

Not sure about the status, might not be complete yet.

Submitted by: trasz

- Creation-time properties

- Regression tests for zpool(8) command.

Obtained from: OpenSolaris