Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r246586,246587,246619,246624,246768,246808:

LZ4 compression support in ZFS.

(Note: sys/conf/files change omitted from this changeset).

MFC r226549,r226550,r226551,r226552,r226553,r226554,r226568,r226569,r226611,
r226612:

r226549:

Remove redundant size calculation.

r226550:

Initialize 'rc' properly before using it. This error could lead to infinite
loop when data reconstruction was needed.

r226551:

Don't mark vdev as healthy too soon, so we won't try to use invalid vdevs.

r226552:

Never pass NULL block pointer when reading. This is neither expected nor
handled by lower layers like vdev_raidz, which uses bp for checksum
verification. This bug could lead to NULL pointer reference and resets
during boot.

r226553:

Always pass data size for checksum verification function, as using
physical block size declared in bp may not always be what we want.
For example in case of gang block header physical block size declared
in bp is much larger than SPA_GANGBLOCKSIZE (512 bytes) and checksum
calculation failed. This bug could lead to accessing unallocated
memory and resets/failures during boot.

r226554:

Fix missing return when LOADER_GPT_SUPPORT is defined, but LOADER_MBR_SUPPORT
is not.

r226568:

- Correctly read gang header from raidz.
- Decompress assembled gang block data if compressed.
- Verify checksum of a gang header.
- Verify checksum of assembled gang block data.
- Verify checksum of uber block.

Submitted by: avg

r226569:

With LOADER_MBR_SUPPORT defined and LOADER_GPT_SUPPORT undefined we would
never call disk_openmbr().

Submitted by: avg

r226611:

- Allow to specify multiple files to check, eg.

zfsboottest gpt/system0 gpt/system1 - /boot/kernel/kernel /boot/zfsloader

- Instead of printing file's content calculate MD5 hash of the file,
so it can be easly compared to the hash calculated via file system.
- Some other minor improvements.

r226612:

Because ZFS boot code was very fragile in the past and real PITA to debug,
introduce zfsboottest.sh script that will verify if it will be possible to boot
from the given pool.

# zfsboottest.sh system

Where "system" is pool name of the pool we want to boot from.

What is being verified by the script:
- Does the pool exist?
- Does it have bootfs property configured?
- Is mountpoint property of the boot dataset set to 'legacy'?

Dataset configured in bootfs property has to be mounted to perform more
checks:
- Does the /boot directory in boot dataset exist?
- Is this dataset configured as root file system in /etc/fstab or set
in vfs.root.mountfrom variable in /boot/loader.conf?

By using zfsboottest tool the script will read all the files in /boot
directory using ZFS boot code and calculate their checksums.
Then, it will walk /boot directory using find(1) though regular file sytem
and also read all the files in /boot directory and calculate their checksums.
If any of the files cannot be looked up, read or checksum is invalid it will
be reported and booting off of this pool is probably not possible.

Some additional checks may be interesting as well. For example if the disks
contain proper pmbr and gptzfsboot code or if all expected files in /boot/
are present.

When upgrading FreeBSD, one should snapshot datasets that contain operating
system, upgrade (install new world and kernel) and use zfsboottest.sh to verify
if it will be possible to boot from new configuration. If all is good one
should upgrade boot blocks, by eg.:

# gpart -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1

If something is wrong, one should rollback datasets and report the problems.

Approved by: re (kib)

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

zfs boot subroutines: correctly specify type of an integer literal

Found by adding more warning flags to zfs boot blocks build.

Approved by: re (kib)
MFC after: 1 week

Finally... Import the latest open-source ZFS version - (SPA) 28.

Few new things available from now on:

- Data deduplication.
- Triple parity RAIDZ (RAIDZ3).
- zfs diff.
- zpool split.
- Snapshot holds.
- zpool import -F. Allows to rewind corrupted pool to earlier
transaction group.
- Possibility to import pool in read-only mode.

MFC after: 1 month

Teach the (gpt)zfsboot and zfsloader raidz code to use its buffers
more efficiently.

Before this patch, in the worst case memory use would increase
exponentially on the number of drives in the raidz vdev.

Submitted by: Matt Reimer <mattjreimer@gmail.com>
Sponsored by: VPOP Technologies, Inc.
Silence from: dfr

Correct another case of not doing 64bit math. This allows mine and
other raidz2 volumes to boot.

Submitted by: Matt Reimer <mattjreimer@gmail.com>

Nobody spoke up, so assume my interpretation was correct and enable keyword
expansion for this file.

Unexpand $FreeBSD$.

Remove svn:keywords on a file that had fbsd:nokeywords (though I don't
understand the reason for the latter)

Expand $FreeBSD$

Add support for booting from raidz1 and raidz2 pools.

Some zfsboot fixes from Norikatsu Shigemura:

1. zfsboot2 (boot2) doesn't %d (printf), so change %d to %u.
2. chase new zpool versioning as SPA_VERSION.
Obtained from: sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h

Submitted by: nork

Update ZFS from version 6 to 13 and bring some FreeBSD-specific changes.

This bring huge amount of changes, I'll enumerate only user-visible changes:

- Delegated Administration

Allows regular users to perform ZFS operations, like file system
creation, snapshot creation, etc.

- L2ARC

Level 2 cache for ZFS - allows to use additional disks for cache.
Huge performance improvements mostly for random read of mostly
static content.

- slog

Allow to use additional disks for ZFS Intent Log to speed up
operations like fsync(2).

- vfs.zfs.super_owner

Allows regular users to perform privileged operations on files stored
on ZFS file systems owned by him. Very careful with this one.

- chflags(2)

Not all the flags are supported. This still needs work.

- ZFSBoot

Support to boot off of ZFS pool. Not finished, AFAIK.

Submitted by: dfr

- Snapshot properties

- New failure modes

Before if write requested failed, system paniced. Now one
can select from one of three failure modes:
- panic - panic on write error
- wait - wait for disk to reappear
- continue - serve read requests if possible, block write requests

- Refquota, refreservation properties

Just quota and reservation properties, but don't count space consumed
by children file systems, clones and snapshots.

- Sparse volumes

ZVOLs that don't reserve space in the pool.

- External attributes

Compatible with extattr(2).

- NFSv4-ACLs

Not sure about the status, might not be complete yet.

Submitted by: trasz

- Creation-time properties

- Regression tests for zpool(8) command.

Obtained from: OpenSolaris