- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

add detection of serial console presence to btx and boot2-like blocks

Note that this commit slightly increases size of boot blocks.

Reviewed by: jhb
Tested by: Olivier Cochard-Labbe <olivier@cochard.me>
MFC after: 26 days

zfsboot: simplify probe_drive() a little bit

The first discovered pool, whether it covers the whole boot disk or not,
is going to be first in zfs_pools list. So there is no need at all
for spapp parameter.

This commit also fixes a bug where NULL would be assigned to NULL
pointer when probe_drive was called with the spapp parameter of NULL.

MFC after: 21 days

zfs boot: export boot/primary pool and vdev guid all the way to kenv

This is work in progress to for znextboot and it also provides
some convenient infrastructure.

MFC after: 20 days

zfsboot: use the same zfs dataset naming format as loader

Also, pool name alone now names a root dataset of the pool regardless
of bootfs property value.

MFC after: 15 days

zfsboot/zfsloader: support accessing filesystems within a pool

In zfs loader zfs device name format now is "zfs:pool/fs",
fully qualified file path is "zfs:pool/fs:/path/to/file"
loader allows accessing files from various pools and filesystems as well
as changing currdev to a different pool/filesystem.

zfsboot accepts kernel/loader name in a format pool:fs:path/to/file or,
as before, pool:path/to/file; in the latter case a default filesystem
is used (pool root or bootfs). zfsboot passes guids of the selected
pool and dataset to zfsloader to be used as its defaults.

zfs support should be architecture independent and is provided
in a separate library, but architectures wishing to use this zfs support
still have to provide some glue code and their devdesc should be
compatible with zfs_devdesc.
arch_zfs_probe method is used to discover all disk devices that may
be part of ZFS pool(s).

libi386 unconditionally includes zfs support, but some zfs-specific
functions are stubbed out as weak symbols. The strong definitions
are provided in libzfsboot.
This change mean that the size of i386_devspec becomes larger
to match zfs_devspec.

Backward-compatibility shims are provided for recently added sparc64
zfs boot support. Currently that architecture still works the old
way and does not support the new features.

TODO:
- clear up pool root filesystem vs pool bootfs filesystem distinction
- update sparc64 support
- set vfs.root.mountfrom based on currdev (for zfs)

Mid-future TODO:
- loader sub-menu for selecting alternative boot environment

Distant future TODO:
- support accessing snapshots, using a snapshot as readonly root

Reviewed by: marius (sparc64),
Gavin Mu <gavin.mu@gmail.com> (sparc64)
Tested by: Florian Wagner <florian@wagner-flo.net> (x86),
marius (sparc64)
No objections: fs@, hackers@
MFC after: 1 month

i386 boot: consolidate MAXBDDEV definition

MFC after: 1 month

btxldr: future-proof argument passing from boot1/2-ish to loader

Place the arguments at a fixed offset of 0x800 withing the argument area
(of size 0x1000). Allow variable size extended arguments first of which
should be a size of the extended arguments (including the size
parameter).

Consolidate all related definitions in a new i386/common/bootargs.h header.

Many thanks to jhb and bde for their guidance and reviews.

Reviewed by: jhb, bde
Approved by: jhb
MFC after: 1 month

zfsboot: honor -q if it's present in boot.config

Before r228267 the option was honored but the original content of
boot.config was not preserved. I tried to fix that but missed the idea.
Now the proper way of doing things is taken from i386/boo2.
Also, a comment is added to explain this a little bit unobvious
behavior.

Inspired by: jhb
MFC after: 5 days

Define several extra macros in bsd.sys.mk and sys/conf/kern.pre.mk, to
get rid of testing explicitly for clang (using ${CC:T:Mclang}) in
individual Makefiles.

Instead, use the following extra macros, for use with clang:
- NO_WERROR.clang (disables -Werror)
- NO_WCAST_ALIGN.clang (disables -Wcast-align)
- NO_WFORMAT.clang (disables -Wformat and friends)
- CLANG_NO_IAS (disables integrated assembler)
- CLANG_OPT_SMALL (adds flags for extra small size optimizations)

As a side effect, this enables setting CC/CXX/CPP in src.conf instead of
make.conf! For clang, use the following:

CC=clang
CXX=clang++
CPP=clang-cpp

MFC after: 2 weeks

zfsboot: print boot.config contents before parsing it

as parsing can be a destructive operation

MFC after: 2 weeks

Look for /boot/config in addition to /boot.config, with the former taking
precedence over the latter if it exists.

MFC after: 3 weeks

gpt/zfs boot blocks: reduce optimizing CFLAGS to -O1

gpt and zfs boot blocks are not nearly as size-constrained as boot2
from which they inherited their current optimization and anti-optimization
options. As such the current options do not provide any benefit, but
make debugging of the code much harder.
Also, it has been demonstrated that combination of -mrtd and
-fno-unit-at-a-time may result in mis-compilation of the boot code
with the current base gcc.

Additionally, intermediate assembly file filtering is removed for
zfsboot.

The new boot blocks are all compile- and boot- tested using qemu.
gptzfsboot is tested with real hardware.

Reported by: Peter Jeremy <peterjeremy@acm.org> [miscompilation]
Discussed with: bde, jhb
Tested by: Sebastian Chmielewski <chmielsster@gmail.com> [gptzfsboot]
Approved by: re (kib)
MFC after: 3 weeks

When building some of the boot loaders with clang, and DEBUG_FLAGS or
CFLAGS having '-g' in it, clang outputs several assembly directives that
are too new for our version of binutils.

Therefore, assemble the resulting .s files with clang instead. A more
general solution can be implemented when a GNU as-compatible driver for
clang's integrated assembler appears.

Reported by: dougb

Revert the entry point label to 'start' to unbreak the build.

Pointy hat to: jhb

- Remove the fake BPB from zfsldr. zfsldr doesn't support booting from
floppies, so it will not be used as the start of an emulated floppy
image on a bootable CD which is what the fake BPB was used for.
- Only check that EDD packet mode is available once at the start of
zfsldr rather than for each disk sector now that we read data in one
sector at a time. As a result, collapse the remaining bits of read
up into nread and rename nread to read.
- Restore a return at the end of putstr that I removed in the previous
revision.

Tested by: Henri Hennebert (earlier version)
MFC after: 1 week

The recent change to increase the zfsboot size to 64k made a few BIOSes
unhappy (probably they don't handle crossing the 64k boundary, etc.).
Fix this by changing zfsldr to use a loop reading from the disk one
sector at a time. To avoid trashing the saved copy of the MBR which is
used for disk I/O, read zfsboot2 at address 0x9000. This has the
advantage that BTX no longer needs to be relocated as it is read into
the correct location. However, the loop to relocate zfsboot2.bin can
now cross a 64k boundary, so change it to use relative segments instead.
(This will need further work if zfsboot2.bin ever exceeds 64k.)

While here, stop storing a relocated copy of zfsldr at 0x700. This was
only used by the xread hack which has recently been removed (and even
that use was dubious). Also, include the BIOS error code as hex when
reporting read errors to aid in debugging.

Much thanks to Henri Hennebert for patiently testing various iterations
of the patch as well as fixing the zfsboot2.bin relocation to use
relative segments.

MFC after: 1 week

Upgrade our copy of llvm/clang to r132879, from upstream's trunk.

Due to space constraints, the UFS boot2 and boot1 use an evil hack where
boot2 calls back into boot1 to perform disk reads. The ZFS MBR boot blocks
do not have the same space constraints, so remove this hack for ZFS.
While here, remove commented out code to support C/H/S addressing from
zfsldr. The ZFS and GPT bootstraps always just use EDD LBA addressing.

MFC after: 2 weeks

Remove all object files during 'make clean'.

MFC after: 1 week

The size of zfsboot2 grown up to 64 Kbytes in r219089.
Increase NSECT up to 128 sectors too.

Reviewed by: jhb, pjd

Finally... Import the latest open-source ZFS version - (SPA) 28.

Few new things available from now on:

- Data deduplication.
- Triple parity RAIDZ (RAIDZ3).
- zfs diff.
- zpool split.
- Snapshot holds.
- zpool import -F. Allows to rewind corrupted pool to earlier
transaction group.
- Possibility to import pool in read-only mode.

MFC after: 1 month

Clang's integrated assembler can now handle sys/boot/i386/boot2/sio.S.
It used to choke on the notation "inb (%dx),%al" for "inb %dx,%al"; GNU
as accepts both forms. Which notation is more 'correct' is an open
question. :)

Upgrade our copy of llvm/clang to r126079, from upstream's trunk.

This contains many improvements, primarily better C++ support, an
integrated assembler for x86 and support for -pg.

Remove superfluous -mno-(mmx|3dnow|sse|sse2|sse3) flags in Makefiles
under sys/boot/{i386,pc98}, since these are already added via
sys/boot/{i386,pc98}/Makefile.inc.

Submitted by: arundel

- Split code shared by almost any boot loader into separate files and
clean up most layering violations:

sys/boot/i386/common/rbx.h:

RBX_* defines
OPT_SET()
OPT_CHECK()

sys/boot/common/util.[ch]:

memcpy()
memset()
memcmp()
bcpy()
bzero()
bcmp()
strcmp()
strncmp() [new]
strcpy()
strcat()
strchr()
strlen()
printf()

sys/boot/i386/common/cons.[ch]:

ioctrl
putc()
xputc()
putchar()
getc()
xgetc()
keyhit() [now takes number of seconds as an argument]
getstr()

sys/boot/i386/common/drv.[ch]:

struct dsk
drvread()
drvwrite() [new]
drvsize() [new]

sys/boot/common/crc32.[ch] [new]

sys/boot/common/gpt.[ch] [new]

- Teach gptboot and gptzfsboot about new files. I haven't touched the
rest, but there is still a lot of code duplication to be removed.

- Implement full GPT support. Currently we just read primary header and
partition table and don't care about checksums, etc. After this change we
verify checksums of primary header and primary partition table and if
there is a problem we fall back to backup header and backup partition
table.

- Clean up most messages to use prefix of boot program, so in case of an
error we know where the error comes from, eg.:

gptboot: unable to read primary GPT header

- If we can't boot, print boot prompt only once and not every five
seconds.

- Honour newly added GPT attributes:

bootme - this is bootable partition
bootonce - try to boot from this partition only once
bootfailed - we failed to boot from this partition

- Change boot order of gptboot to the following:

1. Try to boot from all the partitions that have both 'bootme'
and 'bootonce' attributes one by one.
2. Try to boot from all the partitions that have only 'bootme'
attribute one by one.
3. If there are no partitions with 'bootme' attribute, boot from
the first UFS partition.

- The 'bootonce' functionality is implemented in the following way:

1. Walk through all the partitions and when 'bootonce'
attribute is found without 'bootme' attribute, remove
'bootonce' attribute and set 'bootfailed' attribute.
'bootonce' attribute alone means that we tried to boot from
this partition, but boot failed after leaving gptboot and
machine was restarted.
2. Find partition with both 'bootme' and 'bootonce' attributes.
3. Remove 'bootme' attribute.
4. Try to execute /boot/loader or /boot/kernel/kernel from that
partition. If succeeded we stop here.
5. If execution failed, remove 'bootonce' and set 'bootfailed'.
6. Go to 2.

If whole boot succeeded there is new /etc/rc.d/gptboot script coming
that will log all partitions that we failed to boot from (the ones with
'bootfailed' attribute) and will remove this attribute. It will also
find partition with 'bootonce' attribute - this is the partition we
booted from successfully. The script will log success and remove the
attribute.

All the GPT updates we do here goes to both primary and backup GPT if
they are valid. We don't touch headers or partition tables when
checksum doesn't match.

Reviewed by: arch (Message-ID: <20100917234542.GE1902@garage.freebsd.pl>)
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
MFC after: 2 weeks

Before VirtualBox is fixed, mark with #ifdef what has to be done to make
it possible to boot from ZFS RAIDZ for example from within VirtualBox.
The problem with VirtualBox is that its BIOS reports only one disk present.
If we choose to ignore this report, we can find all the disks available.
We can't have this work-around to be turned on by default, because some broken
BIOSes report true when it comes to number of disks, but present the same disk
multiple times.

MF tbemd: Minor tweaks, prefer MACHINE_CPUARCH generally to MACHINE_ARCH

- Set 'dmadat' earlier so that we properly setup the heap if we fail to
locate a high memory area for the heap using the SMAP.
- Read the number of hard drive devices from the BIOS instead of hardcoding
a limit of 128. Some BIOSes duplicate disk devices once you get beyond
the maximum drive number.

MFC after: 1 month

Our boot loader is capable of booting both i386 and amd64 kernels so
call it "x86" instead of "i386".

Suggested by: jhb in response to Alexander Best's loader proposal
MFC after: 1 month

Fix a confusing typo in the EDD packet structure used in gptboot and
gptzfsboot. I got the segment and offset fields reversed in the structure,
but I also succeeded in crossing the assignments so the actual EDD packet
ended up correct.

MFC after: 1 week

- Port bios_getmem() from libi386 to {gpt,}zfsboot() and use it to
safely allocate a heap region above 1MB. This enables {gpt,}zfsboot()
to allocate much larger buffers than before.
- Use a larger buffer (1MB instead of 128K) for temporary ZFS buffers. This
allows more reliable reading of compressed files in a raidz/raidz2 pool.

Submitted by: Matt Reimer mattjreimer of gmail
MFC after: 1 week

Create a seperate ZFS enabled loader.

This adds zfsloader which will be called by zfsboot/gptzfsboot code rather
than the tradional loader. This eliminates the need to set the
LOADER_ZFS_SUPPORT variable in order to get a ZFS enabled loader.

Note however, that you must reinstall your bootcode (zfsboot/gptzfsboot)
in order for the boot process to use the new loader.

New installations will no longer be required to build a ZFS enabled
loader for a working ZFS boot system. Installing zfsboot/gptzfsboot is
sufficient for acknowledging the use of CDDL code and therefore the ZFS
enabled loader.

Based on a previous patch from jhb@

Reviewed by: jhb@
MFC after: 2 weeks

Always use 64-bit LBAs for disk addresses in zfsboot and gptzfsboot to
fully support booting from large volumes.

Tested by: Emil Smolenski ambsd of raisa.eu.org
Submitted by: Matt Reimer mattjreimer of gmail (most of the C bits)
MFC after: 1 week

Correct some issues with zfs boot.

- Teach it to read gang blocks. (essentially untested)
If you see "ZFS: gang block detected!", please let
me know, so we can either remove the printf if it
works, or fix it if it doesn't.

- If multiple partitions exist on a disk, probe them all.
We also need to reset dsk->start to 0 to read the right
sector here.

- With GPT, we can have 128 partitions.

- If the bootfs property has ever been set on a pool
it seems that it never goes away. zpool won't allow
you to add to the pool with the bootfs property set.
However, if you clear the property back to default
we end up getting 0 for the object number and read
a bogus block pointer and fail to boot.

- Fix some error printfs. The printf in the loader is
only capable of c,s and u formats.

- Teach printf how to display %llu

Reviewed by: dfr, jhb
MFC after: 2 weeks

Use zfs_read() instead of xfsread() to read /boot.config. xfsread() fails
short read requests, so the result was that a /boot.config smaller than 512
bytes was ignored. boot2 uses fsread() instead of xfsread() to read
/boot.config already, so this makes zfsboot more like boot2.

Submitted by: Johny Mattsson johny-freebsd of earthmagic org
Reviewed by: dfr
MFC after: 3 days

Add support for booting from raidz1 and raidz2 pools.

Use full 64bit arithmetic when converting file offsets to block numbers - fixes
booting on filesystems with inode numbers with values above 4194304.

Submitted by: ps

Add a GPT-aware variant of zfsboot which should be used in a similar manner
to gptboot, i.e. installed in a freebsd-boot partition using /sbin/gpart or
/sbin/gpt.

Tweak the /boot/loader ZFS support so that it can find ZFS pools that are
contained in GPT partitions.

Update ZFS from version 6 to 13 and bring some FreeBSD-specific changes.

This bring huge amount of changes, I'll enumerate only user-visible changes:

- Delegated Administration

Allows regular users to perform ZFS operations, like file system
creation, snapshot creation, etc.

- L2ARC

Level 2 cache for ZFS - allows to use additional disks for cache.
Huge performance improvements mostly for random read of mostly
static content.

- slog

Allow to use additional disks for ZFS Intent Log to speed up
operations like fsync(2).

- vfs.zfs.super_owner

Allows regular users to perform privileged operations on files stored
on ZFS file systems owned by him. Very careful with this one.

- chflags(2)

Not all the flags are supported. This still needs work.

- ZFSBoot

Support to boot off of ZFS pool. Not finished, AFAIK.

Submitted by: dfr

- Snapshot properties

- New failure modes

Before if write requested failed, system paniced. Now one
can select from one of three failure modes:
- panic - panic on write error
- wait - wait for disk to reappear
- continue - serve read requests if possible, block write requests

- Refquota, refreservation properties

Just quota and reservation properties, but don't count space consumed
by children file systems, clones and snapshots.

- Sparse volumes

ZVOLs that don't reserve space in the pool.

- External attributes

Compatible with extattr(2).

- NFSv4-ACLs

Not sure about the status, might not be complete yet.

Submitted by: trasz

- Creation-time properties

- Regression tests for zpool(8) command.

Obtained from: OpenSolaris