Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC 239054,239057,239058,239060,239066,239067,239068,239070,239073,
239087,239088,239127,239210,239211,239230,239231,239232,239243,
239292,239293,239294,239325,240272,240273,240274,240275,240276,
240277,240335,240481,241023,241047,241053,241065,241068,241069,
241070,241164,241809,241876

239054:
Create the interface to work with various partition tables from the
loader(8). The following partition tables are supported: BSD label, GPT,
MBR, EBR and VTOC8.
239057:
Remove unused variables.
239058:
Introduce new API to work with disks from the loader's drivers.
It uses new API from the part.c to work with partition tables.
239060:
When GPT signature is invalid in the primary GPT header, then try to
read backup GPT header.
239066:
Add offset field to the i386_devdesc structure to be compatible with
disk_devdesc structure. Update biosdisk driver to the new disk API.
239067:
Remove unneeded flag.
239068:
Teach the ZFS use new partitions API when probing.
Note: now ZFS does probe only for partitions with type "freebsd-zfs"
and "freebsd".
239070:
Add simple test program that uses the partition tables handling code.
It is useful to test and debug how boot loader handles partition tables
metadata.
239073:
Bump USERBOOT_VERSION.
239087:
Add to the debug output the offset from the parent partitioning scheme.
239088:
Fix start offset calculation for the EBR partitions.
239127:
As it turned out, there are some installations, where BSD label
contains partitions with type zero. And it has worked.
So, allow detect these partitions.
239210:
Add more debug messages.
239211:
Add another debug message.
239230:
Unbreak booting from the true dedicated disks.
When we open the disk, check the type of partition table, that has
been detected. If this is BSD label, then we assume this is DD mode.
239231:
Remove colons from the debug message, device name returned by the
disk_fmtdev() already has the colons.
239232:
Restore the old behaviour. If requested partition is a BSD slice,
but d_partition isn't explicitly set, then try to open BSD label and its
first partition.
239243:
After r239066, reinitialize v86.ctl and v86.addr for int 13 EDD probing
in sys/boot/i386/libi386/biosdisk.c. Otherwise, when DISK_DEBUG is
enabled, the DEBUG() macros will clobber those fields, and cause the
probing to always fail mysteriously when debugging is enabled.
239292:
Explicitly terminate the string after strncpy(3).
239293:
Rework r239232 to unbreak ZFS detection on MBR slices.
239294:
Some BIOSes return incorrect number of sectors, make checks less
strictly, to do not lost some partitions.
239325:
Add comment why the code has been disabled.
240272:
Make struct uboot_devdesc compatible with struct disk_devdesc.
240273:
Use disk_fmtdev() and disk_parsedev() functions from the new DISK API.
240274:
Update uboot's disk driver to use new DISK API.
240275:
Build disk.c only when DISK_SUPPORT is enabled.
240276:
Update according to the change of struct uboot_devdesc.
240277:
Handle LOADER_NO_DISK_SUPPORT knob in the arm and powerpc ubldr.
240335:
Slightly reduce an overhead for the open() call in the zfsloader.
libstand(3) tries to detect file system in the predefined order,
but zfsloader usually is used for the booting from ZFS, and there is
no need to try detect several file system types for each open() call.
240481:
The MBR data is not necessarily aligned. This is a problem on ARM.
241023:
Make the loader a bit smarter, when it tries to open disk and the slice
number is not exactly specified. When the disk has MBR, also try to read
BSD label after ptable_getpart() call. When the disk has GPT, also set
d_partition to 255. Mostly, this is how it worked before.
241047:
Disable splitfs support, since we aren't support floppies for a long
time. This slightly reduces an overhead, when loader tries to open
file that doesn't exist.
241053:
Almost each time when loader opens a file, this leads to calling
disk_open(). Very often this is called several times for one file.
This leads to reading partition table metadata for each call. To
reduce the number of disk I/O we have a simple block cache, but it
is very dumb and more than half of I/O operations related to reading
metadata, misses this cache.

Introduce new cache layer to resolve this problem. It is independent
and doesn't need initialization like bcache, and will work by default
for all loaders which use the new DISK API. A successful disk_open()
call to each new disk or partition produces new entry in the cache.
Even more, when disk was already open, now opening of any nested
partitions does not require reading top level partition table.
So, if without this cache, partition table metadata was read around
20-50 times during boot, now it reads only once. This affects the booting
from GPT and MBR from the UFS.
241065:
Fix disk_cleanup() to work without DISK_DEBUG too.
241068:
Reduce the number of attempts to detect proper kld format for the amd64
loader.
241069:
Remember the file format of the last loaded module and try to use it for
next files.
241070:
Fix the style.
241164:
Replace all references to loader_callbacks_v1 with loader_callbacks.
241809:
Add the flags parameter to the disk_open() function and DISK_F_NOCACHE
flag, that disables the caching of partition tables metadata.
Use this flag for floppies in the libi386/biosdisk driver.
241876:
When loader tries to open GPT partition, but partition table is not GPT,
then try automatically detect an appropriate partition type.

MFC r235329,235343,235361,235364: zfsboot/zfsloader: support accessing
filesystems within a pool

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Add new loader console type: "spinconsole". This console selects the
video console which doesn't take any input from keyboard and hides
all output replacing it with ``spinning'' character (useful for
embedded products and custom installations).

Sponsored by: Sippy Software, Inc.

Update ZFS from version 6 to 13 and bring some FreeBSD-specific changes.

This bring huge amount of changes, I'll enumerate only user-visible changes:

- Delegated Administration

Allows regular users to perform ZFS operations, like file system
creation, snapshot creation, etc.

- L2ARC

Level 2 cache for ZFS - allows to use additional disks for cache.
Huge performance improvements mostly for random read of mostly
static content.

- slog

Allow to use additional disks for ZFS Intent Log to speed up
operations like fsync(2).

- vfs.zfs.super_owner

Allows regular users to perform privileged operations on files stored
on ZFS file systems owned by him. Very careful with this one.

- chflags(2)

Not all the flags are supported. This still needs work.

- ZFSBoot

Support to boot off of ZFS pool. Not finished, AFAIK.

Submitted by: dfr

- Snapshot properties

- New failure modes

Before if write requested failed, system paniced. Now one
can select from one of three failure modes:
- panic - panic on write error
- wait - wait for disk to reappear
- continue - serve read requests if possible, block write requests

- Refquota, refreservation properties

Just quota and reservation properties, but don't count space consumed
by children file systems, clones and snapshots.

- Sparse volumes

ZVOLs that don't reserve space in the pool.

- External attributes

Compatible with extattr(2).

- NFSv4-ACLs

Not sure about the status, might not be complete yet.

Submitted by: trasz

- Creation-time properties

- Regression tests for zpool(8) command.

Obtained from: OpenSolaris

MFp4: add FireWire/dcons support in loader for i386/amd64.

It is disabled by default. You need to put
LOADER_FIREWIRE_SUPPORT=yes in /etc/make.conf
and rebuild loader to enable it.
(cd /sys/boot/i386 && make clean && make && make install)

You can find a short introduction of dcons at
http://wiki.freebsd.org/DebugWithDcons

Add the loader side of support for preloading ELF relocatable object
format modules, which are currently only used on the amd64 platform.
This initial implementation just parses enough of the module to
allow it to extract dependencies and load all the bits into the
right place in memory, so the kernel must still do the full relocation
and linking. The details of the loaded sections are passed to the
kernel by supplying a copy of the ELF section header table as module
metadata with the MODINFOMD_SHDR tag.

Use __FBSDID().
Also some minor copyright style cleanups.

Enable the i386 loader to load and run an amd64 kernel. If this puts
things over floppy size limits, I can exclude it for release builds or
something like that. Most of the changes are to get the load_elf.c file
into a seperate elf32_ or elf64_ namespace so that you can have two
ELF loaders present at once. Note that for 64 bit kernels, it actually
starts up the kernel already in 64 bit mode with paging enabled. This
is really easy because we have a known minimum feature set.

Of note is that for amd64, we have to pass in the bios int 15 0xe821
memory map because once in long mode, you absolutely cannot make VM86
calls. amd64 does not use 'struct bootinfo' at all. It is a pure loader
metadata startup, just like sparc64 and powerpc. Much of the
infrastructure to support this was adapted from sparc64.

Renamed the loader's zipfs to gzipfs. zipfs.c was repo-copied to gzipfs.c.

Initiate deorbit burn of i386 a.out kld "support" in loader. Note that
this was quite broken, it never was updated for metadata support.
The a.out kld file support was never really used, as it wasn't necessary.
You could always load elf kld's, even in an a.out kernel.

(forced)

Splitfs is subject of:

MFC after: 2 weeks

Add splitfs vfs layer into libstand, which allows loading big kernels and
modules split across several physical medias. Following is how it works:

The splitfs code, when asked to open "foo" looks for a file "foo.split"
which is a text file containing a list of filenames and media names, e.g.

foo.aa "Kernel floppy 1"
foo.ab "Kernel floppy 2"
foo.ac "Kernel and modules floppy"

For each file segment, the process is:

- try to open the file
- prompt "Insert the disk labelled <whatever> and press any key..."
- try to open the file
- return error if file could not be located

RE team is free to use this feature in the upcoming 5.0-DP1.

Reviewed by: msmith, dcs

Hook up the bioscd driver and the cd9660 filesystem.

Add support for loading bzip2-compressed kernels and modules. This support
is turned off by default and could be enabled by defining LOADER_BZIP2_SUPPORT
make variable. Also make gzip support optional (turned on by default) -
it could be turned off via LOADER_NO_GZIP_SUPPORT make variable.

Please note, that due to limit on the amount of memory available to the
loader(8), it is possible to load modules/kernels compressed with the smallest
block size supported by the bzip2 - 100k (`-1' bzip2(1) option), however
even in this mode bzip2(1) usually provides better compression ratio than
gzip(1) in its best compression mode.

MFC after: 1 month

Add support for the "nullconsole" console type, for systems with
neither a video console nor a serial port. Use it if the RB_MUTE
flag is set.

Submitted by: Doug Ambrisko <ambrisko@whistle.com>
Reviewed by: jhb

Update loader logic to distinguish modules vs. files.
Add support for module metadata. The old way of dependancy
handling will be supported for a while.

Reviewed by: peter

Teach the loader about the ext2fs filesystem, extended partitions, and
the new readdir function.

Make PXE use the UDP API. This allows for both TFTP and NFS support.
You may specify TFTP or NFS via compile time options in the loader,
but not both at this time.

Also, remove a warning about not knowing how to boot from network
devices. We can obviously do that now.

Mega i386 loader commit.
- Don't hard code 0x10000 as the entry point for the loader. Instead add
src/sys/boot/i386/Makefile.inc which defines a make variable with the
entry point for the loader. Move the loader's entry point up to
0x20000, which makes PXE happy.
- Don't try to use cpp to parse btxldr for the optional BTXLDR_VERBOSE,
instead use m4 to achieve this. Also, add a BTXLDR_VERBOSE knob in the
btxldr Makefile to turn this option on.
- Redo parts of cdldr's Makefile so that it now builds and installs cdboot
instead of having i386/loader/Makefile do that. Also, add in some more
variables to make the pxeldr Makefile almost identical and thus to ease
maintainability.
- Teach cdldr about the a.out format. Cdldr now parsers the a.out header
of the loader binary and relocates it based on that. The entry point of
the loader no longer has to be hardcoded into cdldr. Also, the boot
info table from mkisofs is no longer required to get a useful cdboot.
- Update the lsdev function for BIOS disks to parse other file systems
(such as DOS FAT) that we currently support. This is still buggy as
it assumes that a floppy with a DOS boot sector actually has a MBR and
parses it as such. I'll be fixing this in the future.
- The biggie: Add in support for booting off of PXE-enabled network
adapters. Currently, we use the TFTP API provided by the PXE BIOS.
Eventually we will switch to using the low-level NIC driver thus
allowing both TFTP and NFS to be used, but for now it's just TFTP.

Submitted by: ps, alfred
Testing by: Benno Rice <benno@netizen.com.au>

$Id$ -> $FreeBSD$

Enable the PCI BIOS PnP enumerator.

Add a dependancy on the BTX crt0 object
Add the biospnp handler to the pnp handler array
Drop some old debugging code

- Enable PnP and ISA PnP code.
- Use the ISA PnP enumerator.
- Use the new linker set code, throw out the gensetdefs stuff.
- Produce an intermediate loader image that has symbols stripped, to aid
- in debugging.
- Supply ISA port access functions required for ISA PnP

Enable the DOS filesystem. NOTE: you will have to rebuild libstand in
order to be able to build again.

Turn on i386-elf

Reactivate the a.out kernel loader code.

Enable the biosdisk driver, duplicate -lstand as it both calls and is called
by the i386 platform library.

Initial integration of the i386 bootloader and BTX.

- Discard large amounts of BIOS-related code in favour of the more compact
BTX vm86 interface.
- Build the loader module as ELF, although the resulting object is a.out,
make gensetdefs 32/64-bit sensitive and use a single copy of it.
- Throw away installboot, as it's no longer required.
- Use direct bcopy operations in the i386_copy module, as BTX
maps the first 16M of memory. Check operations against the
detected size of actual memory.

Resynch with working sources before BTX integration.

- Use format-independant module allocator.
- Conditionalise ISA PnP support.
- Simplify PnP enumerator interface.
- Improve module/object searching.
- Add missing depend/install targets in BTX makefiles.
- Pass the kernel environment and module data in extended bootinfo fields.
- Add a pointer to the end of the kernel + modules in bootinfo.
- Fix parsing of old-style kernel arguments.

This commit was generated by cvs2svn to compensate for changes in r38465,
which included commits to RCS files with non-trunk default branches.

This is the new unified bootstrap, sometimes known previously as the
'three-stage' bootstrap.
There are a number of caveats with the code in its current state:
- The i386 bootstrap only supports booting from a floppy.
- The kernel and kld do not yet know how to deal with the extended
information and module summary passed in.
- PnP-based autodetection and demand loading of modules is not implemented.
- i386 ELF kernel loading is not ready yet.
- The i386 bootstrap is loaded via an ugly blockmap.

On the alpha, both net- and disk-booting (SRM console machines only) is
supported. No blockmaps are used by this code.

Obtained from: Parts from the NetBSD/i386 standalone bootstrap.