Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

zfs boot: export boot/primary pool and vdev guid all the way to kenv

This is work in progress to for znextboot and it also provides
some convenient infrastructure.

MFC after: 20 days

zfs boot: add lszfs command to i386 loader

... to list child datasets of a specified dataset.
Dataset name should be provided in poolname/dsname format.

MFC after: 17 days

loader/i386: replace ugly inb/outb re-implementations with cpufunc.h

Use of __builtin_constant_p in a function that is only called via
a pointer is a good example of how out-of-date it was.

Suggested by: bde
MFC after: 1 week

a bit of trailing whitespace cleanup

MFC after: 1 week

Teach the ZFS use new partitions API when probing.
Note: now ZFS does probe only for partitions with type "freebsd-zfs"
and "freebsd".

zfsboot/zfsloader: support accessing filesystems within a pool

In zfs loader zfs device name format now is "zfs:pool/fs",
fully qualified file path is "zfs:pool/fs:/path/to/file"
loader allows accessing files from various pools and filesystems as well
as changing currdev to a different pool/filesystem.

zfsboot accepts kernel/loader name in a format pool:fs:path/to/file or,
as before, pool:path/to/file; in the latter case a default filesystem
is used (pool root or bootfs). zfsboot passes guids of the selected
pool and dataset to zfsloader to be used as its defaults.

zfs support should be architecture independent and is provided
in a separate library, but architectures wishing to use this zfs support
still have to provide some glue code and their devdesc should be
compatible with zfs_devdesc.
arch_zfs_probe method is used to discover all disk devices that may
be part of ZFS pool(s).

libi386 unconditionally includes zfs support, but some zfs-specific
functions are stubbed out as weak symbols. The strong definitions
are provided in libzfsboot.
This change mean that the size of i386_devspec becomes larger
to match zfs_devspec.

Backward-compatibility shims are provided for recently added sparc64
zfs boot support. Currently that architecture still works the old
way and does not support the new features.

TODO:
- clear up pool root filesystem vs pool bootfs filesystem distinction
- update sparc64 support
- set vfs.root.mountfrom based on currdev (for zfs)

Mid-future TODO:
- loader sub-menu for selecting alternative boot environment

Distant future TODO:
- support accessing snapshots, using a snapshot as readonly root

Reviewed by: marius (sparc64),
Gavin Mu <gavin.mu@gmail.com> (sparc64)
Tested by: Florian Wagner <florian@wagner-flo.net> (x86),
marius (sparc64)
No objections: fs@, hackers@
MFC after: 1 month

r235154 followup: add stddef.h for offsetof

MFC after: 1 month

btxldr: future-proof argument passing from boot1/2-ish to loader

Place the arguments at a fixed offset of 0x800 withing the argument area
(of size 0x1000). Allow variable size extended arguments first of which
should be a size of the extended arguments (including the size
parameter).

Consolidate all related definitions in a new i386/common/bootargs.h header.

Many thanks to jhb and bde for their guidance and reviews.

Reviewed by: jhb, bde
Approved by: jhb
MFC after: 1 month

Improve the algorithm the loader uses to choose a memory range for its
heap when using a range above 1MB.

Previously the loader would always use the last 3MB in the first memory
range above 1MB for the heap. However, this memory range is also where the
kernel and any modules are loaded. If this memory range is "small", then
using the high 3MB for the heap may not leave enough room for the kernel
and modules.

Now the loader will use any range below 4GB for the heap, and the logic to
choose the "high" heap region has moved into biosmem.c. It sets two
variables that the loader can use for a high heap if it desires. When a
high heap is enabled (BZIP2, FireWire, GPT, or ZFS), then the following
memory ranges are preferred for the heap in order from best to worst:
- The largest memory region in the SMAP with a start address greater than
1MB. The memory region must be at least 3MB in length. This leaves the
region starting at 1MB purely for use by the kernel and modules.
- The last 3MB of the memory region starting at 1MB if it is at least 3MB
in size. This matches the current behavior except that the current loader
would break horribly if the first region was not at least 3MB in size.
- The memory range from the end of the loader up to the 640k window. This
is the range the loader uses when none of the high-heap-requesting options
are enabled.

Tested by: hrs
MFC after: 1 week

Various small whitespace and style fixes.

- Make it possible to disable GPT support by setting LOADER_NO_GPT_SUPPORT
in make.conf or src.conf.
- When GPT is enabled (which it is by default), use memory above 1 MB and
leave the memory from the end of the bss to the end of the 640k window
purely for the stack. The loader has grown and now it is much more
common for the heap and stack to grow into each other when both are
located in the 640k window.

PR: kern/129526
MFC after: 1 week

Update ZFS from version 6 to 13 and bring some FreeBSD-specific changes.

This bring huge amount of changes, I'll enumerate only user-visible changes:

- Delegated Administration

Allows regular users to perform ZFS operations, like file system
creation, snapshot creation, etc.

- L2ARC

Level 2 cache for ZFS - allows to use additional disks for cache.
Huge performance improvements mostly for random read of mostly
static content.

- slog

Allow to use additional disks for ZFS Intent Log to speed up
operations like fsync(2).

- vfs.zfs.super_owner

Allows regular users to perform privileged operations on files stored
on ZFS file systems owned by him. Very careful with this one.

- chflags(2)

Not all the flags are supported. This still needs work.

- ZFSBoot

Support to boot off of ZFS pool. Not finished, AFAIK.

Submitted by: dfr

- Snapshot properties

- New failure modes

Before if write requested failed, system paniced. Now one
can select from one of three failure modes:
- panic - panic on write error
- wait - wait for disk to reappear
- continue - serve read requests if possible, block write requests

- Refquota, refreservation properties

Just quota and reservation properties, but don't count space consumed
by children file systems, clones and snapshots.

- Sparse volumes

ZVOLs that don't reserve space in the pool.

- External attributes

Compatible with extattr(2).

- NFSv4-ACLs

Not sure about the status, might not be complete yet.

Submitted by: trasz

- Creation-time properties

- Regression tests for zpool(8) command.

Obtained from: OpenSolaris

- Initialize the vm86 structure to a known-good state. Specifically, always
set the %eflags used during a BIOS call via BTX to 0x202. Previously
the flags field was uninitialized garbage, and thus it was "random" if
interrupts were enabled or not during BIOS calls.
- Use constants from <machine/psl.h> for fields in %eflags.

MFC after: 3 days

Slightly cleanup the 'bootdev' concept on x86 by changing the various
macros to treat the 'slice' field as a real part of the bootdev instead
of as hack that spans two other fields (adaptor (sic) and controller)
that are not used in any modern FreeBSD boot code.

MFC after: 1 week

MFp4: add FireWire/dcons support in loader for i386/amd64.

It is disabled by default. You need to put
LOADER_FIREWIRE_SUPPORT=yes in /etc/make.conf
and rebuild loader to enable it.
(cd /sys/boot/i386 && make clean && make && make install)

You can find a short introduction of dcons at
http://wiki.freebsd.org/DebugWithDcons

Remove an unused variable.

Extend struct devdesc with a unit field, called d_unit. Promote the
device (kind) specific unit field to the common field. This change
allows a future version of libefi to work without requiring anything
more than what is defined in struct devdesc and as such makes it
possible to compile said version of libefi for different platforms
without requiring that those platforms have identical derivatives
of struct devdesc.

Commit the results of the typo hunt by Darren Pilgrim.
This change affects documentation and comments only,
no real code involved.

PR: misc/101245
Submitted by: Darren Pilgrim <darren pilgrim bitfreak org>
Tested by: md5(1)
MFC after: 1 week

For the cases when loading bzip2-compressed kernels enabled use last
3MB of physical memory for heap instead of range between 1MB and 4MB.
This makes this feature working with PAE and amd64 kernels, which are
loaded at 2MB. Teach i386_copyin() to avoid using range allocated by
heap in such case, so that it won't trash heap in the low memory
conditions.

This should make loading bzip2-compressed kernels/modules/mfs images
generally useable, so that re@ team is welcome to evaluate merits
of using this feature in the installation CDs.

Valuable suggestions by: jhb

(forced)
We loaded PAE kernels at 2MB and non-PAE kernels at 4MB to skip
the first PSE page (PSE pages are 2MB on PAE), not at 8MB as it
was incorrectly specified in the previous commit.

Submitted by: jhb

If LOADER_BZIP2_SUPPORT is defined allocate heap in the 1MB-4MB range to
provide enough room for decompression (up to 2.5MB is necessary). This
should be safe to do since we load i386 kernels after 8MB mark now, so
that 16MB is the minimum amount of RAM necessary to even boot FreeBSD.

This makes bzip2-support practically useable.

Cause all flags passed by boot2 to set the respective loader(8)
boot_* variable. The end effect is that all flags from boot2
are now passed to the kernel.

Scan static SMBIOS structures and export the following environment
variables to loader:

hint.smbios.0.enabled "YES" when SMBIOS is detected

hint.smbios.0.bios.vendor BIOS vendor
hint.smbios.0.bios.version BIOS version
hint.smbios.0.bios.reldate BIOS release date

hint.smbios.0.system.maker System manufacturer
hint.smbios.0.system.product System product name
hint.smbios.0.system.version System version number

hint.smbios.0.planar.maker Base board manufacturer
hint.smbios.0.planar.product Base board product name
hint.smbios.0.planar.version Base board version number

hint.smbios.0.chassis.maker Enclosure manufacturer
hint.smbios.0.chassis.version Enclosure version

These strings can be used to detect hardware quirks and to set appropriate
flags. For example, Compaq R3000 series and some HP laptops require

hint.atkbd.0.flags="0x9"

to boot. See amd64/67745 for more detail.

Note: Please do not abuse this feature to resolve general problem when it
can be fixed programmatically. This must be used as a last resort.

PR: kern/81449
Approved by: anholt (mentor)

- Add support to the loader for multiple consoles.
- Teach the i386 and pc98 loaders to honor multiple console requests from
their respective boot2 binaries so that the same console(s) are used in
both boot2 and the loader.
- Since the kernel doesn't support multiple consoles, whichever console is
listed first is treated as the "primary" console and is passed to the
kernel in the boot_howto flags.

PR: kern/66425
Submitted by: Gavin Atkinson gavin at ury dot york dot ac dot uk
MFC after: 1 week

Respect RB_MULTIPLE flag.

Fixed misspellings of 0 as NULL.

Use __FBSDID().
Also some minor copyright style cleanups.

Enable the i386 loader to load and run an amd64 kernel. If this puts
things over floppy size limits, I can exclude it for release builds or
something like that. Most of the changes are to get the load_elf.c file
into a seperate elf32_ or elf64_ namespace so that you can have two
ELF loaders present at once. Note that for 64 bit kernels, it actually
starts up the kernel already in 64 bit mode with paging enabled. This
is really easy because we have a known minimum feature set.

Of note is that for amd64, we have to pass in the bios int 15 0xe821
memory map because once in long mode, you absolutely cannot make VM86
calls. amd64 does not use 'struct bootinfo' at all. It is a pure loader
metadata startup, just like sparc64 and powerpc. Much of the
infrastructure to support this was adapted from sparc64.

Don't call INT 12H anymore in boot program.
Many recent machine have a broken INT 12H (Get base memory size)
implementation and boot program stops if INT 12H is called.

This commit should solve the problem at very first step of FreeBSD
installation occurred on newer some machines.

Reviewed by: bde, jhb
MFC after: 1 week

- If we are booted via cdboot, use bc_add() to instantiate the cd0 device
from the loader.
- Cleanup extract_currdev() some and add support for setting the currdev
to cd0 when booted via cdboot.

Teach the loader how to find the system ACPI information, and autoload
the ACPI module if the system apperars to be ACPI compliant.

This is an initial cut; the load should really be done by Forth support
code, and we should check both the BIOS build date and a blacklist.

Add support for the "nullconsole" console type, for systems with
neither a video console nor a serial port. Use it if the RB_MUTE
flag is set.

Submitted by: Doug Ambrisko <ambrisko@whistle.com>
Reviewed by: jhb

Cleanup warnings. Most of these are signed/unsigned warnings, as well as
some added const's.

Remove the static heap. It is unknown why it was needed in the
beginning, but it no longer is required. This has been tested with
many different revisions of the PXE rom from Intel.

Add a cleanup function. This is needed for PXE where you should
shutdown the UNDI and unload the stack.

Make PXE use the UDP API. This allows for both TFTP and NFS support.
You may specify TFTP or NFS via compile time options in the loader,
but not both at this time.

Also, remove a warning about not knowing how to boot from network
devices. We can obviously do that now.

Mega i386 loader commit.
- Don't hard code 0x10000 as the entry point for the loader. Instead add
src/sys/boot/i386/Makefile.inc which defines a make variable with the
entry point for the loader. Move the loader's entry point up to
0x20000, which makes PXE happy.
- Don't try to use cpp to parse btxldr for the optional BTXLDR_VERBOSE,
instead use m4 to achieve this. Also, add a BTXLDR_VERBOSE knob in the
btxldr Makefile to turn this option on.
- Redo parts of cdldr's Makefile so that it now builds and installs cdboot
instead of having i386/loader/Makefile do that. Also, add in some more
variables to make the pxeldr Makefile almost identical and thus to ease
maintainability.
- Teach cdldr about the a.out format. Cdldr now parsers the a.out header
of the loader binary and relocates it based on that. The entry point of
the loader no longer has to be hardcoded into cdldr. Also, the boot
info table from mkisofs is no longer required to get a useful cdboot.
- Update the lsdev function for BIOS disks to parse other file systems
(such as DOS FAT) that we currently support. This is still buggy as
it assumes that a floppy with a DOS boot sector actually has a MBR and
parses it as such. I'll be fixing this in the future.
- The biggie: Add in support for booting off of PXE-enabled network
adapters. Currently, we use the TFTP API provided by the PXE BIOS.
Eventually we will switch to using the low-level NIC driver thus
allowing both TFTP and NFS to be used, but for now it's just TFTP.

Submitted by: ps, alfred
Testing by: Benno Rice <benno@netizen.com.au>

Add the new cdldr CD bootstrap loader. This patch includes the following:

- Fix btxldr to preserve a NULL bootinfo pointer when it copies the kernel
arguments.
- Add the cdldr bootstrap program. This program is tacked onto the
beginning of the standard 3rd stage boot loader (/boot/loader) to form
the CD boot loader (/boot/cdboot). When a CD is booted, the cdboot file
is copied into memory instead and executed. The cdldr stub emulates the
environment normally provided by boot2 and then starts the loader. This
booting method does not emulate a floppy drive, but boots directly off of
the CD. This should fix the problems some BIOS's have with emulating a
2.88 MB floppy image.
- Add support to the loader to recognize that it has been booted by cdldr
instead of boot2 and use a simpler method of extracting the BIOS boot
device.

Substantially revamp the way that we determine the amount of memory available
for our use. Use the same search order for BIOS memory size functions
as the kernel will later use.

Allow the loader to use all of the detected physical memory (this will
greatly help people trying to load enormous memory disk images).

More correctly handle running out of memory when loading an object.

Use the end of base memory for the top of the heap, rather than
blindly hoping that there is 384k left.

Add copyrights to a couple of files I forgot.

$Id$ -> $FreeBSD$

Implement a simple LRU block cache. By default this is initialised to 16k,
and will bypass transfers for more than 8k. Blocks are invalidated after
2 seconds, so removable media should not confuse the cache.

The 8k threshold is a compromise; all UFS transfers performed by
libstand are 8k or less, so large file reads thrash the cache.
However many filesystem metadata operations are also performed using
8k blocks, so using a lower threshold gives poor performance.

Those of you with an eye for cache algorithms are welcome to tell me
how badly this one sucks; you can start with the 'bcachestats' command
which will print the contents of the cache and access statistics.

Add a dependancy on the BTX crt0 object
Add the biospnp handler to the pnp handler array
Drop some old debugging code

- Enable PnP and ISA PnP code.
- Use the ISA PnP enumerator.
- Use the new linker set code, throw out the gensetdefs stuff.
- Produce an intermediate loader image that has symbols stripped, to aid
- in debugging.
- Supply ISA port access functions required for ISA PnP

Improve the handling of the initial bootdev value from the previous loader.
Remove some unused code.

Missing newline in heap command display.
Noticed by: jkh

Set $currdev according to our best guess at the BIOS device that the previous
bootstrap loaded us from.

The comconsole mode is accessed as 'comconsole' not 'com'.

Recover the arguments passed in from the previous load stage, use them to set
the default console.
Print malloc stats from the new libstand allocator.

Enable the biosdisk driver, duplicate -lstand as it both calls and is called
by the i386 platform library.

Initial integration of the i386 bootloader and BTX.

- Discard large amounts of BIOS-related code in favour of the more compact
BTX vm86 interface.
- Build the loader module as ELF, although the resulting object is a.out,
make gensetdefs 32/64-bit sensitive and use a single copy of it.
- Throw away installboot, as it's no longer required.
- Use direct bcopy operations in the i386_copy module, as BTX
maps the first 16M of memory. Check operations against the
detected size of actual memory.

Resynch with working sources before BTX integration.

- Use format-independant module allocator.
- Conditionalise ISA PnP support.
- Simplify PnP enumerator interface.
- Improve module/object searching.
- Add missing depend/install targets in BTX makefiles.
- Pass the kernel environment and module data in extended bootinfo fields.
- Add a pointer to the end of the kernel + modules in bootinfo.
- Fix parsing of old-style kernel arguments.

Bootstrap updates.

- Move some startup code from MD to MI sections
- Add a 'copyout' and some copyout-related functions. These will be
obsoleted when BTX is available for the 386 and the kernel load
area becomes directly addressable.
- Add the ability load an arbitrary file as a module, associating
and arbitrary type string with it. This can be used eg. for loading
splash-screen images etc.
- Add KLD module dependancy infrastructure. We know how to look for
dependancies inside KLD modules, how to resolve these dependancies
and what to do if things go wrong. Only works for a.out at the
moment, due to lack of an MI ELF loader. Attach KLD module information
to loaded modules as metadata, but don't pass it to the kernel (it
can find it itself).
- Load a.out KLD modules on a page boundary. Only pad the a.out BSS
for the kernel, as it may want to throw symbols away. (We might want
to do this for KLD modules too.)
- Allow commands to be hidden from the '?' display, to avoid cluttering
it with things like 'echo'. Add 'echo'.
- Bring the 'prompt' command into line with the parser syntax.
- Fix the verbose 'ls'; it was using an uninitialised stack variable.
- Add a '-v' flag to 'lsmod' to have it display module metadata as well
(not terribly useful for the average user)
- Support a 'module searchpath' for required modules.
- The bootstrap file on i386 is now called 'loader' to permit the
/boot directory to use that name.
- Discard the old i386 pread() function, as it's replaced by
arch_readin()

Bootloader update.

- Implement a new copyin/readin interface for loading modules.
This allows the module loaders to become MI, reducing code duplication.
- Simplify the search for an image activator for the loaded kernel.
- Use the common module management code for all module metadata.
- Add an 'unload' command that throws everything away.
- Move the a.out module loader to MI code, add support for a.out
kld modules.

Submitted by: Alpha changes fixed by Doug Rabson <dfr@freebsd.org>

This commit was generated by cvs2svn to compensate for changes in r38465,
which included commits to RCS files with non-trunk default branches.

This is the new unified bootstrap, sometimes known previously as the
'three-stage' bootstrap.
There are a number of caveats with the code in its current state:
- The i386 bootstrap only supports booting from a floppy.
- The kernel and kld do not yet know how to deal with the extended
information and module summary passed in.
- PnP-based autodetection and demand loading of modules is not implemented.
- i386 ELF kernel loading is not ready yet.
- The i386 bootstrap is loaded via an ugly blockmap.

On the alpha, both net- and disk-booting (SRM console machines only) is
supported. No blockmaps are used by this code.

Obtained from: Parts from the NetBSD/i386 standalone bootstrap.