MFC r358643:

Link stand/i386 components using a linker script

LLD 10.0.0 changed the behavior of the -Ttext option, so that using
-Ttext=0x0 now causes linking of the loaders to fail with:

ld: error: output file too large: 18446744073707016908 bytes

I reported this in https://bugs.llvm.org/show_bug.cgi?id=44715, and
initially reverted the upstream change in r357259 to work around it.

However, after some discussion with Fangrui Song in the upstream ticket,
I think we can classify this as an unfortunate interaction between using
-Ttext=0 in combination with --no-rosegment. (We added the latter
in r332090, because btxld does not correctly handle input with more
than 2 PT_LOAD segments.)

Fangrui suggested to use a linker script instead, and Warner was already
attempting this in r305353, but had to revert it due to "crypto-using
boot problems" (not sure what those were :).

This review updates the stand/i386/boot.ldscript to handle more
sections, inserts some symbols like _edata and such that we use in
libsa, and also discards any .interp section.

It uses ORG which is defined on the linker command line using
--defsym ORG=value to set the start of all the sections.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D23952

Revert r360605, as it is actually the cause for the linker set errors:

Un-MFC r352032 (by tsoome):

loader: --gc-sections needs sections to work with

--gc-sections is not really useful unless we generate sections with
-ffunction-sections -fdata-sections

While there, i386/loader would win from --gc-sections too.

MFC r352032 (by tsoome):

loader: --gc-sections needs sections to work with

--gc-sections is not really useful unless we generate sections with
-ffunction-sections -fdata-sections

While there, i386/loader would win from --gc-sections too.

MFC Loader Interpreter Coexistance: r337806, r337868, r337914, r337927,
r337989, r338052, r338064, r338407, r338418, r338474

r337806:
Create a loader for each interpreter for x86 BIOS and all EFI

Create loader_{4th,lua,simp}{,.efi}. All of these are installed by
default. Create LOADER_DEFAULT_INTERP to specify the default
interpreter when no other is specified. LOADER_INTERP is the current
interpreter language building. Turn building of lua on by default to
match 4th. simploader is a simplified loader build w/o any interpreter
language (but with a simple loader). This is the historic behavir you
got with WITHOUT_FORTH. Make a hard link to the default loader. This
has to be a hard link rather than the more desirable soft link because
older zfsboot blocks don't support symlinks.

r337868:
stand: Use -Oz/-Os for all loader/stand builds.

While we're not super size constrained, the x86 BIOS /boot/loader has
to be less than about 520k-530k to be reliable. The LUA loader is at
this size today. -Oz saves 15-20% on the size, keeping us safely small
enough (comparable to where we were with the 4th loader). This will
also help with sjg's work on bringing in bearssl, though we may again
be looking for space in the LUA loader.

Size table for clang 6.0.0:
default -O1 -Os -Oz
4th 442368 417792 389120 376832
lua 524288 479232 446464 430080

r337914:
Install links for loader.efi.

r337927:
Add ashldi3 and ashrdi3 to mips.

Now that we're using -Os, mips needs these routines.

r337989:
Copy the boot loader from the new location for the co-existing
loaders.

r338052:
libsa: Add lshrdi3.c for powerpc* and mips

r338064:
Turn back the clock just a little: make userboot.so always be 4th

Turns out there was a hidden dependency we hasn't counted upon. The
host load /boot/userboot.so to boot the VMs it runs. This means that
the change to lua meant suddently that nobody could run their older
VMs because LUA wasn't in 10.0, last month's HardenedBSD, 11.2 or
whatever. Even more than for the /boot/loader* binaries, we need a
good coexistance strategy for this. While that's being designed and
implemented, drop back to always 4th for userboot.so. This will fail
safe in all but the most extreme environments (but lua-only hacks
to .lua files won't be processes in VMs until we fix it).

r338407:
lualoader: Print error messages from command failures at the prompt

Previously lualoader would remain silent, rather than printing
command_errmsg or noting that a command had failed or was not found.

r338418:
userboot: handle guest interpreter mismatches more intelligently

The switch to lualoader creates a problem with userboot: the host is
inclined to build userboot with Lua, but the host userboot's interpreter
must match what's available on the guest. For almost all FreeBSD guests in
the wild, Lua is not yet available and a Lua-based userboot will fail.

This revision updates userboot protocol to version 5, which adds a
swap_interpreter callback to request a different interpreter, and tries to
determine the proper interpreter to be used based on how the guest
/boot/loader is compiled. This is still a bit of a guess, but it's likely
the best possible guess we can make in order to get it right. The
interpreter is now embedded in the resulting executable, so we can open
/boot/loader on the guest and hunt that down to derive the interpreter it
was built with.

Using -l with bhyveload will not allow an intepreter swap, even if the
loader specified happens to be a userboot with the wrong interpreter. We'll
simply complain about the mismatch and bail out.

For legacy guests without the interpreter marker, we assume they're 4th.
For new guests with the interpreter marker, we'll read it and swap over
to the proper interpreter if it doesn't match what the userboot we're using
was compiled with.

Both flavors of userboot are installed by default, userboot_4th.so and
userboot_lua.so. This fixes the build WITHOUT_FORTH as a coincidence, which
was broken by userboot being forced to 4th.

r338474:
Be a little conservative about when to force size optimizations.

Reports have come in that there's issue with powerpc and sparc64 since
we've switched to using -Oz / -Os. We don't strictly need them for
!x86, so be conservative about when we enable them.

MFC GELI Loader Improvements: r336252, r336254, r336256, r336354,
r336532-r336534, r336537, r336626, r337326, r337349, r341071, r341160,
r341420, r341473, r341651, r342793

Note that this MFC contains some seemingly unrelated zfsloader bits -- this
was needed in order to pull in some later fixes for GELI hand-off w/ ZFS
bits included.

r336252:
Extend loader(8) geli support to all architectures and all disk-like devices.

This moves the bulk of the geli support from lib386/biosdisk.c into a new
geli/gelidev.c which implements a devsw-type device whose dv_strategy()
function handles geli decryption. Support for all arches comes from moving
the taste-and-attach code to the devopen() function in libsa.

After opening any DEVT_DISK device, devopen() calls the new function
geli_probe_and_attach(), which will "attach" the geli code to the open_file
struct by creating a geli_devdesc instance to replace the disk_devdesc
instance in the open_file. That routes all IO for the device through the
geli code.

A new public geli_add_key() function is added, to allow arch/vendor-specific
code to add keys obtained from custom hardware or other sources.

With these changes, geli support will be compiled into all variations of
loader(8) on all arches because the default is WITH_LOADER_GELI.

r336254:
Use if rather than case for a simple boolean. gcc thinks blks is
undefined sometimes with the case, but enc is always 0 or 1, so
and if / else is better anyway.

r336256:
Fix glitched indentation (and rewrap as needed due to deeper indent).
No functional changes.

r336354:
zfsboot: fix build with WITHOUT_LOADER_GELI

r336532:
Collapse zfsloader functionality back down into loader.

We no longer really need a separate zfsloader. It was useful when we
were first supporting ZFS and had limited ability to properly boot off
of ZFS without the special boot loader. Now that the boot loader has
matured, go the way loader.efi pioneered and just build one
binary. Change the name of the loader to load in the secondary boot
blocks to be just /boot/loader. Provide a symbolic link from zfsloader
to loader so people who have not upgraded their boot blocks are not
affected. This has the happy benefit of making coexistence easier as
well (fewer binaries in the matrix).

r336533:
Eliminate zfsloader man page.

Remove all cross references to zfsloader.8 and /boot/zfsloader.
Move ZFS specific info into loader.8.

r336534:
NM and OBJCOPY are already defined for all builds. There's no need to
conditionally define them here.

r336537:
Mention zfsloader being folded into loader in UPDATING.

r336626:
Older zfs boot blocks don't support symlinks. install the link to
zfsloader as a hard link. While newer ones do, the whole point of the
link was to transition to the new world order smoothly. A hard link is
less flexible, but it works and will result in fewer bumps. Adjust
UPDATING entry to match.

r337326:
loader: biosdisk.c has leftover geli header.

A small cleanup, remove unneeded #include.

r337349:
zfsboot: Fix startup crash

On a FreeNAS mini XL, with geli encrypted drives the loader crashed in
geli_read().

When we iterate over the list of disks and allocate the zfsdsk structures we
don’t zero out the gdev pointer. In one case that resulted in geli_read()
(called on the bogus pointer) dividing by zero.

Use calloc() to ensure the zfsdsk structure is always zeroed, so the pointer is
initialised to NULL. As a side benefit it gets rid of one #ifdef
LOADER_GELI_SUPPORT.

r341071:
Restore the ability to override the disk unit/partition at the boot: prompt
in gptboot.

When arch-independent geli support was added, a new static 'gdsk' struct
was added, but there was still a static 'dsk' struct, and when you typed
in an alternate disk/partition, the string was parsed into that struct,
which was then never used for anything. Now the string gets parsed into
gdsk.dsk, the struct that's actually used.

r341160:
Add comments describing the bootargs handoff between loader(8) and gptboot
or zfsboot, when loader(8) is the BTX loader. No functional changes.

r341420:
Eliminate duplicated code and struct member definitions in the handoff
of args data between gptboot/zfsboot and loader(8).

Despite what seems like a lot of changes here, there are no actual
changes in behavior, or in the data layout in the structures involved.
This is just eliminating identical code pasted into multiple locations.

In detail, the changes are...

- Move struct zfs_boot_args definition from libsa/zfs/libzfs.h to
i386/common/bootargs.h because it is specific to x86 booting and the
handoff between zfsboot and loader, and has no relation to the zfs
library code in general.

- The geli_boot_args and zfs_boot_args structs both contain an identical
set of member variables containing geli information. Extract this out
to a new geli_boot_data struct, and embed it in the arg-passing structs.

- Provide new routines geli_import_boot_data() and geli_export_boot_data()
that can be shared between gptboot, zfsboot, and loader instead of
pasting identical code into several different .c files.

- Remove some checks for a NULL pointer that can never be true because the
pointer being tested was set using pointer math (kargs + 1) and that can
never result in NULL in this code.

r341473:
Fix args cross-threading between gptboot(8) and loader(8) with zfs support.

When loader(8) is built with zfs support enabled, it assumes that any extarg
data present is a zfs_boot_args struct, but if the first-stage loader was
gptboot(8) the extarg data is actually a geli_boot_args struct. Luckily,
zfsboot(8) and gptzfsboot(8) have always passed KARGS_FLAGS_ZFS along with
KARGS_FLAGS_EXTARG, so we can use KARGS_FLAGS_ZFS to decide whether the
extarg data is a zfs_boot_args struct.

To avoid similar problems in the future, gptboot(8) now passes a new
KARGS_FLAGS_GELI to indicate that extarg data is geli_boot_args. In
loader(8), if the neither KARGS_FLAGS_ZFS nor KARGS_FLAGS_GELI is set but
extarg data is present (which will be the case for gptboot compiled before
this change), we now check for the known size of the geli_boot_args struct
passed by the older versions of gptboot as a way of confirming what type of
extarg data is present.

In a semi-related tidying up, since loader's main() has already decided
what type of extarg data is present and set the global 'zargs' var
accordingly, don't repeat the check in extract_currdev, just check whether
zargs is NULL or not.

r341651:
Don't reference zfs-specific variables if LOADER_ZFS_SUPPORT is undefined
because the variables will be undefined too.

r342793:
MK_ZFS -> {MK_ZFS|MK_LOADER_ZFS}, this is so we can diable userland / kernel
ZFS but keep the boot-loaders when using ZoL port.

Relnotes: yes (GELI support extended)
Relnotes: yes (zfsloader has been collapsed into loader and may be
removed after boot blocks have been updated)

MFC r336084-r336085, r336118: Move zfs impl into libsa

r336084:
Move ZFS files into libsa

Move the libzfs stuff into libsa. There's no need for it to be a
separate library. The separate library adds to the issues of build
ordering that we see from time to time. Move the filesystem support
into libsa, like all the other filesystem support rather than making
zfs the odd-duck out.

Discussed with: allanjude@

r336085:
Stop using ../zfs/libzfs.h but instead use libzfs.h.

While ../zfs/libzfs.h mostly works, there are a few situations where
it does not. Eliminate the problem by using plain libzfs.h, like we do
for ufs support. This fixes the weird cases, and is easier to
understand. It also follows the general style convetion of avoiding
../ in #includes.

r336118:
Remove dirs that git svn should have removed but didn't.

MFC r335321, r335336: stand: move libgeliboot into libsa

r335321:
stand: move libgeliboot into libsa.

Reduce by 1 the number of crazy libraries we need in stand by moving
geli into libsa (where architecturally it belonged all along). This
just moves things around without any code changes.

r335336:
Remove now-empty geli directory

MFC r329726, r329737, r329740

r329726:
Honor settings for including / excluding cd9660, ufs, ext2fs and msdos.

The Makefile gives the impression that ext2fs and msdos were excluded
(they weren't) and that you could exclude cd9660 and ufs support (you
couldn't). Allow those to be excluded.

We need to look, in the future, at trimming the number of supported
filesystems, and this will make that easier.

r329737:
Purely whitespace changes bringing this file closer to style(9).

Curiously, changing whitespace seems to cause the md5 of the .o files to differ
these days hence the following testing strategy:

Tested by: objdump -d | md5 (both in-tree clang and lang/gcc6)

r329740:
Further style(9) changes.

Tested by: objdump -d | md5 (both in-tree clang and lang/gcc6)

MFC Loader Fixes Final: r327612,r327703,r327704,r327878,r327879,r327881,
r328007,r328029,r328030,r328031,r328061,r328156,r328169,r328288,r328289,
r328290,r328291,r328292,r328411,r328536,r328603,r328614,r328642,r328769,
r328779,r328780,r328781,r328782,r328783,r328806,r328808,r328826,r328835,
r328911,r328986,r328987,r328990,r328999,r329000,r329019,r329050,r329054,
r329060

r327612: Invent new #defines for the biospci_{read,write}_config function

r327703: Define __dmadat after #include'ing ufsread.c.

r327704: Fix printf missing format variables warnings.

r327878: Add GUID for UEFI boot manager variables.

r327879: Report the boot order and where we are in that boot order.

r327881: Allow this file to be included

r328007: Fix booting on some arm64 systems after r327879 by fixing the call
to utf8_to_ucs2

r328029: When returning an error and freeing allocated memory from
ucs2_to_utf8, NULL the return pointer.

r328030: Check the return value from utf8_to_ucs2 instead of whether or not
uv is NULL.

r328031: Need to free uv after we're done using it.

r328061: utf8_to_ucs2() should check for malloc failure

r328156: stand: Move sections around to fix stand/ build with ld.lld on
armv7

r328169: Remove extra copy of bootinfo.c. It's a bit rotted copy of the one
in efi/loader.

r328288: Fix some resource leaks.

r328289: Don't leak memory when displaying help.

r328290: On malloc failure, be sure to close the include file that triggered
it.

r328291: getenv does not return tainted data in the boot loader. Attempt to
clue Coverity into that fact.

r328292: There's no tainted data here, tag it as such to avoid false
positives.

r328411: loader.efi: add missing EFI GUIDs

r328536: loader: support for mixed-endianness ELF/loader and POWER8

r328603: Add missing non-POWERPC case to give the scr value something
non-zero.

r328614: Move libstand.3 to libsa.3. Update libsa.3 to include functions

r328642: Break out the interpreters (simple and forth) w/o ifdefs.

r328769: Centralize several variables.

r328779: Retire pnp.4th and the code needed only for 4th words used here.

r328780: These 4th words were an attempt to allow integration into the boot
loader scripts. However, that path won't be taken after all it
seems.

r328781: Remove pcibios forth support.

r328782: Now that we no longer conditionally compile some files outside of
ficl

r328783: Invent new LDR_INTERP for the loader interpreter to use.

r328806: We need more heap space to properly load newer powerpc kernels.

r328808: Implement strcoll as strcmp.

r328826: Make cross-endian loader changes apply only to powerpc

r328835: Fix regression introduced in r328806, preventing boot on many
platforms.

r328911: Ignore relocation tables for non-memory-resident sections.

r328986: Fix relative location of USB sources after recent move.

r328987: A more definitions to kernel emulation shim in order to build
stand/usb.

r328990: Move the stand/usb test loader into its own directory.

r328999: Fix indentation to FreeBSD standard for interp files

r329000: Move simple interpreter 'perform' into interp.c and call it

r329019: Move to tabs for indentation and to 8-space notches, per style(9).

r329050: Fix build of userboot.so

r329054: Set script.lang in the environment to either 'forth' or 'simple' to
reflect what scripting language was compiled into the loader.

r329060: loader: fix endianness conversion

PR: 225323

MFC Loader Fixes 2017q4p7: r324844,r326089,r326926,r326440,r326484,r326494,
r326588,r326708,r326784,r326914,r327390,r328446,r326090,r326143,r326144,
r326182,r326384,r326421,r326440,r326441,r326442,r326443,r326444,r326445,
r326446,r326447,r326448,r326484,r326485,r326486,r326487,r326488,r326490,
r326491,r326492,r326493,r326494,r326495,r326504,r326507,r326509,r326584,
r326585,r326586,r326587,r326588,r326589,r326590,r326591,r326592,r326593,
r326594,r326600,r326616,r326671,r326707,r326708,r326709,r326710,r326711,
r326712,r326714,r326720,r326768,r326772,r326784,r326792,r326812,r326854,
r326855,r326856,r326858,r326886,r326887,r326914,r326926,r326927,r326960,
r326961,r326962,r326963,r327351,r327453,r327390,r327523,r327524,r326489,
r327880,r328437,r328438,r328439,r328441,r328446,r328448,r328449,r328612,
r328613,r328615

While here, undo our libfdt hack of not including <stdlib.h> if we're
compiling _STANDALONE.

r324844: When building standalone, don't define errno. Let the definition from
stand.h override. This is similar to what we do in the kernel.

r326089: loader.efi: efipart does not recognize partitionless disks

r326090: net_parse_rootpath() has no parameters

r326143: Fix theoretical integer overflow issues. If the product here is

r326144: Mark the func pointer as __dead2. It looks up loader_main, which

r326182: Modify all FreeBSD bootloaders on PowerPC AIM (Book-S) systems

r326384: Use const pointers to avoid casting away constness.

r326421: loader.efi: efipart should exclude iPXE stub block protocol

r326440: Remove stale dependency on ufsread.c

r326441: Minor flags cleanup

r326442: Cleanup CFALGS usage here

r326443: We don't need both _STAND and _STANDALONE, use the latter.

r326444: Move geli to common DO32 stuff

r326445: Fix random() and srandom() prototypes to match the standard.

r326446: Undefine _STANDALONE since this is test code.

r326447: Tweaks to the beri boot loader so that it builds w/o warnings.

r326448: Fix all warnings related to geli and ZFS support on x86.

r326484: Const poison the propname.

r326485: Delcare md_load in libofw.h. Make all prototypes match for ofw

r326486: Include machine/md_var to pick up __syncicache prototype.

r326487: Cast mdp (a vm_offset_t) to void * to match prototype.

r326488: e_entry can be smaller than a pointer. Cast it to an intptr_t

r326490: Declare our strange brand of main().

r326491: Disconnet ps3 from the build. There's too many warnings to fix.

r326492: Cast void * pointer to char * so the arg matches the %s format.

r326493: Provide a md_load64 prototype.

r326494: Mark two things as unused (since they are only sometimes used)

r326495: Now it's safe to bump WARNS to 1.

r326504: Switch to proper MK_LOADER_GELI tests.

r326507: increase maximum size of zfsboot

r326509: loader.efi: add note about iPXE into the efipart.c

r326584: When building standalone, include stand.h rather than the kernel

r326585: Include ficl.h before anything else

r326586: No need to include the userland md5.h, the kernel one is just fine.

r326587: Use the kernel relative paths, rather than the userland relative
paths

r326588: Need to include skein in the include path

r326589: Make sure we include the right path for skein.h

r326590: Prefer stdint.h to inttypes.h

r326591: This isn't NetBSD specific code. Include these for any kernel /

r326592: Don't inherit CFLAGS. This a specialized test program.

r326593: Stop building with the standard system headers.

r326594: Now that we offer a semi-sane standards-ish set of #include files,
stop hacking includes with sed.

r326600: Since this is contrib code, create an upstreamable version of my

r326616: dhcp_try_rfc1048() is not used any more

r326671: Avoid setting -Wno-tentative-definition-incomplete-type with gcc.

r326707: Add partial support signal.h functioanlity. Pull in
machine/signal.h

r326708: Remove _KERNEL hack now that errno.h does the right thing when
built standalone.

r326709: Provide implementations for iscntrl, ispunct and isgraph.

r326710: Put the files we're copying over into a few variables and clean
hings up.

r326711: Const poison a couple of interfaces.

r326712: Create interp class.

r326714: boot1.c needs EFI_ZFS_BOOT too, so add it globally.

r326720: This path belongs in ficl/Makefile, not the common defines for
users

r326768: Fix a comment to be more accurate

r326772: Fix regression with lua import

r326784: Revert part of 362772. It was causing problems for includes

r326792: Attempt to unbreak buildworld

r326812: Revert r326792, r326784, r326772, r326712

r326854: libefi: make efichar.h more usable in stand code

r326855: Cargo cut a fix for the regressions r326585 caused.

r326856: Fix comments after bump in size.

r326858: Revert r326855: Cargo cut a fix for the regressions r326585 caused.

r326886: Panic in sbrk if setheap hasn't been called yet. This is preferable
o a mysterious crash

r326887: Remove the 'mini libstand in libstand' that util.[ch] provided.

r326914: Move loader-only defines to loader.mk from defs.mk

r326926: Move loader help file definitions to being 100% inside of
loader.mk.

r326927: libficl is only ever used in a loader (never a boot) program. Move
it.

r326960: Simplify things a little. The RETURN macro isn't required.

r326961: Interact is always called with NULL. Simplify code a little

r326962: Hoist btx include stuff to i386/Makefile.inc

r326963: No need to use relative paths like this here.

r327351: Fix ubldr. uboot/lib uses defines for the loader.

r327453: Add a validbcd() routine that uses the bcd2bin_data[] array

r327390: Garbage-collect loader.ps3. It is currently disconnected from the
build and kboot replaces.

r327523: Don't clobber system LDFLAGS for beri boot loaders.

r327524: Use 'extern uint8_t' instead of 'extern void' for external symbols.

r326489: Allow this file to be used in libsa without warning...

r327880: Move getsecs() prototype to stand.h from net.h so it can be used

r328437: Split panic routine

r328438: Implement abort() as a call to panic.

r328439: Provide abs form stdlib.h.

r328441: abort() should be marked __dead2 since it won't return.

r328446: Now that exit is __dead2, we need to tag ub_exit() as __dead2.

r328448: Make exit() never return until host_exit can be written.

r328449: Tag unreachable places as such. I left the while (1); in place

r328612: Move strtold wrapper from strtol.c to its own strtold.c.

r328613: Kill copies of strtol and strtoul.

r328615: Update stand.h for changes for strto*l

PR: 223969

MFC r325834,r325997,326502: Move sys/boot to stand/

This is effectively a direct commit to stable/11, due to differences between
stable/11 and head. Changes to DTS in sys/boot/fdt/dts were often
accompanied by kernel changes. Many of these were also risc-v updates that
likely had many more dependencies to MFC.

Because of this, sys/boot/fdt/dts remains as-is while everything else in
sys/boot relocates to stand/.

r325834: Move sys/boot to stand. Fix all references to new location

r325997: Remove empty directories.

r326502: Document the sys/boot -> stand move in hier.7 and the top-level README.

MFC Loader Fixes 2017q4p6: r324649,r324650,r324651,r324652,r324653,r324654,
r324700,r324702,r324709,r324717,r324719,r324841,r324842,r324843,r324845,
r324850,r324876,r324877,r324878,r324879,r324880,r324881,r324883,r324980,
r324981,r324982,r324995,r325014,r325093,r325094,r325114,r325170,r325171,
r325172,r325173,r325174,r325175,r325176,r325248,r325286,r325310,r325332,
r325338,r325339,r325376,r325377,r325379,r325380,r325382,r325478,r325479,
r325480,r325482,r325483,r325484,r325485,r325556,r325641,r325681,r325685,
r325686,r325687,r325688,r325689,r325690,r325691,r325692,r325693,r325694,
r325743,r325744,r325748,r325775,r325779,r325780

r324649: Move common/Makefile.inc to sys/boot/loader.mk.

r324650: tweak style

r324651: create defs.mk for common definitions

r324652: Move all the ficl common code into ficl.mk

r324653: LOADER_foo_SUPPORTED

r324654: Explicitly inlcude SYSDIR in the include path

r324700: loader: initialize dv_cleanup in md.c to eliminate clang warning

r324702: We need to include disk.o in libuboot.a when we're building with
support for disk access.

r324709: Revert "Unify boot1 with loader" change r324646

r324717: libsa/ip.c: misplaced comment, ip_v is half char, not ip_p

r324719: libsa/ip: stop read loop on bad fragments

r324841: Use BOOTDIR more consistently in defs.mk rather than repeat
sys/boot.

r324842: Introduce BOOTOBJ: The top level object directory for the boot tree

r324843: Stopgap fix to the mistmatch between LOADER_GELI_SUPPORT and
LOADER_NO_GELI_SUPPORT.

r324845: Use BOOTOBJ and BOOTDIR to find geli includes and libraries.

r324850: Define LIBSA32 to LIBSA on i386 to fix build.

r324876: Move fdt and uboot defines into common uboot.mk.

r324877: End source directories with SRC rather than a hodgepodge of names

r324878: Make at91 boot loader compile again.

r324879: Prefer SRCTOP paths for bits we're grabbing from libc.

r324880: Use BOOTSRC here.

r324881: Use SYSDIR instead of ${.CURDIR}/../..<etc>/sys.

r324883: Use preferred defined paths, rather than relative paths in fdt.

r324980: Use BOOTDIR consistently.

r324981: Move BINDIR definition to defs.mk, and override where it isn't
/boot

r324982: Remove sys/boot/arm/at91 and ixp425

r324995: loader.efi: Make framebuffer commands available for arm64

r325014: Add a 'place holder' arm struct efi_fb until a real one comes

r325093: Define new EFI variables

r325094: Cleanup non-arch Makefiles

r325114: Use defs.mk defins in most MD code

r325170: Use defs.mk values for userboot

r325171: Use defs.mk name and prefer bsd.init.mk

r325172: Remove the -nostdlib stuff I added. Instead, fix LDFLAGS to be
honored correctly with the new Makefile.inc include order.

r325173: We don't need to build a special ficl for userboot.

r325174: Minor cleanup

r325175: For amd64, compile both zfs and zfs32 libraries.

r325176: Actually add zfs32/Makefile

r325248: loader ptblread() is broken with >512B sectors

r325286: efipart_strategy is using wrong offset with >512B sectors

r325310: zfs.c:vdev_read() needs to be careful about large sectors

r325332: loader: re-enable gzip support for x86

r325338: loader: fix BOOTSRC -> BOOTOBJ in a library path

r325339: This used to have bzip2 support too.

r325376: WIP: centralize machine links

r325377: mostly libsa

r325379: Revert "mostly libsa"

r325380: Revert "WIP: centralize machine links"

r325382: Cleanup stray libstand names to be libsa names.

r325478: Powerpc is a 32-bit boot loader.

r325479: Define LIBFICL32 to be libficl.a on i386 and libficl32.a on amd64.

r325480: Use DO32 for all the places that we need to flag we're building a
32-bit version of a library.

r325482: Move machine and other link creation to defs.mk

r325483: MACHINE can never be powerpc64, so cleanup code that thinks it can.

r325484: Prefer bsd.init.mk to src.opts.mk

r325485: Centralize all 32-bit builds on 64-bit platform stuff.

r325556: loader: set options before including bsd.init.mk

r325641: loader.efi: efi_devpath_is_prefix should return bool

r325681: boot1: avoid using NULL device path

r325685: libsa32 isn't needed for i386. It's already a 32-bit platform.

r325686: Simplify this if to a direct assignment.

r325687: Remove all the empty help files from the powerpc build.

r325688: FDT support doesn't make sense for ps3, remove it.

r325689: Remove LOADER_FDT_SUPPORT as a Makefile variable.

r325690: Remove LOADER_ZFS_SUPPORT as a Makefile variable

r325691: Remove useless PNP define here.

r325692: Replace LOADER_FIREWIRE_SUPPORT variable

r325693: Move LOADER_{NO,}_GELI_SUPPORT to MK_LOADER_GELI

r325694: Install the 4th files in sys/boot/forth instead of each loader

r325743: Make sure the proper loader.rc gets installed.

r325744: boot1: also check for NULL device

r325748: Use proper include file.

r325775: Add loader.conf to the list of files that are MD.

r325779: Add /boot/dts to the list of default modules.

r325780: Don't add /boot/dt*s* but /boot/dt*b*. Stupid think-o.

MFC Loader Fixes 2017q4p3: r324460,r324471

r324460: Rather than laying whack-a-mole with including the path to stand.h,
always include it.

r324471: Fix typos: `stand alon{e,g}` should be spelled like `standalone`

MFC Loader Fixes 2017q4p1: r324321,r324359,r324360,r324388,r324449,r324450,
r324451,r324452

r324321: This README file was quite relevant for FreeBSD 3 or 4. No more.

r324359: Add efi_devpath_is_prefix

r324360: Encapsulate ZFS preferences into efi_zfs_is_preferred

r324388: Unbreak building efiboot with MK_ZFS == no

r324449: Prefer ${LIBSTAND} to -lstand

r324450: Define SASRC and use it

r324451: Add missing -I${SASRC} in a couple of places so that stand.h
doesn't accidentally come in via host pollution.

r324452: Define LIBSA* and use them instead of overloaded LIBSTAND

MFC Loader Fixes 2017q2: r316437,r316577,r316578,r316585,r316590,r316612,
r316625,r316628,r316654,r316682,r316704,r316771,r317092,r317097,r317099,
r317652,r317785,r317886,r317887,r318142,r318320,r318356,r318678,r318754,
r318982,r318986,r318987,r318988,r318989,r318990,r318991,r318992,r318993,
r318994,r318999,r319083,r319084,r319085,r320011,r320234,r320288,r320304,
r320467,r320482

r316437: Small cleanup to make i386/loader match efi/loader boot environment
code

r316577: loader: part.c cstyle cleanup

r316578: loader: want mechanism to avoid RA with bcache

r316585: loader: zfs reader should check all labels

r316590: libstand/dosfs: cache FAT32 in 128 Kb blocks to save loader memory

r316612: In r298230 the value of HEAP_MIN was changed from 3MB to 64MB.
Correct a comment.

r316625: Do not use -msoft-float with intention of disabling FP on aarch64

r316628: Silence GCC warning by initializing the local variable.

r316654: loader: r316585 did miss userboot update

r316682: loader: r316585 did miss sparc/ofw

r316704: loader.efi: only fetch zfs pool guid for the actual boot device

r316771: loader: Avoid possible overflow via environment variable

r317092: loader: zfs reader vdev_probe should check for minimum device size

r317097: loader: F_READ/F_WRITE should be checked against masked flag

r317099: loader: uboot disk ioctl should call disk_ioctl

r317652: loader.efi: ResetSystem does not use data with EFI_SUCCESS

r317785: zfsboot: drvsize() may be unusable on some systems

r317886: distinguish NFS versus TFTP boot by rootpath

r317887: loader: network read rework

r318142: libstand: NULL pointer dereference in rarp

r318320: loader: add ip layer code into libstand

r318356: libstand: increase nfs max read size to 16k

r318678: Replacing iterating over rootpath by strsep(3).

r318754: Pass -N directly to ld via -Wl rather than passing it to the
compiler driver.

r318982: Pass a "FREEBSD" user-class in PXE dhcp request

r318986: add a comment on vendor index 19 and 20 to avoid confusion

r318987: Support URI scheme for root-path in netbooting

r318988: Always build tftpfs support along with nfs for pxeboot

r318989: Always issue the pxe request

r318990: Partially revert r314948

r318991: Document recent changes on pxeboot

r318992: Capitalize DHCP

r318993: Use the usual FreeBSD spelling for the DHCP user class

r318994: Catch with the change in the user class

r318999: Update the comments concerning net_parse_rootpath to reflect what
it is now

r319083: Followup on the user-class changes

r319084: Small cleanup in dev_net.c

r319085: use the same option list for dhcp discovery and request

r320011: Add chain loader support for loader

r320234: Make structure padding explicit in EFI_MEMORY_DESCRIPTOR

r320288: Allow Clang's integrated assembler to assemble boot0

r320304: loader.efi: Disable smbios for arm

r320467: loader: chain load relocate data declaration is bad

r320482: As with arm64 mark the EFI PE header as allocated on arm.

PR: 218473

MFC Loader Fixes 2017q1: r311458,r312237,r312314,r312374,r312947,r313042,
r313047,r313166,r313328,r313332,r313333,r313337,r313348,r313349,r313389,
r313442,r313451,r313575,r313645,r313710,r314114,r314213,r314275,r314945,
r314948,r315008,r315408,r315427,r315645,r315646,r315648,r315653,r315850,
r316064,r316078,r316079,r316100,r316104,r316111,r316112,r316171,r316279,
r316280,r316287,r316311,r316343,r316424,r316436

r311458: Use compiler driver to link BERI boot loaders

r312237: loader.efi: find_currdev() can leak memory

r312314: loader: move device path definitions to include/efidevp.h

r312374: loader: efi devpath api usage should be more aware of NULL pointers

r312947: Remove "-Xassembler -G0" from CFLAGS.

r313042: loader.efi environment related cleanups

r313047: loader: disk/part api needs to use uint64_t offsets

r313166: loader: libefi/env.c warnings in arm build

r313328: loader: Implement disk_ioctl() to support DIOCGSECTORSIZE and
DIOCGMEDIASIZE.

r313332: loader: bcache read ahead block count should take account the large
sectors

r313333: loader: Replace EFI part devices.

r313337: loader: 313329 missed ZFS guard in loader/main.c

r313348: loader: biosdisk fix for 2+TB disks

r313349: loader: disk io should not use alloca()

r313389: efipart is also using the '%S' printf format, add -Wno-format for
it.

r313442: loader: possible NULL pointer dereference in efipart.c

r313451: loader: possible NULL pointer dereference in bcache.c

r313575: makefs: make the buffer functions look exactly like the kernel ones

r313645: loader: implement MEDIA_FILEPATH_DP support in efipart

r313710: loader: cstyle fixes and DIOCGMEDIASIZE should use uint64_t

r314114: Use LDFLAGS with CC instead of _LDFLAGS.

r314213: Remove control+r handling from geliboot's pwgets()

r314275: Remove unused macro from common/drv.c.

r314945: Some style(9) fixes. No functional changes.

r314948: Try to extract the RFC1048 data from PXE.

r315008: r314948 seems to be missing a variable or two that will break

r315408: loader: remove open_disk cache

r315427: loader: biosdisk should report IO error from INT13

r315645: loader: disk_cleanup was left in userboot_disk.c

r315646: loader: pxe.h constants have wrong values

r315648: libstand: verify value provided by nfs.read_size

r315653: loader: verify the value from dhcp.interface-mtu and use snprintf
o set mtu

r315850: The original author abused Nd (one-line description, used by
makewhatis)

r316064: Fix build with path names with 'align' or 'nop' in them.

r316078: gpt*boot: Save a bit more memory when LOADER_NO_GELI_SUPPORT is
specified

r316079: Simply retire the sedification of the boot2.s file.

r316100: Remove -fno-guess-branch-probability and -fno-unit-at-a-time.

r316104: Use `NO_WCAST_ALIGN` instead of spelling it out as -Wno-cast-align
in CFLAGS

r316111: loader: move bios getsecs into time.c

r316112: loader: ls command should display file types properly

r316171: xfsread inlined uses more space, so remove the inline tag.

r316279: loader: efipart should check disk size from partition table

r316280: loader: simplify efi_zfs_probe and avoid double probing for zfs.

r316287: Remove OLD_NFSV2 from loader and libstand

r316311: Add explicit_bzero() to libstand, and switch GELIBoot to using it

r316343: Implement boot-time encryption key passing (keybuf)

r316424: Fix sparc64 build broken by r316343 and r316076

r316436: Restore EFI boot environment functionality broken in r313333

PR: 216940 217298 217935

MFC r307322,r307323,r307324,r307326,r307327,r307338,r307879,r307908,r307911,
r307942,r307950,r307951,r307954,r307955,r308125,r308195,r308476,r308534,
r308535,r308776,r308843,r310236,r310726: Loader fixes, 2016q4

r307322: Remove /boot/boot.conf, deprecated for 16 years

r307323: Remove fetching of pInterp.

r307324: Create a new linker set, Xficl_compile_set which contains a list of
functions to call to register new forth words.

r307326: In UEFI mode expose the SMBIOS anchor base address via kenv

r307327: Update i386 build of loader.efi (but leave it disabled) so that we
at least build it now.

r307338: Create a pcibios-version environment FORTH word.

r307879: Preliminary support for EFI in boot loader.

r307908: Fix the build on both arm64 and when WITHOUT_FORTH is defined.

r307911: Add better comment...

r307942: Really make WITHOUT_FORTH (MK_FORTH==no) work.

r307950: Add it (Makefile.ficl) to the right place

r307951: Fix two backwards tests.

r307954: Back out the move to the loader script from -N.

r307955: LIBSTAND goes last, so put it last here too.

r308125: In loader.efi, instead of exiting directly, try to fallback.

r308195: efinet_dev_print should honor verbose option.

r308476: boot/forth spelling issue in forth word

r308534: The file_loadraw function grew an argument, update install function

r308535: Add support for LOADER_RC setting in the pkgfs manifes

r308776: loader: zfs toplevel vdev must have spa set.

r308843: loader: smbios version check is not correct

r310236: Renumber license clauses to avoid skipping #3

r310726: cdboot: add explict suffix to ambiguous or instruction

PR: 214375

MFC r310267: Deduplicate loader vers.c Makefile rules

The Makefile rule to create vers.c for loader version info was
previously duplicated in each of the various loader Makefiles.
Instead, share a common rule in Makefile.inc.

Sponsored by: The FreeBSD Foundation

MFC r310661: loader: use strip -o instead of cp and strip in place

It is simpler and cleaner to have strip produce the stripped output
directly than copying the input to the output first.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Implement GELI (AES-XTS and AES-CBC only) in gptboot and gptzfsboot

Allows booting from a GELI encrypted root file system, via UFS or ZFS

Reviewed by: gnn, smh (previous version), delphij (previous version)
Relnotes: yes
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D4593

Add more .NOMETA missed in r291320

Sponsored by: EMC / Isilon Storage Division

Revert r269030. CLEANFILES is already added to .NOPATH since r241298.

Sponsored by: EMC / Isilon Storage Division

Remove guards around overwriting loader.rc and menu.rc

There have been .local version of each for user modifications for some time
This allows users to receive future updates to these files

PR: 183765
Submitted by: Bertram Scharpf, Nikolai Lifanov (patch)
Reviewed by: dteske, loos, eadler
Approved by: bapt (mentor)
MFC after: 1 month
Relnotes: yes
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D3176

Install loader(8) and zfsloader(8) only once

Differential Revision: https://reviews.freebsd.org/D2841
Reviewed by: imp

Revert .PATH changes to fix mips build.

Reported by: bz
Pointy hat to: jkim

Install newly added brand-*.4th and logo-*.4th files and reduce duplication.

Reviewed by: dteske
Pointy hat to: dteske

Add support for calling pcibios routines from the
bootloader. Implement the following routines:
pcibios-device-count count the number of instances of a devid
pcibios-read-config read pci config space
pcibios-write-config write pci config space
pcibios-find-devclass find the nth device with a given devclass
pcibios-find-device find the nth device with a given devid
pcibios-locator convert bus device function ti pcibios locator
These commands are thin wrappers over their PCI BIOS 2.1 counterparts. More
informaiton, such as it is, can be found in the standard.

Export a nunmber of pcibios.X variables into the environment to report
what the PCI IDENTIFY command returned.

Also implmenet a new command line primitive (pci-device-count), but don't
include it by default just yet, since it depends on the recently added
words and any errors here can render a system unbootable.

This is intended to allow the boot loader to do special things based
on the hardware it finds. This could be have special settings that are
optimized for the specific cards, or even loading special drivers. It
goes without saying that writing to pci config space should not be
done without a just cause and a sound mind.

Sponsored by: Netflix

Revert r267233 for now. PIE support needs to be reworked.

1. 50+% of NO_PIE use is fixed by adding -fPIC to INTERNALLIB and other
build-only utility libraries.
2. Another 40% is fixed by generating _pic.a variants of various libraries.
3. Some of the NO_PIE use is a bit absurd as it is disabling PIE (and ASLR)
where it never would work anyhow, such as csu or loader. This suggests
there may be better ways of adding support to the tree. Many of these
cases can be fixed such that -fPIE will work but there is really no
reason to have it in those cases.
4. Some of the uses are working around hacks done to some Makefiles that are
really building libraries but have been using bsd.prog.mk because the code
is cleaner. Had they been using bsd.lib.mk then NO_PIE would not have
been needed.

We likely do want to enable PIE by default (opt-out) for non-tree consumers
(such as ports). For in-tree though we probably want to only enable PIE
(opt-in) for common attack targets such as remote service daemons and setuid
utilities. This is also a great performance compromise since ASLR is expected
to reduce performance. As such it does not make sense to enable it in all
utilities such as ls(1) that have little benefit to having it enabled.

Reported by: kib

Add .NOPATH to the machine target so that builds in sys/boot work and don't
error out building zfs loader.

Submitted by: sjg@freebsd.org

In preparation for ASLR [1] support add WITH_PIE to support building with -fPIE.

This is currently an opt-in build flag. Once ASLR support is ready and stable
it should changed to opt-out and be enabled by default along with ASLR.

Each application Makefile uses opt-out to ensure that ASLR will be enabled by
default in new directories when the system is compiled with PIE/ASLR. [2]

Mark known build failures as NO_PIE for now.

The only known runtime failure was rtld.

[1] http://www.bsdcan.org/2014/schedule/events/452.en.html
Submitted by: Shawn Webb <lattera@gmail.com>
Discussed between: des@ and Shawn Webb [2]

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

Build boot/ficl as 64-bit library on amd64

The 32-bit bootloaders on amd64 now use the 32-bit version in ficl32,
as is done with libstand32. The native 64-bit ficl will be used by the
upcoming UEFI loader.

Sponsored by: The FreeBSD Foundation

Build a 32-bit libstand under sys/boot/

A 32-bit libstand is needed on 64-bit platforms for use by various
bootloaders. Previously only the 32-bit version was built, installed as
/usr/lib/libstand.a.

A new 64-bit libstand consumer will arrive in the near future, so move
the bootloader-specific 32-bit version to sys/boot/libstand32/.

Explicitly link against this version in the 32-bit loaders.

Sponsored by: The FreeBSD Foundation

Hook in new files menusets.4th and manual.

Approved by: adrian (co-mentor) (implicit)

Reduce the number of attempts to detect proper kld format for the amd64
loader.

Remove unneeded flag.

Import work done under project/nand (@235533) into head.

The NAND Flash environment consists of several distinct components:
- NAND framework (drivers harness for NAND controllers and NAND chips)
- NAND simulator (NANDsim)
- NAND file system (NAND FS)
- Companion tools and utilities
- Documentation (manual pages)

This work is still experimental. Please use with caution.

Obtained from: Semihalf
Supported by: FreeBSD Foundation, Juniper Networks

i386 zfsloader: rename LIBZFS to LIBZFSBOOT

... to avoid name clash with the other libzfs

MFC after: 1 month

- Whitespaces fixed only

Approved by: rwatson (mentor)

New boot loader menus from Devin Teske.
Discussed on hackers and recommended for inclusion into 9.0 at the devsummit.
All support email to devin dteske at vicor dot ignoreme dot com .

Submitted by: dteske at vicor dot ignoreme dot com
Reviewed by: me and many others

For consistency, change all 'i386' and MACHINE_ARCH to x86.

Reviewed by: jhb
MFC after: 1 week

MF tbemd: Minor tweaks, prefer MACHINE_CPUARCH generally to MACHINE_ARCH

Correctly setup LDADD with regards to libstand. The submitter points
out that "on amd64, libstand.a is compiled for i386, but is still installed
under ${WORLDTMP}/usr/lib instead of ${WORLDTMP}/usr/lib32. Even if it
would be installed there, ld on amd64 is set up incorrectly with a
${TOOLS_PREFIX}/usr/lib/i386 default path, so it wouldn't link. The reason
it does link under gcc is that gcc passes -L${WORLDTMP}/usr/lib twice,
even for -m32 builds, which is also incorrect, but accidentally works in
this case."

Submitted by: Dimitry Andric <dimitry at andric.com>

Create a seperate ZFS enabled loader.

This adds zfsloader which will be called by zfsboot/gptzfsboot code rather
than the tradional loader. This eliminates the need to set the
LOADER_ZFS_SUPPORT variable in order to get a ZFS enabled loader.

Note however, that you must reinstall your bootcode (zfsboot/gptzfsboot)
in order for the boot process to use the new loader.

New installations will no longer be required to build a ZFS enabled
loader for a working ZFS boot system. Installing zfsboot/gptzfsboot is
sufficient for acknowledging the use of CDDL code and therefore the ZFS
enabled loader.

Based on a previous patch from jhb@

Reviewed by: jhb@
MFC after: 2 weeks

- Make it possible to disable GPT support by setting LOADER_NO_GPT_SUPPORT
in make.conf or src.conf.
- When GPT is enabled (which it is by default), use memory above 1 MB and
leave the memory from the end of the bss to the end of the 640k window
purely for the stack. The loader has grown and now it is much more
common for the heap and stack to grow into each other when both are
located in the 640k window.

PR: kern/129526
MFC after: 1 week

Fix build when WITH_SSP is set explicitly.

Submitted by: Jeremie Le Hen

Fix building without ZFS (can't find library)

Update ZFS from version 6 to 13 and bring some FreeBSD-specific changes.

This bring huge amount of changes, I'll enumerate only user-visible changes:

- Delegated Administration

Allows regular users to perform ZFS operations, like file system
creation, snapshot creation, etc.

- L2ARC

Level 2 cache for ZFS - allows to use additional disks for cache.
Huge performance improvements mostly for random read of mostly
static content.

- slog

Allow to use additional disks for ZFS Intent Log to speed up
operations like fsync(2).

- vfs.zfs.super_owner

Allows regular users to perform privileged operations on files stored
on ZFS file systems owned by him. Very careful with this one.

- chflags(2)

Not all the flags are supported. This still needs work.

- ZFSBoot

Support to boot off of ZFS pool. Not finished, AFAIK.

Submitted by: dfr

- Snapshot properties

- New failure modes

Before if write requested failed, system paniced. Now one
can select from one of three failure modes:
- panic - panic on write error
- wait - wait for disk to reappear
- continue - serve read requests if possible, block write requests

- Refquota, refreservation properties

Just quota and reservation properties, but don't count space consumed
by children file systems, clones and snapshots.

- Sparse volumes

ZVOLs that don't reserve space in the pool.

- External attributes

Compatible with extattr(2).

- NFSv4-ACLs

Not sure about the status, might not be complete yet.

Submitted by: trasz

- Creation-time properties

- Regression tests for zpool(8) command.

Obtained from: OpenSolaris

Enable GCC stack protection (aka Propolice) for userland:
- It is opt-out for now so as to give it maximum testing, but it may be
turned opt-in for stable branches depending on the consensus. You
can turn it off with WITHOUT_SSP.
- WITHOUT_SSP was previously used to disable the build of GNU libssp.
It is harmless to steal the knob as SSP symbols have been provided
by libc for a long time, GNU libssp should not have been much used.
- SSP is disabled in a few corners such as system bootstrap programs
(sys/boot), process bootstrap code (rtld, csu) and SSP symbols themselves.
- It should be safe to use -fstack-protector-all to build world, however
libc will be automatically downgraded to -fstack-protector because it
breaks rtld otherwise.
- This option is unavailable on ia64.

Enable GCC stack protection (aka Propolice) for kernel:
- It is opt-out for now so as to give it maximum testing.
- Do not compile your kernel with -fstack-protector-all, it won't work.

Submitted by: Jeremie Le Hen <jeremie@le-hen.org>

MFp4: add FireWire/dcons support in loader for i386/amd64.

It is disabled by default. You need to put
LOADER_FIREWIRE_SUPPORT=yes in /etc/make.conf
and rebuild loader to enable it.
(cd /sys/boot/i386 && make clean && make && make install)

You can find a short introduction of dcons at
http://wiki.freebsd.org/DebugWithDcons

Don't unconditionally compile-in the bcache code. It's only used on
i386/amd64 and pc98. Remove useless calls to bcache_init() from the
ia64 and sparc64 loaders, as well as from the OFW common code.

Reimplementation of world/kernel build options. For details, see:

http://lists.freebsd.org/pipermail/freebsd-current/2006-March/061725.html

The src.conf(5) manpage is to follow in a few days.

Brought to you by: imp, jhb, kris, phk, ru (all bugs are mine)

Add back some bits.

Temperarly disable building in the bzip2 support by default so we can fit
on the i386 floppies. Sigh, I hate floppies.

Put bzip2 support on equal footing with gzip support.
Enable bzip2 support by default, set LOADER_NO_BZIP2_SUPPORT to disable it.

Pointy hat to: sobomax

NOFORTH -> NO_FORTH

Pick up loader.rc from its old home.

Disable the beastie menu. It offends some and annoys everyone else, and I'm
frankly tired of the controversy. When people ask me why FreeBSD isn't user-
friendly, I'll tell them that I tried. RIP.

For some reason crt0.o needs to be linked first for pxeboot(8) to
work. This is odd because loader(8) doesn't suffer from this problem.
Perhaps pxeboot bootstrap can be fixed to handle this better.
Anyway, PXE booting should work again.

Clean CLEANFILES.

kzip(8) is long dead.

- Factor out -nostdlib to an upper level Makefile.inc.

- Now that bsd.prog.mk deals with programs linked with -nostdlib
better, and has a notion of an "internal" program, use PROG
where possible. This has a good impact on the contents of
.depend files and causes programs to be linked with cc(1).

XXX: boot2 couldn't be converted as it's actually two programs.

Tested on: i386, amd64

Argh, unbreak "make depend" for AMD64.

Reported by: kris

- Removed -elf which is not a valid gcc(1) option anymore.
- Removed -ffreestanding; it's enforced by ../Makefile.inc.

Untangle building of AMD64 boot code.

Tested on: amd64 (sledge)

First round of cleanups to sys/boot/ makefiles:

- do not use PROG for what's not a real C program,
- use sys.mk transformation rules where possible,
- only create the "machine" symlink on AMD64,
- removed MAINTAINER lines in individual makefiles,
- added the LIBSTAND defitinion to <bsd.libnames.mk>,
- somewhat better contents in .depend files.

Tested on: i386, amd64
Prodded by: bde

Inherit BINDIR from a parent Makefile.inc.

Only include ../Makefile.inc once in loader/Makefile.

Build on amd64. Yes, I know this isn't particularly nice.

Enable the new bootloader for i386 only. The new loader.rc is will only
be installed if an old one does not exist, i.e. only during install, not
during upgrades.

Approved by: re

Add a new bootloader menu. Pull in screen.4th and frames.4th from the
examples directory to support it. This is installed only on i386 for
now. It will be enabled in a later commit.

Approved by: re

Simplify the Makefile by just using our standard PROG variable.

Revert previous untested revision. The i386 loader consists of three parts:
At the front is btxldr, in the middle is BTX itself (our mini-kernel), and
then the 'client' (loader.bin) which is the actual loader itself. boot2
just executes a raw ELF or a.out binary with the only setup provided being
that a bootinfo structure is passed on the stack. Now, since loader.bin
is a BTX client, the loader needs to be able to locate a BTX kernel for
the client to execute in the context. Thus, just like pxelder, btxldr
uses the a.out header on the loader binary to find the BTX kernel stored
in the loader and set it up. It does _not_ just reuse the BTX kernel
that boot2 invoked it with. This is because it can't assume that it will
_have_ a "spare" BTX kernel lying around. For example, when cdboot
loads the loader there isn't an existing BTX kernel. In fact, cdboot
will only work with an a.out loader as well since it also "borrows" the
BTX kernel in the loader binary (which it finds by parsing the a.out
header) just as pxeldr does. The only difference between cdboot and
pxeldr is where they get /boot/loader from.

If we wanted to make /boot/loader be an actual ELF binary we would need
to change the following utilites to handle that (and they all have to be
able to handle locating the BTX kernel inside of an ELF binary somehow):
- btxldr
- pxeldr
- cdboot

If we didn't want to require a flag day but make the transition smooth
then we need to be able to support both a.out and ELF versions of
/boot/loader which isn't exactly trivial since all three of these utilities
are written in assembly.

Pointy-hat to: peter

Try #2 at having /boot/loader default to ELF. Have pxeldr build its
own a.out version of loader.bin rather than depend on ../loader/loader
being a.out.

Initiate deorbit burn of i386 a.out kld "support" in loader. Note that
this was quite broken, it never was updated for metadata support.
The a.out kld file support was never really used, as it wasn't necessary.
You could always load elf kld's, even in an a.out kernel.

Revert the part of Kirks UFS2 commit which added divdi3.c and moddi3.c
to libi386, this issue was resolved already in a cleaner way.

This commit adds basic support for the UFS2 filesystem. The UFS2
filesystem expands the inode to 256 bytes to make space for 64-bit
block pointers. It also adds a file-creation time field, an ability
to use jumbo blocks per inode to allow extent like pointer density,
and space for extended attributes (up to twice the filesystem block
size worth of attributes, e.g., on a 16K filesystem, there is space
for 32K of attributes). UFS2 fully supports and runs existing UFS1
filesystems. New filesystems built using newfs can be built in either
UFS1 or UFS2 format using the -O option. In this commit UFS1 is
the default format, so if you want to build UFS2 format filesystems,
you must specify -O 2. This default will be changed to UFS2 when
UFS2 proves itself to be stable. In this commit the boot code for
reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c)
as there is insufficient space in the boot block. Once the size of the
boot block is increased, this code can be defined.

Things to note: the definition of SBSIZE has changed to SBLOCKSIZE.
The header file <ufs/ufs/dinode.h> must be included before
<ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and
ufs_lbn_t.

Still TODO:
Verify that the first level bootstraps work for all the architectures.
Convert the utility ffsinfo to understand UFS2 and test growfs.
Add support for the extended attribute storage. Update soft updates
to ensure integrity of extended attribute storage. Switch the
current extended attribute interfaces to use the extended attribute
storage. Add the extent like functionality (framework is there,
but is currently never used).

Sponsored by: DARPA & NAI Labs.
Reviewed by: Poul-Henning Kamp <phk@freebsd.org>

-ffreestanding is the word for /sys.

Install files via FILES, there's no reason to compare them before installing.

Needs a.out support built into the loader.

Add support for loading bzip2-compressed kernels and modules. This support
is turned off by default and could be enabled by defining LOADER_BZIP2_SUPPORT
make variable. Also make gzip support optional (turned on by default) -
it could be turned off via LOADER_NO_GZIP_SUPPORT make variable.

Please note, that due to limit on the amount of memory available to the
loader(8), it is possible to load modules/kernels compressed with the smallest
block size supported by the bzip2 - 100k (`-1' bzip2(1) option), however
even in this mode bzip2(1) usually provides better compression ratio than
gzip(1) in its best compression mode.

MFC after: 1 month

Create backup copies using install(1).

OK, now I am scared of pxeldr. It had /boot/loader appended onto
the end of it and decoded the headers.

Submitted by: jhb

Generate an ELF /boot/loader instead of fake a.out. The fake a.out wrapper
did not work with old a.out-only bootblocks anyway. :-(

Remove -DNEW_LINKERSET, it is not used here anymore. This is now native.

Strip the .comment and .note sections when stripping. There's no
point in retaining this info, particularly under BTX.

MF4: Add LOADER_TFTP_SUPPORT make.conf flag.

Look for libstand in the built tree rather than in /usr/lib. This
likely could be done better, but the tree is broken and I wanted to
get a fix into the tree.

Reviewed by: msmith

Move man page directives to common/Makefile.inc.

Now that we are compiling PXE into libi386, we don't need ../libi386 in
.PATH anymore.

Move the building of the PXE module into libi386.

Make PXE use the UDP API. This allows for both TFTP and NFS support.
You may specify TFTP or NFS via compile time options in the loader,
but not both at this time.

Also, remove a warning about not knowing how to boot from network
devices. We can obviously do that now.

Mega i386 loader commit.
- Don't hard code 0x10000 as the entry point for the loader. Instead add
src/sys/boot/i386/Makefile.inc which defines a make variable with the
entry point for the loader. Move the loader's entry point up to
0x20000, which makes PXE happy.
- Don't try to use cpp to parse btxldr for the optional BTXLDR_VERBOSE,
instead use m4 to achieve this. Also, add a BTXLDR_VERBOSE knob in the
btxldr Makefile to turn this option on.
- Redo parts of cdldr's Makefile so that it now builds and installs cdboot
instead of having i386/loader/Makefile do that. Also, add in some more
variables to make the pxeldr Makefile almost identical and thus to ease
maintainability.
- Teach cdldr about the a.out format. Cdldr now parsers the a.out header
of the loader binary and relocates it based on that. The entry point of
the loader no longer has to be hardcoded into cdldr. Also, the boot
info table from mkisofs is no longer required to get a useful cdboot.
- Update the lsdev function for BIOS disks to parse other file systems
(such as DOS FAT) that we currently support. This is still buggy as
it assumes that a floppy with a DOS boot sector actually has a MBR and
parses it as such. I'll be fixing this in the future.
- The biggie: Add in support for booting off of PXE-enabled network
adapters. Currently, we use the TFTP API provided by the PXE BIOS.
Eventually we will switch to using the low-level NIC driver thus
allowing both TFTP and NFS to be used, but for now it's just TFTP.

Submitted by: ps, alfred
Testing by: Benno Rice <benno@netizen.com.au>

Add a NOFORTH variable so we can build a smaller loader without
Forth support, for use with PicoBSD

Approved-By: jordan

Fix bogon in previous commit. Re-enable Forth in the loader.

Noticed by: dcs
Approved by: jkh

Allow for cdldr to be built in ${.OBJDIR}/../cdldr/cdldr instead of
only looking for it in ${.CURDIR}/../cdldr/cdldr. This fixes world.

Add the new cdldr CD bootstrap loader. This patch includes the following:

- Fix btxldr to preserve a NULL bootinfo pointer when it copies the kernel
arguments.
- Add the cdldr bootstrap program. This program is tacked onto the
beginning of the standard 3rd stage boot loader (/boot/loader) to form
the CD boot loader (/boot/cdboot). When a CD is booted, the cdboot file
is copied into memory instead and executed. The cdldr stub emulates the
environment normally provided by boot2 and then starts the loader. This
booting method does not emulate a floppy drive, but boots directly off of
the CD. This should fix the problems some BIOS's have with emulating a
2.88 MB floppy image.
- Add support to the loader to recognize that it has been booted by cdldr
instead of boot2 and use a simpler method of extracting the BIOS boot
device.

Add machine-specific include path to ficl's sysdep.h. Wishes I had
gone to bed earlier.

Pointed by: peter

$Id$ -> $FreeBSD$

Reconcile all the loader newvers.sh's into one common one.

Move loader.conf.5, while it is still in it's first revision.

Add loader.4th man page (worst man page I ever wrote -- reviews
are welcomed). Correct a nasty bug in loader.4th before anyone
notices, add a couple of features.

Installs a default loader.rc if none exists. Document loader.conf.

The man page, version 0.1.

Install new loader.rc stuff.

Don't try protecting page 0; it seems that BIOS writers now feel it's
appropriate to check for Windows95 by polling for state in low
memory, with fatal results.

Submitted by: rnordier@freebsd.org

Transition from using Perl to using awk for our text-manipulation
needs. This removes the dependancy on Perl for the generation of the
loader, allowing the world to be built on a perl-free system.

Submitted by: Joe Abley <jabley@clear.co.nz>

Preserve the previous loader as loader.old.

Note no matching commit for the Alpha, as the alpha boot0 stage does
not have the ability to prompt for user input.

PR: kern/9406
Submitted by: "Daniel C. Sobral" <dcs@newsguy.com>

Disable kzipping the loader; it seems to be breaking the aout-to-elf build

Explicitly look for kzip in /usr/bin, again this is a stopgap, to avoid
making kzip a build tool.

kzip the loader, this shrinks it to about 60% of its original size.
Note that this is a stopgap with dependancies on the a.out tools.

Restore dependancy to build loader.help here

Reenable generation of the loader.help file

Disable build/install of boot.help until we sort out how to construct it
in the makeworld environment.

Build the composite help file from the generic and architecture-specific
help files, sorting topics and subtopics to allow maximum flexibility.

bootforth seems to be working well enough for me and others to turn
on by default. If it causes trouble, it's very easy to switch off again.

Turn off -g accidentally left in from testing.

Respect ${.OBJDIR} properly when looking for libficl.a

Add required parts for BootForth building (currently disabled and
untested). Only suitable for i386 at the moment, as we are missing
setjmp/longjmp on the Alpha.

Clean the ${BASE}.sym file as well.

Add a dependancy on the BTX crt0 object
Add the biospnp handler to the pnp handler array
Drop some old debugging code

- Enable PnP and ISA PnP code.
- Use the ISA PnP enumerator.
- Use the new linker set code, throw out the gensetdefs stuff.
- Produce an intermediate loader image that has symbols stripped, to aid
- in debugging.
- Supply ISA port access functions required for ISA PnP

Try and get the sys/* and machine/* includes via relative paths. This
saves having to do a 'make includes' after touching any header file for
the boot code.

Use rather than for linking. This allows the new rather
than stale version of libstand.a to be found.

After this change, the new boot code is apparently building correctly
in a make world.

Enable the DOS filesystem. NOTE: you will have to rebuild libstand in
order to be able to build again.

Override STRIP so installation doesn't try to strip the loader.

Make 'make install' do something that might be useful.

Recover the arguments passed in from the previous load stage, use them to set
the default console.
Print malloc stats from the new libstand allocator.

Make these compile when there is an obj dir.

Make libstand movable.

This completes the basic work required to make the loader work with BTX. We
now have a third-stage bootstrap shared by the i386 and Alpha.

Enable the biosdisk driver, duplicate -lstand as it both calls and is called
by the i386 platform library.

Initial integration of the i386 bootloader and BTX.

- Discard large amounts of BIOS-related code in favour of the more compact
BTX vm86 interface.
- Build the loader module as ELF, although the resulting object is a.out,
make gensetdefs 32/64-bit sensitive and use a single copy of it.
- Throw away installboot, as it's no longer required.
- Use direct bcopy operations in the i386_copy module, as BTX
maps the first 16M of memory. Check operations against the
detected size of actual memory.

Resynch with working sources before BTX integration.

- Use format-independant module allocator.
- Conditionalise ISA PnP support.
- Simplify PnP enumerator interface.
- Improve module/object searching.
- Add missing depend/install targets in BTX makefiles.
- Pass the kernel environment and module data in extended bootinfo fields.
- Add a pointer to the end of the kernel + modules in bootinfo.
- Fix parsing of old-style kernel arguments.

This commit was generated by cvs2svn to compensate for changes in r38465,
which included commits to RCS files with non-trunk default branches.

This is the new unified bootstrap, sometimes known previously as the
'three-stage' bootstrap.
There are a number of caveats with the code in its current state:
- The i386 bootstrap only supports booting from a floppy.
- The kernel and kld do not yet know how to deal with the extended
information and module summary passed in.
- PnP-based autodetection and demand loading of modules is not implemented.
- i386 ELF kernel loading is not ready yet.
- The i386 bootstrap is loaded via an ugly blockmap.

On the alpha, both net- and disk-booting (SRM console machines only) is
supported. No blockmaps are used by this code.

Obtained from: Parts from the NetBSD/i386 standalone bootstrap.