MFC r354245, r354833, r354837: add flua to the base system

r354245: stand: consolidate knowledge of lua path

Multiple places coordinate to 'know' where lua scripts are installed. Knock
this down to being formally defined (and overridable) in exactly one spot,
defs.mk, and spread the knowledge to loaders and liblua alike. A future
commit will expose this to lua as loader.lua_path, so it can build absolute
paths to lua scripts as needed.

r354833: Add flua to the base system, install to /usr/libexec

FreeBSDlua ("flua") is a FreeBSD-private lua, flavored with whatever
extensions we need for base system operations. We currently support a subset
of lfs and lposix that are used in the rewrite of makesyscall.sh into lua,
added in r354786.

flua is intentionally written such that one can install standard lua and
some set of lua modules from ports and achieve the same effect.

linit_flua is a copy of linit.c from contrib/lua with lfs and lposix added
in. This is similar to what we do in stand/. linit.c has been renamed to
make it clear that this has flua-specific bits.

luaconf has been slightly obfuscated to make extensions more difficult. Part
of the problem is that flua is already hard enough to use as a bootstrap
tool because it's not in PATH- attempting to do extension loading would
require a special bootstrap version of flua with paths changed to protect
the innocent.

src.lua.mk has been added to make it easy for in-tree stuff to find flua,
whether it's bootstrap-flua or relying on PATH frobbing by Makefile.inc1.

r354837: flua: newer GCC complains about format-nonliteral at WARNS=2

Disable that one, too.

MFC r341101, r341231, r341276, r341329, r341433, r341780, r342054-r342055,
r342721, r342742, r342840, r343008, r343225

r341101:
powerpcspe: Don't crash the loader on ubldr with SPE instructions.

-msoft-float seems to be insufficient for disabling the SPE on powerpcspe.
Force it off with -mno-spe as well. This prevents a crash in ubldr on
powerpcspe.

r341231:
loader: command_bcache() should print unsigned values

All bcache counters are unsigned.

r341276:
When handling CMD_CRIT error set command_errmsg to NULL after we dump it out,
so that it does not result in error message printed twice.

OK load doodoo
can't find 'doodoo'
can't find 'doodoo'
OK

r341329:
loader.efi: fix EFI getchar() for multiple consoles

This fix is ported from illumos (issue #9970), the analysis and initial
implementation was done by John Levon.

See also: https://www.illumos.org/issues/9970

Currently, efi_cons_getchar() will wait for a key. While this seems to make
sense, the implementation of getchar() in common/console.c will loop across
getchar() for all consoles without doing ischar() first.

This means that if we've configured multiple consoles, we can't input into
the serial, as getchar() will be sat waiting for input only from efi_console.c

This patch does implement a bit more generic key buffer to support
translation of input keys, and we use generic efi_readkey() to reduce
duplication from calls from getchar() and poll().

r341433:
Move inclusion of src.opts.mk later.

src.opts.mk includes bsd.own.mk. This in turn defines CTFCONVERT_CMD
depending on the MK_CTF value. We then set MK_CTF to no, which has no
real effect. The solution is to set all the MK_foo values before
including src.opts.mk.

This should stop the cdboot binary from exploding in size for releases
built WITH_CTF=yes in src.conf.

r341780:
powerpc/ubldr: Teach powerpc's ubldr to boot 64-bit kernels

This is just a copy of powerpc/ofw's ppc64_elf_freebsd.c modified to fit
ubldr's boot format.

r342054:
Print an error message in efi_main.c if we can't allocate memory for the heap

With the default Qemu parameters, only 128MB RAM gets given to a VM. This causes
the loader to be unable to allocate the 64MB it needs for the heap. This change
makes the cause of the error more obvious.

r342055:
Cast error message in efi_main.c to CHAR16* to avoid build error

r342721:
loader.efi: update memmap command to recognize new attributes

Also move memory type to string translation to libefi for later use.

r342742:
loader.efi: efi variable rework and lsefi command added

This update does add diag and debug capabilities to interpret the efi
variables, configuration and protocols (lsefi).

The side effect is that we add/update bunch of related headers.

r342840:
Create MK_LOADER_VERBOSE and connect it to ELF_VERBOSE in the loader
code.

r343008:
Add Dell Chromebook to the list of devices with E820 extmem quirk enabled

Just like for Acer C270 chromebook the E820 extmem workaround is required for
FreeBSD to boot on Dell chromebook.

r343225:
Unbreak mip64 build after r328437

Add exit and getchar functions to beri/boot2 code. They are required by
panic_action functin introduced in r328437

PR: 18498, 204916

MFC Loader Interpreter Coexistance: r337806, r337868, r337914, r337927,
r337989, r338052, r338064, r338407, r338418, r338474

r337806:
Create a loader for each interpreter for x86 BIOS and all EFI

Create loader_{4th,lua,simp}{,.efi}. All of these are installed by
default. Create LOADER_DEFAULT_INTERP to specify the default
interpreter when no other is specified. LOADER_INTERP is the current
interpreter language building. Turn building of lua on by default to
match 4th. simploader is a simplified loader build w/o any interpreter
language (but with a simple loader). This is the historic behavir you
got with WITHOUT_FORTH. Make a hard link to the default loader. This
has to be a hard link rather than the more desirable soft link because
older zfsboot blocks don't support symlinks.

r337868:
stand: Use -Oz/-Os for all loader/stand builds.

While we're not super size constrained, the x86 BIOS /boot/loader has
to be less than about 520k-530k to be reliable. The LUA loader is at
this size today. -Oz saves 15-20% on the size, keeping us safely small
enough (comparable to where we were with the 4th loader). This will
also help with sjg's work on bringing in bearssl, though we may again
be looking for space in the LUA loader.

Size table for clang 6.0.0:
default -O1 -Os -Oz
4th 442368 417792 389120 376832
lua 524288 479232 446464 430080

r337914:
Install links for loader.efi.

r337927:
Add ashldi3 and ashrdi3 to mips.

Now that we're using -Os, mips needs these routines.

r337989:
Copy the boot loader from the new location for the co-existing
loaders.

r338052:
libsa: Add lshrdi3.c for powerpc* and mips

r338064:
Turn back the clock just a little: make userboot.so always be 4th

Turns out there was a hidden dependency we hasn't counted upon. The
host load /boot/userboot.so to boot the VMs it runs. This means that
the change to lua meant suddently that nobody could run their older
VMs because LUA wasn't in 10.0, last month's HardenedBSD, 11.2 or
whatever. Even more than for the /boot/loader* binaries, we need a
good coexistance strategy for this. While that's being designed and
implemented, drop back to always 4th for userboot.so. This will fail
safe in all but the most extreme environments (but lua-only hacks
to .lua files won't be processes in VMs until we fix it).

r338407:
lualoader: Print error messages from command failures at the prompt

Previously lualoader would remain silent, rather than printing
command_errmsg or noting that a command had failed or was not found.

r338418:
userboot: handle guest interpreter mismatches more intelligently

The switch to lualoader creates a problem with userboot: the host is
inclined to build userboot with Lua, but the host userboot's interpreter
must match what's available on the guest. For almost all FreeBSD guests in
the wild, Lua is not yet available and a Lua-based userboot will fail.

This revision updates userboot protocol to version 5, which adds a
swap_interpreter callback to request a different interpreter, and tries to
determine the proper interpreter to be used based on how the guest
/boot/loader is compiled. This is still a bit of a guess, but it's likely
the best possible guess we can make in order to get it right. The
interpreter is now embedded in the resulting executable, so we can open
/boot/loader on the guest and hunt that down to derive the interpreter it
was built with.

Using -l with bhyveload will not allow an intepreter swap, even if the
loader specified happens to be a userboot with the wrong interpreter. We'll
simply complain about the mismatch and bail out.

For legacy guests without the interpreter marker, we assume they're 4th.
For new guests with the interpreter marker, we'll read it and swap over
to the proper interpreter if it doesn't match what the userboot we're using
was compiled with.

Both flavors of userboot are installed by default, userboot_4th.so and
userboot_lua.so. This fixes the build WITHOUT_FORTH as a coincidence, which
was broken by userboot being forced to 4th.

r338474:
Be a little conservative about when to force size optimizations.

Reports have come in that there's issue with powerpc and sparc64 since
we've switched to using -Oz / -Os. We don't strictly need them for
!x86, so be conservative about when we enable them.

MFC GELI Loader Improvements: r336252, r336254, r336256, r336354,
r336532-r336534, r336537, r336626, r337326, r337349, r341071, r341160,
r341420, r341473, r341651, r342793

Note that this MFC contains some seemingly unrelated zfsloader bits -- this
was needed in order to pull in some later fixes for GELI hand-off w/ ZFS
bits included.

r336252:
Extend loader(8) geli support to all architectures and all disk-like devices.

This moves the bulk of the geli support from lib386/biosdisk.c into a new
geli/gelidev.c which implements a devsw-type device whose dv_strategy()
function handles geli decryption. Support for all arches comes from moving
the taste-and-attach code to the devopen() function in libsa.

After opening any DEVT_DISK device, devopen() calls the new function
geli_probe_and_attach(), which will "attach" the geli code to the open_file
struct by creating a geli_devdesc instance to replace the disk_devdesc
instance in the open_file. That routes all IO for the device through the
geli code.

A new public geli_add_key() function is added, to allow arch/vendor-specific
code to add keys obtained from custom hardware or other sources.

With these changes, geli support will be compiled into all variations of
loader(8) on all arches because the default is WITH_LOADER_GELI.

r336254:
Use if rather than case for a simple boolean. gcc thinks blks is
undefined sometimes with the case, but enc is always 0 or 1, so
and if / else is better anyway.

r336256:
Fix glitched indentation (and rewrap as needed due to deeper indent).
No functional changes.

r336354:
zfsboot: fix build with WITHOUT_LOADER_GELI

r336532:
Collapse zfsloader functionality back down into loader.

We no longer really need a separate zfsloader. It was useful when we
were first supporting ZFS and had limited ability to properly boot off
of ZFS without the special boot loader. Now that the boot loader has
matured, go the way loader.efi pioneered and just build one
binary. Change the name of the loader to load in the secondary boot
blocks to be just /boot/loader. Provide a symbolic link from zfsloader
to loader so people who have not upgraded their boot blocks are not
affected. This has the happy benefit of making coexistence easier as
well (fewer binaries in the matrix).

r336533:
Eliminate zfsloader man page.

Remove all cross references to zfsloader.8 and /boot/zfsloader.
Move ZFS specific info into loader.8.

r336534:
NM and OBJCOPY are already defined for all builds. There's no need to
conditionally define them here.

r336537:
Mention zfsloader being folded into loader in UPDATING.

r336626:
Older zfs boot blocks don't support symlinks. install the link to
zfsloader as a hard link. While newer ones do, the whole point of the
link was to transition to the new world order smoothly. A hard link is
less flexible, but it works and will result in fewer bumps. Adjust
UPDATING entry to match.

r337326:
loader: biosdisk.c has leftover geli header.

A small cleanup, remove unneeded #include.

r337349:
zfsboot: Fix startup crash

On a FreeNAS mini XL, with geli encrypted drives the loader crashed in
geli_read().

When we iterate over the list of disks and allocate the zfsdsk structures we
don’t zero out the gdev pointer. In one case that resulted in geli_read()
(called on the bogus pointer) dividing by zero.

Use calloc() to ensure the zfsdsk structure is always zeroed, so the pointer is
initialised to NULL. As a side benefit it gets rid of one #ifdef
LOADER_GELI_SUPPORT.

r341071:
Restore the ability to override the disk unit/partition at the boot: prompt
in gptboot.

When arch-independent geli support was added, a new static 'gdsk' struct
was added, but there was still a static 'dsk' struct, and when you typed
in an alternate disk/partition, the string was parsed into that struct,
which was then never used for anything. Now the string gets parsed into
gdsk.dsk, the struct that's actually used.

r341160:
Add comments describing the bootargs handoff between loader(8) and gptboot
or zfsboot, when loader(8) is the BTX loader. No functional changes.

r341420:
Eliminate duplicated code and struct member definitions in the handoff
of args data between gptboot/zfsboot and loader(8).

Despite what seems like a lot of changes here, there are no actual
changes in behavior, or in the data layout in the structures involved.
This is just eliminating identical code pasted into multiple locations.

In detail, the changes are...

- Move struct zfs_boot_args definition from libsa/zfs/libzfs.h to
i386/common/bootargs.h because it is specific to x86 booting and the
handoff between zfsboot and loader, and has no relation to the zfs
library code in general.

- The geli_boot_args and zfs_boot_args structs both contain an identical
set of member variables containing geli information. Extract this out
to a new geli_boot_data struct, and embed it in the arg-passing structs.

- Provide new routines geli_import_boot_data() and geli_export_boot_data()
that can be shared between gptboot, zfsboot, and loader instead of
pasting identical code into several different .c files.

- Remove some checks for a NULL pointer that can never be true because the
pointer being tested was set using pointer math (kargs + 1) and that can
never result in NULL in this code.

r341473:
Fix args cross-threading between gptboot(8) and loader(8) with zfs support.

When loader(8) is built with zfs support enabled, it assumes that any extarg
data present is a zfs_boot_args struct, but if the first-stage loader was
gptboot(8) the extarg data is actually a geli_boot_args struct. Luckily,
zfsboot(8) and gptzfsboot(8) have always passed KARGS_FLAGS_ZFS along with
KARGS_FLAGS_EXTARG, so we can use KARGS_FLAGS_ZFS to decide whether the
extarg data is a zfs_boot_args struct.

To avoid similar problems in the future, gptboot(8) now passes a new
KARGS_FLAGS_GELI to indicate that extarg data is geli_boot_args. In
loader(8), if the neither KARGS_FLAGS_ZFS nor KARGS_FLAGS_GELI is set but
extarg data is present (which will be the case for gptboot compiled before
this change), we now check for the known size of the geli_boot_args struct
passed by the older versions of gptboot as a way of confirming what type of
extarg data is present.

In a semi-related tidying up, since loader's main() has already decided
what type of extarg data is present and set the global 'zargs' var
accordingly, don't repeat the check in extract_currdev, just check whether
zargs is NULL or not.

r341651:
Don't reference zfs-specific variables if LOADER_ZFS_SUPPORT is undefined
because the variables will be undefined too.

r342793:
MK_ZFS -> {MK_ZFS|MK_LOADER_ZFS}, this is so we can diable userland / kernel
ZFS but keep the boot-loaders when using ZoL port.

Relnotes: yes (GELI support extended)
Relnotes: yes (zfsloader has been collapsed into loader and may be
removed after boot blocks have been updated)

MFC r336084-r336085, r336118: Move zfs impl into libsa

r336084:
Move ZFS files into libsa

Move the libzfs stuff into libsa. There's no need for it to be a
separate library. The separate library adds to the issues of build
ordering that we see from time to time. Move the filesystem support
into libsa, like all the other filesystem support rather than making
zfs the odd-duck out.

Discussed with: allanjude@

r336085:
Stop using ../zfs/libzfs.h but instead use libzfs.h.

While ../zfs/libzfs.h mostly works, there are a few situations where
it does not. Eliminate the problem by using plain libzfs.h, like we do
for ufs support. This fixes the weird cases, and is easier to
understand. It also follows the general style convetion of avoiding
../ in #includes.

r336118:
Remove dirs that git svn should have removed but didn't.

MFC r335069-r335071, r335110, r335116: stand housecleaning

r335069:
Remove fail: label. It's unused.

r335070:
Remove unused variables.

[kevans@: removed ufs.c modifications; those depend on ufs rewrite bits]

r335071:
Initialize variables we later free so they aren't used
uninitialized in the error path.
Remove unused variables.

r335110:
Remove redundant defs.mk includes. They aren't needed.

r335116:
Use bool for vargood, since it's a boolean.

MFC lualoader: r326353, r328440, r328443, r329166-r329167, r329274, r329329,
r329349-r329352, r329355-r329359, r329366-r329369, r329386-r329387, r329393,
r329413-r329415, r329417, r329424-r329436, r329457, r329473-r329474,
r329496-r329501, r329503-r329504, r329543, r329547-r329551, r329576-r329578,
r329580, r329583, r329585-r329590, r329592-r329596, r329609-r329611,
r329614, r329619, r329621-r329622, r329624, r329626-r329627,
r329629-r329632, r329640-r329641, r329643-r329646, r329649-r329650, r329654,
r329656, r329662, r329669-r329671, r329673-r329674, r329680,
r329684-r329689, r329692-r329693, r329696-r329700, r329709, r329716,
r329731, r329733-r329734, r329747-r329748, r329756, r329779, r329782,
r329784, r329786, r329804, r329806, r329809, r329811, r329836,
r329850-r329852, r329854, r329856-r329858, r329861, r329895,
r329897-r329899, r329901-r329903, r329922-r329924, r329927-r329928,
r329944-r329949, r329986-r329987, r330008-r330010, r330012, r330020,
r330082-r330084, r330087-r330088, r330098-r330101, r330138-r330139,
r330261-r330263, r330267-r330269, r330281-r330284, r330287, r330339-r330342,
r330345-r330346, r330369-r330370, r330434-r330435, r330564, r330616-r330618,
r330620, r330625, r330690, r330701, r330703, r330825, r331211, r331257,
r331259, r331281-r331282, r331304, r331314, r331476-r331477,
r331563-r331564, r331854-r331857, r331859-r331860, r332106, r334723,
r334879, r334891, r334907, r334912, r334939, r334986, r335009, r335371,
r336759, r337711, r337807-r337810, r338054, r338063, r338065-r338067,
r338083, r338085-r338086, r338108, r338167-r338168, r338173, r338203,
r338255, r338259, r338309, r338394, r338438, r338886, r338893, r339173,
r339200, r339218, r339222, r339301, r339307, r339677-r339678, r339702,
r339805, r339831, r339849, r340040, r340152

Some notes for this MFC:
- This is still pre-forth/lua coexistance; that will come shortly-ish so
that forth/lua may be installed together (with forth remaining the default)

- module_blacklist support for lualoader has been MFC'd, but the drm modules
are not blacklisted in this stable branch.

r326353: Import lua 5.3.4 to contrib
r328440: Preserve the original luaconf.h in a convenient place. Clients will
r328443: Gross hack to omit printing hex floating point when the lua number
r329166: Add Lua as a scripting langauge to /boot/loader
r329167: Add the lua scripts from the lua-bootloader SoC
r329274: stand/lua: Exit sub-menus on backspace
r329329: stand/lua: Always boot on 'enter' keypress in menus
r329349: stand/lua: Reduce magic numbers
r329350: stand/lua: Don't descend into an empty kernels submenu
r329351: stand/lua: Set reasonable ACPI default based on presence
r329352: stand/lua: Consistently use semicolons for line endings
r329355: stand/lua: Move kernel selection into main menu
r329356: stand/lua: Allow menu items to not have explicit aliases
r329357: stand/lua: Remove explicit alias from "Back to main menu"
r329358: stand/lua: Say "loader prompt" instead of "lua interpreter"
r329359: stand/lua: Remove a magic number/string (not a trivial literal)
r329366: stand/lua: Set ACPI's default the proper way (setACPI)
r329367: stand/lua: Create a "carousel" menu entry type
r329368: stand/lua: Create/use some MENU_ constants where applicable
r329369: stand/lua: Remove sneaky kernel assignment
r329386: stand/lua: Correct usage and acceptance of BACKSPACE/DELETE keys
r329387: stand/lua: Chop off the decimal for numbers passed to setcursor
r329393: stand/lua: Use escaped dot instead of single character class
r329413: stand/lua: Allow MENU_RETURN items to have func, fix esc. to prompt
r329414: stand/lua: Don't try to divide by 0; do nothing
r329415: stand/lua: Don't reload kernel config if we only have one kernel
r329417: stand/lua: Make CAROUSEL_ENTRY func parameters consistent with name
r329424: stand/lua: Don't set autoboot_delay=NO in menu autoboot sequence
r329425: stand/lua: Enable menu autoboot; it seems to work
r329426: stand/lua: Correct interpretation of autoboot_delay
r329427: stand/lua: Color non-default kernels blue
r329428: stand/lua: Correct some trivial errors in config
r329429: stand/lua: Add debug method to dump modules
r329430: stand/lua: Address some nits
r329431: stand/lua: Correct test sense, this should have been 'not nil'
r329432: stand/lua: Try to load alternate kernels as directories first
r329433: stand/lua: Add optional GELI passphrase prompt
r329434: stand/lua: Check for nil (GELI prompt)
r329435: stand/lua: Style pass
r329436: stand/lua: Debugging string snuck in...
r329457: stand/lua: dumpModules => lsModules
r329473: liblua: Clean up io/loader C module registration
r329474: liblua: Emulate DIR, opendir, fdopendir, closedir
r329496: stand/lua: Fix verbiage and some typos
r329497: stand/lua: Fix module_path handling with multiple kernels
r329498: stand/lua: Remove some debugging bits that snuck in... gr...
r329499: interp_lua: Register io/loader with regular Lua module system
r329500: Lua loader: Add barebones "lfs" module
r329501: lua loader: Auto detect eligible list of kernels to boot
r329503: liblua: Fix missing '}' in lutil.c after r329499
r329504: stand/lua: More style nits, config.lua
r329543: Create style.lua(9)
r329547: stand/lua: Allow menu items to be conditionally (in)visible
r329548: stand/lua: Addres style.lua(9) concern
r329549: stand/lua: Clear the screen before prompting for passwords
r329550: stand/lua: Store the loaded kernel as config.kernel_loaded
r329551: stand/lua: reload previously loaded kernel at config-load/reload
r329576: stand/lua: Defer kernel/module loading until boot or menu escape
r329577: stand/lua: Rename bootserial for clarity
r329578: stand/lua: Menu style.lua(9) nits
r329580: stand/lua: Remove some unused local declarations
r329583: stand/lua: Store menu entries in an "entries" table
r329585: stand/lua: Add core.isSingleUserBoot
r329586: stand/lua: Call menu_entries if it's a function
r329587: stand/lua: Swap single-/multi- user boot entries as needed
r329588: stand/lua: Re-wrap menu.lua now that I've added indentation...
r329589: stand/lua: Track env changes that come in via loader.conf(5)
r329590: stand/lua: Restore environment upon config reload
r329592: stand/lua: Remove inaccurate comment after r329590
r329593: stand/lua: Change boot menu items' names when swapped
r329594: stand/lua: Round up some more style.lua(9) concerns
r329595: stand/lua: Re-order locals after copyright notice; require first
r329596: stand/lua: Add copyright notice in places
r329609: stand/lua: Cache swapped menu, and don't create locals for swapping
r329610: style.lua(9): Note that wrapping at 80-columns is not rigid
r329611: stand/lua: Wrap tuple assignment earlier for readability
r329614: stand/lua: Don't set ACPI off just because we can't detect it.
r329619: stand/lua: Extract menu handlers out into menu.handlers table
r329621: stand/lua: Add and use drawer.menu_name_handlers
r329622: stand/lua: Move drawer.menu_name_handlers further up
r329624: stand/lua: Reduce exposure of the drawer module
r329626: stand/lua: Refactor logos into drawer.logodefs table
r329627: stand/lua: Refactor brands into drawer.branddefs
r329629: stand/lua: Use 'graphic' instead of 'logo' for depicting graphics
r329630: stand/lua: Stick a copyright notice on drawer.lua
r329631: stand/lua: Insert helpful comment for drawer.branddefs
r329632: style.lua(9): Clarify local variable guideline
r329640: stand/lua: Consistently declare local functions at module scope
r329641: stand/lua: Consistently organize modules
r329643: Implement loader.command
r329644: lualoader: Add ability to intercept cli commands
r329645: lualoader: Move carousel storage out into config
r329646: lualoader: Eliminate global namespace pollution in loader.lua
r329649: Lua lfs.attributes: Provide a more consistent error return
r329650: liblua: Add loader.machine and loader.machine_arch properties
r329654: lualoader: Ignore ACPI bits on !i386
r329656: loader.lua: Expose errno table to lua
r329662: lualoader: Replace invalid construct with valid construct
r329669: lualoader: Prefer selected kernel to currently loaded
r329670: lualoader: Don't try so hard to load a kernel
r329671: lualoader: Prepare for interception of "boot" CLI cmd
r329673: lualoader: Intercept boot cli command
r329674: lualoader: Intercept the 'autoboot' cli command
r329680: lualoader: When restoring environment, only restore unchanged vars
r329684: lualoader: Drop terminating semicolons
r329685: lualoader: Drop excessive parenthesizing
r329686: style.lua(9): Drop notes about semicolons
r329687: lualoader: Drop explicit boolean tests; b or not b
r329688: lualoader: Don't return false for failure to open config on silent
r329689: lualoader: Output "Failed to parse" messages
r329692: lualoader: Bring in local.lua module if it exists
r329693: lualoader: Return only argstr if with_kernel not requested
r329696: lualoader: Add "menu.default", initialized to menu.welcome
r329697: lualoader: Drop name requirement for menu separators
r329698: lualoader: Directly reference submenu definition with submenu key
r329699: lualoader: Simplify menu definitions a little further
r329700: lualoader: Allow carousel 'items' to be a table as well
r329709: lualoader: Don't autodetect kernels if 'kernels' is explicitly set
r329716: lualoader: Use the key that interrupts autoboot as a menu choice
r329731: lualoader: Add boot environment support
r329733: lualoader: Make kernel autodetect. contingent on loader.conf(5) var
r329734: lualoader: Don't execute menu.autoboot() for every opened menu
r329747: lualoader: Replace 8-space indentation with a single tab
r329748: lualoader: Drop password length restrictions
r329756: lualoader: Remove nasty hack for not printing out ".0"
r329779: lualoader: Split cli bits out into a cli module
r329782: lualoader: Unbreak 'boot [kernel]' by including config
r329784: lualoader: Pull argument extraction for cli funcs to cli.arguments
r329786: lualoader: Attach cli command functions to cli module
r329804: lualoader: Eliminate some unused locals
r329806: lualoader: Consistently use double quotes
r329809: lualoader: Address some 'luacheck' concerns
r329811: lualoader: Clear up an empty conditional branch
r329836: lualoader: Attend to some 80-col issues, pointed out by luacheck
r329850: lualoader: Drop unused return values; we'll only use the first
r329851: Add SPDX tags to lua files
r329852: Add copyright notice to core.lua
r329854: lualoader: shallowCopyTable => deepCopyTable
r329856: lualoader: Use "local function x()" instead "local x = function()"
r329857: Centralize lua defines
r329858: When the LUA_FLOAT_TYPE != LUA_FLOAT_INT64, we can't ref. float
r329861: lualoader: Track effective line number, use it for drawing
r329895: liblua: Implement write support
r329897: lualoader: Add nextboot support
r329898: lualoader: Plug file handle not properly closed
r329899: lualoader: Correct test and name
r329901: lualoader: Add comment on trailing space, don't operate on nil
r329902: lualoader: Remove unused variable; we now use effective line number
r329903: lualoader: Explain nextboot stuff a little bit more
r329922: lualoader: Split config file I/O out into a separate function
r329923: lualoader: Strip config.parse of its I/O privileges
r329924: lualoader: throw out nextboot's usage of standard config processing
r329927: lualoader: Clean up naming conventions a little bit
r329928: lualoader: Remove inaccurate part of comment
r329944: lualoader: Don't explicitly index tables without reason
r329945: lualoader: menu: Terminate final values in tables with a comma
r329946: lualoader: Clean up menu handling a little bit
r329947: lualoader: Pull menu redrawing specifics out of menu.process
r329948: lualoader: Pull autoboot handling out into menu.run()
r329949: lualoader: Explain deviation from naming guidelines
r329986: lualoader: Invalidate the screen from menu perspective mnu exit
r329987: lualoader: Track the menu currently drawn, instead of validity
r330008: lualoader: A little more general menu cleanup
r330009: lualoader: More argument name expansion, part 2
r330010: lualoader: screen argument fixes
r330012: style.lua(9): Add some additional notes about naming and commas
r330020: lualoader: Re-work menu skipping bits
r330082: lualoader: Add a twiddle at password prompt
r330083: lualoader: Remove remnants of testing...
r330084: lualoader: Replace instances of \027 with KEYSTR_ESCAPE
r330087: lualoader: Convert instances of KEYSTR_ESCAPE .. "[" -> KEYSTR_CSI
r330088: lualoader: Correct test sense, comments, and add some more comments
r330098: lualoader: Re-do twiddle
r330099: lualoader: Further screen cleanup
r330100: lualoader: Remove debug function
r330101: lualoader: Add note that \027 is a decimal representation
r330138: lualoader: Dedup these "Return to main menu" entries
r330139: lualoader: config: Pull some messages out into constants
r330261: lualoader: Fix some lint-mentioned errors
r330262: lualoader: Use #str instead of tracking length with 'n'
r330263: lualoader: Use string literal \xNN instead of string.char()
r330267: Add core.lua(8), but do not add to distribution
r330268: Add menu.lua(8), but do not add to distribution
r330269: core.lua(8): Add missing note about core.KEYSTR_CSI
r330281: lualoader: Steamroll the box-drawing
r330282: lualoader: Register loader.printc as global printc
r330283: lualoader: Use global printc instead of loader.printc
r330284: liblua: Use putc instead of printf for printc
r330287: lualoader: Reset the cursor position after the menu is drawn
r330339: liblua: Add loader.interpret
r330340: lualoader: Execute menu_timeout_command at the end of menu autoboot
r330341: lualoader: Respect loader_menu_title, prepare for align
r330342: lualoader: Respect loader_menu_title_align
r330345: lualoader: Tweak positioning and fix an off-by-one
r330346: lualoader: Shift menu+brand even for logo=none with customized pos
r330369: lualoader: Return meaningful value in cli_execute
r330370: lualoader: logdef -> logodef typo
r330434: lualoader: Add note about importance of including cli module early
r330435: lualoader: Use FILESDIR instead of BINDIR
r330564: lualoader: Only loadelf before boot/autoboot if no kernel loaded
r330616: lualoader: Expose loader.parse and add cli_execute_unparsed
r330617: lualoader: Fix name, cli.execute_unparsed -> cli_execute_unparsed
r330618: lualoader: Use cli_execute_unparsed instead of loader.interpret
r330620: lualoader: Use cli_execute_unparsed for commands via loader.conf
r330625: lualoader: Return status in cli_execute_unparsed properly
r330690: stand: Fix copy-paste-o, unbreaks libi386 lualoader build
r330701: lualoader: Don't redraw the autoboot message every .05s
r330703: lualoader: Cache kernel list
r330825: lualoader: Sprinkle some verbose_loading salt
r331211: lualoader: Setup default color scheme if we're using colors
r331257: lualoader: Reset attributes and color scheme with color.highlight()
r331259: lualoader: Use less atomic options for resetting colors/attributes
r331281: lualoader: Add primitive hook module to untangle bogus reference
r331282: core.lua(8): Update to reflect recently added function
r331304: lualoader: Clear up some possible naming confusion
r331314: lualoader: Use printc when we expect ANSI escape sequences
r331476: lualoader: Make config env-related bits private API
r331477: lualoader: Privatize some more config.lua bits
r331563: lualoader: Implement try_include and use it for including local
r331564: lualoader: Actually re-raise error in try_include
r331854: lualoader: Do case-insensitive comparison of "yes"
r331855: lualoader: Don't assume that {module}_load is set
r331856: lualoader: revert whitespace change that snuck in
r331857: lualoader: Simplify some expressions
r331859: lualoader: Split logodefs out into logo-* files
r331860: lualoader: Don't try to lookup a nil logo
r332106: lualoader: Fix menu skipping with loader.conf(5) vars
r334723: lualoader: Add a loaded hook for others to execute upon config load
r334879: lualoader: Add hook.lua(8) to tree
r334891: lualoader: Add cli.lua(8) to the tree
r334907: lualoader: Process loader_conf_files properly
r334912: lualoader: Support variable substitution in env var settings
r334939: lualoader: Allow brand-*.lua for adding new brands
r334986: lualoader: More black-on-white fixes
r335009: lualoader: Match Forth module-loading behavior w.r.t flags
r335371: lualoader: Correct kernel_options handling
r336759: lualoader: "nextboot_file" should be spelled "nextboot_conf"
r337711: lualoader: Fix parsing of negative number loader.conf(5) variables
r337807: Prevent a wanring about checkdp being unused.
r337808: When the LUA floating point model is INT64, we don't need to do the
r337809: For our INT64 implementation, we can compare integers and numbers
r337810: MFV r337586: lua: Update to 5.3.5
r338054: Add config.lua(8) to the tree
r338063: lualoader: Stop exporting drawer.draw
r338065: lualoader: Hide most of the internal drawing functions
r338066: lualoader: Hide the rest of the private interfaces
r338067: lualoader: Add drawer-exported variables for default logodefs
r338083: Add drawer.lua(8)
r338085: Add color.lua(8), password.lua(8), and screen.lua(8)
r338086: lualoader: Install all manpages
r338108: Serial console menus for lua.
r338167: lualoader: Just compare expression directly
r338168: lualoader: Refactor config line expressions
r338173: lualoader: Fix loader.conf(5) EOL validation for 'exec' lines
r338203: Turn off LOADER_GELI and LOADER_LUA for sparc64, until functional
r338255: lualoader: Fix (add) Xen support
r338259: lualoader: Accept that people use unquoted values in loader.conf
r338309: lualoader: Fix override of module_path on loader prompt
r338394: lualoader: fix color usage
r338438: lualoader: Handle comma-separated kernels as well
r338886: Improve loader passwords:
r338893: Set the default loader for powerpc(32- and 64-bit) back to to forth
r339173: Set the default loader for powerpc64 back to to forth too.
r339200: lualoader: Don't draw loader menu with autoboot_delay=-1
r339218: lualoader: Create a module blacklist, add DRM modules to it
r339222: lualoader: Honor boot_* variables at lua init
r339301: Loader GELI support, like lua loader, seems to be broken on PowerPC
r339307: lualoader: Provide a 'menu' command to redraw the menu at prompt
r339677: lualoader: unload upon kernel change if a kernel was loaded
r339678: menu.lua: Abort autoboot sequence on failed command
r339702: lualoader: Improve module loading diagnostics
r339805: lualoader: Always return a proper dictionary for blacklist
r339831: Move LUA_ROOT to /boot/lua
r339849: lualoader: Fix try_include error handling
r340040: lualoader: Implement boot-conf
r340152: lualoader: Add chainload menu entry

Relnotes: yes ("lualoader has been merged to stable/11, off by
default, to facilitate testing")

MFC r329190, r329315, r330131: metadata load file unification

r329190: Unify metadata load files for arm, mips, powerpc, sparc64

Summary:
All metadata.c files are very similar, with only trivial changes. Unify them
into a single common file, with minor special-casing where needed.

r329315: stand: Fix ubldr after r329190

metadata load files were consolidated in r329190, and these relocation fixup
bits were inadvertently dropped in the process. Re-add them to fix boot with
ubldr.

r330131: Fix module loading on arm after the metadata.c unification in
r329190. Arm modules need an additional address fixup not needed by other
platforms.

MFC r329725, r329831

r329725:
Consolidate three copies of ZFS commands into a central location.

There's no reason to have multiple copies of lszfs and
reloadbe. Consolidate them into one location. Also ldi_get_size is the
same everywhere (except sparc64). Make it the same everywhere as the
common definition is more general and will work on spar64.

r329831:
Fix userboot w/ ZFS after r329725

r329725 cleaned up ZFS commands duplicated in multiple places, but userboot
was not setting HAVE_ZFS when MK_ZFS != "no". This resulted in a failure to
boot (as seen in PR 226118) in bhyve, with the following message:

/boot/userboot.so: Undefined symbol "ldi_get_size"

MFC Loader Fixes Final: r327612,r327703,r327704,r327878,r327879,r327881,
r328007,r328029,r328030,r328031,r328061,r328156,r328169,r328288,r328289,
r328290,r328291,r328292,r328411,r328536,r328603,r328614,r328642,r328769,
r328779,r328780,r328781,r328782,r328783,r328806,r328808,r328826,r328835,
r328911,r328986,r328987,r328990,r328999,r329000,r329019,r329050,r329054,
r329060

r327612: Invent new #defines for the biospci_{read,write}_config function

r327703: Define __dmadat after #include'ing ufsread.c.

r327704: Fix printf missing format variables warnings.

r327878: Add GUID for UEFI boot manager variables.

r327879: Report the boot order and where we are in that boot order.

r327881: Allow this file to be included

r328007: Fix booting on some arm64 systems after r327879 by fixing the call
to utf8_to_ucs2

r328029: When returning an error and freeing allocated memory from
ucs2_to_utf8, NULL the return pointer.

r328030: Check the return value from utf8_to_ucs2 instead of whether or not
uv is NULL.

r328031: Need to free uv after we're done using it.

r328061: utf8_to_ucs2() should check for malloc failure

r328156: stand: Move sections around to fix stand/ build with ld.lld on
armv7

r328169: Remove extra copy of bootinfo.c. It's a bit rotted copy of the one
in efi/loader.

r328288: Fix some resource leaks.

r328289: Don't leak memory when displaying help.

r328290: On malloc failure, be sure to close the include file that triggered
it.

r328291: getenv does not return tainted data in the boot loader. Attempt to
clue Coverity into that fact.

r328292: There's no tainted data here, tag it as such to avoid false
positives.

r328411: loader.efi: add missing EFI GUIDs

r328536: loader: support for mixed-endianness ELF/loader and POWER8

r328603: Add missing non-POWERPC case to give the scr value something
non-zero.

r328614: Move libstand.3 to libsa.3. Update libsa.3 to include functions

r328642: Break out the interpreters (simple and forth) w/o ifdefs.

r328769: Centralize several variables.

r328779: Retire pnp.4th and the code needed only for 4th words used here.

r328780: These 4th words were an attempt to allow integration into the boot
loader scripts. However, that path won't be taken after all it
seems.

r328781: Remove pcibios forth support.

r328782: Now that we no longer conditionally compile some files outside of
ficl

r328783: Invent new LDR_INTERP for the loader interpreter to use.

r328806: We need more heap space to properly load newer powerpc kernels.

r328808: Implement strcoll as strcmp.

r328826: Make cross-endian loader changes apply only to powerpc

r328835: Fix regression introduced in r328806, preventing boot on many
platforms.

r328911: Ignore relocation tables for non-memory-resident sections.

r328986: Fix relative location of USB sources after recent move.

r328987: A more definitions to kernel emulation shim in order to build
stand/usb.

r328990: Move the stand/usb test loader into its own directory.

r328999: Fix indentation to FreeBSD standard for interp files

r329000: Move simple interpreter 'perform' into interp.c and call it

r329019: Move to tabs for indentation and to 8-space notches, per style(9).

r329050: Fix build of userboot.so

r329054: Set script.lang in the environment to either 'forth' or 'simple' to
reflect what scripting language was compiled into the loader.

r329060: loader: fix endianness conversion

PR: 225323

MFC Loader Fixes 2017q4p7: r324844,r326089,r326926,r326440,r326484,r326494,
r326588,r326708,r326784,r326914,r327390,r328446,r326090,r326143,r326144,
r326182,r326384,r326421,r326440,r326441,r326442,r326443,r326444,r326445,
r326446,r326447,r326448,r326484,r326485,r326486,r326487,r326488,r326490,
r326491,r326492,r326493,r326494,r326495,r326504,r326507,r326509,r326584,
r326585,r326586,r326587,r326588,r326589,r326590,r326591,r326592,r326593,
r326594,r326600,r326616,r326671,r326707,r326708,r326709,r326710,r326711,
r326712,r326714,r326720,r326768,r326772,r326784,r326792,r326812,r326854,
r326855,r326856,r326858,r326886,r326887,r326914,r326926,r326927,r326960,
r326961,r326962,r326963,r327351,r327453,r327390,r327523,r327524,r326489,
r327880,r328437,r328438,r328439,r328441,r328446,r328448,r328449,r328612,
r328613,r328615

While here, undo our libfdt hack of not including <stdlib.h> if we're
compiling _STANDALONE.

r324844: When building standalone, don't define errno. Let the definition from
stand.h override. This is similar to what we do in the kernel.

r326089: loader.efi: efipart does not recognize partitionless disks

r326090: net_parse_rootpath() has no parameters

r326143: Fix theoretical integer overflow issues. If the product here is

r326144: Mark the func pointer as __dead2. It looks up loader_main, which

r326182: Modify all FreeBSD bootloaders on PowerPC AIM (Book-S) systems

r326384: Use const pointers to avoid casting away constness.

r326421: loader.efi: efipart should exclude iPXE stub block protocol

r326440: Remove stale dependency on ufsread.c

r326441: Minor flags cleanup

r326442: Cleanup CFALGS usage here

r326443: We don't need both _STAND and _STANDALONE, use the latter.

r326444: Move geli to common DO32 stuff

r326445: Fix random() and srandom() prototypes to match the standard.

r326446: Undefine _STANDALONE since this is test code.

r326447: Tweaks to the beri boot loader so that it builds w/o warnings.

r326448: Fix all warnings related to geli and ZFS support on x86.

r326484: Const poison the propname.

r326485: Delcare md_load in libofw.h. Make all prototypes match for ofw

r326486: Include machine/md_var to pick up __syncicache prototype.

r326487: Cast mdp (a vm_offset_t) to void * to match prototype.

r326488: e_entry can be smaller than a pointer. Cast it to an intptr_t

r326490: Declare our strange brand of main().

r326491: Disconnet ps3 from the build. There's too many warnings to fix.

r326492: Cast void * pointer to char * so the arg matches the %s format.

r326493: Provide a md_load64 prototype.

r326494: Mark two things as unused (since they are only sometimes used)

r326495: Now it's safe to bump WARNS to 1.

r326504: Switch to proper MK_LOADER_GELI tests.

r326507: increase maximum size of zfsboot

r326509: loader.efi: add note about iPXE into the efipart.c

r326584: When building standalone, include stand.h rather than the kernel

r326585: Include ficl.h before anything else

r326586: No need to include the userland md5.h, the kernel one is just fine.

r326587: Use the kernel relative paths, rather than the userland relative
paths

r326588: Need to include skein in the include path

r326589: Make sure we include the right path for skein.h

r326590: Prefer stdint.h to inttypes.h

r326591: This isn't NetBSD specific code. Include these for any kernel /

r326592: Don't inherit CFLAGS. This a specialized test program.

r326593: Stop building with the standard system headers.

r326594: Now that we offer a semi-sane standards-ish set of #include files,
stop hacking includes with sed.

r326600: Since this is contrib code, create an upstreamable version of my

r326616: dhcp_try_rfc1048() is not used any more

r326671: Avoid setting -Wno-tentative-definition-incomplete-type with gcc.

r326707: Add partial support signal.h functioanlity. Pull in
machine/signal.h

r326708: Remove _KERNEL hack now that errno.h does the right thing when
built standalone.

r326709: Provide implementations for iscntrl, ispunct and isgraph.

r326710: Put the files we're copying over into a few variables and clean
hings up.

r326711: Const poison a couple of interfaces.

r326712: Create interp class.

r326714: boot1.c needs EFI_ZFS_BOOT too, so add it globally.

r326720: This path belongs in ficl/Makefile, not the common defines for
users

r326768: Fix a comment to be more accurate

r326772: Fix regression with lua import

r326784: Revert part of 362772. It was causing problems for includes

r326792: Attempt to unbreak buildworld

r326812: Revert r326792, r326784, r326772, r326712

r326854: libefi: make efichar.h more usable in stand code

r326855: Cargo cut a fix for the regressions r326585 caused.

r326856: Fix comments after bump in size.

r326858: Revert r326855: Cargo cut a fix for the regressions r326585 caused.

r326886: Panic in sbrk if setheap hasn't been called yet. This is preferable
o a mysterious crash

r326887: Remove the 'mini libstand in libstand' that util.[ch] provided.

r326914: Move loader-only defines to loader.mk from defs.mk

r326926: Move loader help file definitions to being 100% inside of
loader.mk.

r326927: libficl is only ever used in a loader (never a boot) program. Move
it.

r326960: Simplify things a little. The RETURN macro isn't required.

r326961: Interact is always called with NULL. Simplify code a little

r326962: Hoist btx include stuff to i386/Makefile.inc

r326963: No need to use relative paths like this here.

r327351: Fix ubldr. uboot/lib uses defines for the loader.

r327453: Add a validbcd() routine that uses the bcd2bin_data[] array

r327390: Garbage-collect loader.ps3. It is currently disconnected from the
build and kboot replaces.

r327523: Don't clobber system LDFLAGS for beri boot loaders.

r327524: Use 'extern uint8_t' instead of 'extern void' for external symbols.

r326489: Allow this file to be used in libsa without warning...

r327880: Move getsecs() prototype to stand.h from net.h so it can be used

r328437: Split panic routine

r328438: Implement abort() as a call to panic.

r328439: Provide abs form stdlib.h.

r328441: abort() should be marked __dead2 since it won't return.

r328446: Now that exit is __dead2, we need to tag ub_exit() as __dead2.

r328448: Make exit() never return until host_exit can be written.

r328449: Tag unreachable places as such. I left the while (1); in place

r328612: Move strtold wrapper from strtol.c to its own strtold.c.

r328613: Kill copies of strtol and strtoul.

r328615: Update stand.h for changes for strto*l

PR: 223969

MFC r325834,r325997,326502: Move sys/boot to stand/

This is effectively a direct commit to stable/11, due to differences between
stable/11 and head. Changes to DTS in sys/boot/fdt/dts were often
accompanied by kernel changes. Many of these were also risc-v updates that
likely had many more dependencies to MFC.

Because of this, sys/boot/fdt/dts remains as-is while everything else in
sys/boot relocates to stand/.

r325834: Move sys/boot to stand. Fix all references to new location

r325997: Remove empty directories.

r326502: Document the sys/boot -> stand move in hier.7 and the top-level README.