pxeboot: improve and simplify rx handling

This pushes the bulk of the rx servicing into a single loop that's only
slightly convoluted, and it addresses a problem with rx handling in the
process. If we hit a tx interrupt while we're processing, we'd
previously drop the frame on the floor completely and ultimately
timeout, increasing boot time on particularly busy hosts as we keep
having to backoff and resend.

After this patch, we don't seem to hit timeouts at all on zoo anymore
though loading a 27M kernel is still relatively slow (~1m20s).

Sponsored By: National Bureau of Economic Research
Sponsored by: Klara, Inc.

(cherry picked from commit 3daa8e165c661c1b45e759f4997f447384c15446)

Git Hash: c0674b71720bb333cbc2da823e2f67412dc00e78
Git Author: kevans@FreeBSD.org

MFC r344855 (by jhb):

Drop "All rights reserved" from my copyright statements.

Reviewed by: rgrimes
Differential Revision: https://reviews.freebsd.org/D19485

MFC r366364:

Correct the documented size of kern.msgbufsize

The correct value is 96KB after r226090.

PR: 249971
Submitted by: johannes@jo-t.de

MFC r361763: lualoader: drop the filename and word "LUA" from errors

The filename is nearly always wrong since it's /boot/lua/loader.lua, which
is not useful for diagnostics. The actual errmsg will include a lua filename
if this is relevant.

Dropping "LUA" while we're here because that's almost universally
irrelevant to whatever error follows, unless the error states that it's
actually a lua problem.

Both of these are minor nits that just detract from identifying the
pertinent information.

MFC r361709: lualoader: improve drawer error handling

At least one user has landed in a scenario where logo files appear to be
misnamed, and we failed to find them. Our fallback for missing logodefs is
orb/orbbw, based on the color status. In a scenario where we can't locate
the logos, though, this is not ideal. Add in one more layer of fallback
to properly just don't draw any logo if the fan has been jam packed with
foreign material.

PR: 246046

loader: fix userboot's ability to detect a guest's interpreter

Some time after r338418, I believe with -Os/-Oz -ffunction-sections
-fdata-sections, the bootprog_interp variable that held our "$Interpreter:"
marker started getting strip from all loaders, with exception to userboot
since it used bootprog_interp to determine what flavor of userboot it was.

At some point, it had been brought to my attention that this was no longer
working and I had worked up some potential solutions to use the variable
that involved printing it out. My vague recollection is that this was
rejected, and I forgot to explore the alternatives; I cannot find records of
this discussion anymore.

Fast forward to present day, Andrew reported that it was non-functional and
offered (effectively) this patch (sans comment) to stop the compiler from
optimizing it out by assigning it to a volatile variable. This removes
concerns about user-facing change while retaining the interpreter marker.
Furthermore, it could certainly be uglier.

MFC r360836:

loader: vdev_read() can corrupt memory

When reading less than sector size but from sector boundary,
the vdev_read() will read full sector into the provided buffer
and therefore corrupting memory past buffer end.

MFC r358643:

Link stand/i386 components using a linker script

LLD 10.0.0 changed the behavior of the -Ttext option, so that using
-Ttext=0x0 now causes linking of the loaders to fail with:

ld: error: output file too large: 18446744073707016908 bytes

I reported this in https://bugs.llvm.org/show_bug.cgi?id=44715, and
initially reverted the upstream change in r357259 to work around it.

However, after some discussion with Fangrui Song in the upstream ticket,
I think we can classify this as an unfortunate interaction between using
-Ttext=0 in combination with --no-rosegment. (We added the latter
in r332090, because btxld does not correctly handle input with more
than 2 PT_LOAD segments.)

Fangrui suggested to use a linker script instead, and Warner was already
attempting this in r305353, but had to revert it due to "crypto-using
boot problems" (not sure what those were :).

This review updates the stand/i386/boot.ldscript to handle more
sections, inserts some symbols like _edata and such that we use in
libsa, and also discards any .interp section.

It uses ORG which is defined on the linker command line using
--defsym ORG=value to set the start of all the sections.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D23952

lualoader: fix loader.lua mis-merge

This mis-merge came from r360599; r331321 introduced core.isUEFIBoot but
will not get merged back to stable/11 due to being somewhat disruptive. I
had resolved the conflict once, but had to re-do the merge locally due to
another conflict and failed to resolve it the second time.

This mistake is explosive to those trying to run lualoader on stable/11, but
is mitigated by the fact that lualoader is not the default here.

PR: 246291

MFC r355940:

Move all sources from the llvm project into contrib/llvm-project.

This uses the new layout of the upstream repository, which was recently
migrated to GitHub, and converted into a "monorepo". That is, most of
the earlier separate sub-projects with their own branches and tags were
consolidated into one top-level directory, and are now branched and
tagged together.

Updating the vendor area to match this layout is next.

Revert r360605, as it is actually the cause for the linker set errors:

Un-MFC r352032 (by tsoome):

loader: --gc-sections needs sections to work with

--gc-sections is not really useful unless we generate sections with
-ffunction-sections -fdata-sections

While there, i386/loader would win from --gc-sections too.

Revert r360606, as it results in linker set errors while building for
amd64, I'll have to re-do this more carefully:

Un-MFC r354043 (partial, by sjg):

Add support for hypervisor check on x86

Add ficl words for isvirtualized
and move ficl inb and outb words to ficl/x86/sysdep.c
so can be shared by i386 and amd64

Reviewed by: imp bdrewery
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22069

MFC r354043 (partial, by sjg):

Add support for hypervisor check on x86

Add ficl words for isvirtualized
and move ficl inb and outb words to ficl/x86/sysdep.c
so can be shared by i386 and amd64

Reviewed by: imp bdrewery
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22069

Note, only the stand/defs.mk changes were applied, as this fixes the
following error during buildworld for TARGET=pc98:

--- cleandir_subdir_stand ---
rm: i386: is a directory
*** [clean] Error code 1

MFC r352032 (by tsoome):

loader: --gc-sections needs sections to work with

--gc-sections is not really useful unless we generate sections with
-ffunction-sections -fdata-sections

While there, i386/loader would win from --gc-sections too.

MFC r359371: loader: Fully reset terminal settings, not just colors

MFC r360199: menu.lua: Give names to menu entries

Make menu customizations easier by naming the entries and using the
names to build the table entries.

MFC r352601: loader_lua: lua color changes should end with reset

The color change should have reset sequence, not switch to white.

MFC lualoader read-conf support: r360420-r360423, r360425, r360427, r360486,
\r360505-r360506

r360420:
lualoader config: don't call loader.getenv() as much

We don't actually need to fetch loader_conf_files as much as we do; we've
already fetched it once at the beginning, we only really need to fetch it
again after each file we've processed. If it changes, then we can stash that
off into our local prefiles.

While here, drop a note about the recursion so that I stop trying to
change it. It may very well make redundant some of the work we're doing, but
that's OK.

r360421:
lualoader: config: start exporting readConfFiles

In the process, change it slightly: readConfFiles will take a string like
loader_conf_files in addition to the loaded_files table that it normally
takes. This is to facilitate the addition of a read-conf CLI command, which
will just pass in the single file to read and an empty table.

r360422:
lualoader: cli: add read-conf

This is a straightforward match to the command used by many in forthloader;
it uses the newly-exported config.readConfFiles() to make sure that any
loader_conf_files gets done as appropriate.

r360423:
lualoader: cli: clobber loader_conf_files before proceeding

This makes sure that config.readConfFiles doesn't see a stale
loader_conf_files from before, in case the newly loaded file doesn't set it.

r360425:
config.lua(8): "may should" is not proper grammar

r360427:
config.lua(8): catch up to recently added hooks

While we're here, let's stylize these as functions instead of just raw text.
A future change may allow arbitrary data arguments to be passed some of
these, and the distinction is useful.

r360486:
loader.conf(5): document that loader_conf_files may be clobbered

A future change in lualoader may take some liberties with the
loader_conf_files in the name of efficiency; namely, it may start omitting
it from the loader environment entirely so that it doesn't need to worry
about maintaining any specific value.

This variable has historically been incredibly volatile anyways, as it may
get set to completely different values in any given configuration file to
trigger a load of more files.

Document now that we may not maintain it in the future, but perhaps we'll
reserve the right to change our minds and eventually formally export all of
the loader configuration files that were read using this variable.

r360505:
lualoader: config: add a table for restricted environment vars

This new table should be used for transient values that don't need to end up
in the loader environment. Generally, these will be things that are internal
details that really aren't needed or interesting outside of the config
module (e.g. if we changed how ${module}_* directives work, they might use
this instead).

To start, populate it with loader_conf_files. Any specific value of
loader_conf_files isn't all that interesting; if we're going to export it,
we should really instead export a loader_conf_files that indicates all of
the configuration files we processed. This will be used to reduce
bookkeeping overhead in a future commit that cleans up readConfFiles.

r360506:
lualoader: config: improve readConfFiles, rename to readConf

The previous interface was pretty bad, and required the caller to get some
implementation details correct that it really shouldn't need to (e.g.
loader_conf_files handling) and pass in an empty table for it to use.

The new and much improved interface, readConf, is much less of a hack;
hiding these implementation details and just doing the right thing.
config.lua will now use it to process /boot/defaults/loader.conf and the
subsequent loader_conf_files from there, and read-conf will also use it.

This improvement submitted by Olivier (cited below), loader_conf_files
handling from the original patch was changed to just clobber it before
processing and not bother restoring it after the fact following r360505
where it's now guaranteed to evade the loader environment.

PR: 244640

MFC r359688: stand: -fno-common fixes for !x86 loaders

- beriloader: archsw is declared extern and defined elsewhere
- ofwloader: ofw_elf{,64} are defined in elf_freebsd.c and
ppc64_elf_freebsd.c respectively
- ubldr: syscall_ptr is defined in start.S for whichever ubldr platform is
building

-fno-common will become the default in GCC10/LLVM11.

MFC -fno-common fixes: r359389, r359394, r359397-r359399, r359403-r359404,
r359406, r359413-r359416, r359425, r359427, r359432-r359433, r359443,
r359675-r359677

Note: this is not necessarily a complete fix to get these programs to build
with -fno-common applied.

r359389:
config(8): fixes for -fno-common

Move this handful of definitions into main.c, properly declare these as
extern in config.h. This fixes the config(8) build with -fno-common.

Unexplained in my previous commit to gas, -fno-common will become the
default in GCC10 and LLVM11, so it's worth addressing these in advance.

r359394:
MFV r359393: tcsh: import 6974bc35a5cd

This removes an extra variable definition that causes the -fno-common build
to fail, which will be a new default in GCC10/LLVM11.

r359397:
zfs: fix -fno-common issues

A similar (or identical?) fix has already landed in OpenZFS.

-fno-common will become the default in GCC10/LLVM11.

r359398:
sh: remove duplicate el definition

el is declared extern in myhistedit.h and defined in histedit.c. Remove the
duplicate definition in input.c to appease the -fno-common build.

-fno-common will become the default in GCC10/LLVM11.

r359399:
telnet: remove some duplicate definitions, mark terminaltype extern

Most of these were already properly declared and defined elsewhere, this is
effectively just a minor cleanup that fixes the -fno-common build.

-fno-common will become the default in GCC10/LLVM11.

r359403:
Revert 359399: telnet -fno-common bits

There was a large misfire from my local diff that I need to investigate, and
this version committed did not build.

r359404:
Re-apply r359399: telnet -fno-common fix

line and auth_level's redefinitions are just extraneous

telnetd will #define extern and then include ext.h to allocate storage for
all of these extern'd vars; however, two of them are actually defined in
libtelnet instead. Instead of doing an #ifdef extern dance around those
function pointers, just add an EXTERN macro to make it easier to
differentiate by sight which ones will get allocated in globals.c and which
ones are defined elsewhere.

r359406:
telnet: kill off remaining duplicate definition

r359413:
ipfilter: remove duplicate definition of 'thishost'

thishost is already defined in lib/initparse.c; no need for this one. This
fixes the ipfilter build with -fno-common.

-fno-common will become the default in GCC10/LLVM11.

r359414:
iscontrol: move definition of vflag/iscsidev to iscontrol.c

Mark the declaration extern as these are used elsewhere; this fixes the
build with -fno-common.

r359415:
userboot: mark host_fsops as extern

This is already defined elsewhere; mark this declaration extern to the fix
the -fno-common build.

r359416:
systat: remove redundant definition of kd

kd is already properly declared in extern.h and defined in main.c, rendering
this definition useless. This fixes the -fno-common build.

r359425:
locate: fix -fno-common build

Just a single variable declaration to extern and define elsewhere here,
myctype.

-fno-common will become a default in GCC10/LLVM11.

r359427:
fsck_ffs/fsdb: fix -fno-common build

This one is also a small list:

- 3x duplicate definition (ufs2_zino, returntosingle, nflag)
- 5x 'needs extern', 3/5 of which are referenced in fsdb

-fno-common will become the default in GCC10/LLVM11.

r359432:
gdb: compile with -fcommon explicitly

As described in the comment, gdb relies on some of the linker magic that
happens with -fcommon. I suspect the life expectancy of gdb-in-base is low
enough that this isn't worth spending much time addressing, especially given
the vintage. Hit it with the -fcommon hammer so that it continues to just
work.

r359433:
bmake: fix -fno-common build

debug was declared extern, but debug_file was not; correct this and define
debug_file in main.c (as debug is) to fix the -fno-common build.

-fno-common will become the default with GCC10/LLVM11.

r359443:
MFV r359442: bmake: import -fno-common fix build back from upstream

sjg@ committed the local patch previously committed upstream; pull it in to
vendor/ to ease any potential stress of future imports.

r359675:
kqueue tests: fix -fno-common build

vnode_fd and kqfd are both shared among multiple CU; define them exactly
once.

In the case of vnode_fd, it was simply the declaration that needed
correction.

-fno-common will become the default in GCC10/LLVM11.

r359676:
ntpd: fix build with -fno-common

Only a small nit here: psl should be declared extern and defined exactly
once.

-fno-common will become the default in GCC10/LLVM11.

r359677:
yp*: fix -fno-common build

This is mostly two problems spread out far and wide:
- ypldap_process should be declared properly
- debug is defined differently in many programs

For the latter, just extern it and define it everywhere that actually needs
it. This mostly works out nicely for ^/libexec/ypxfr, which can remove the
assignment at the beginning of main in favor of defining it properly.

-fno-common will become the default in GCC10/LLVM11.

MFC r359072-r359073: loader: Add a "kernel.loaded" hook

This hook can be useful, for example to run a local function to choose
different modules to load when a user has picked a different kernel
from the menu.

MFC r357828: interp_lua: fix bogus indentation, NFC

MFC r357845: efiloader: don't execute hooks when setting currdev/loaddev

We still need to set the hooks to prevent improper manipulations thereafter
at the loader prompt, but as it is we're actively preventing loaddev from
being set correctly in some circumstances (ZFS in particular) and doing more
work than needed with currdev -- that hook in particular validates it as a
correct device, which we can assume isn't needed in this context.

MFC r357827: lualoader: disable autoboot on high-level interpreter errors

If we hit an error at this level, we are almost certainly not in any kind
of sane state where autoboot can do the right thing. Instead of letting it
try and probably failing, disable autoboot so they immediately get kicked
into a loader prompt for manual remediation/diagnosis.

MFC r357442:

loader: bc_add can not use any other probes than ah=0x4b

CD boot is broken for some systems since bioscd and biosdisk merge. The issue is that we can not use anything else than int 13 ah=0x4b to query cd information.

The patch does restore the same probe as was originally used in bioscd.c. Additionally extra buffer padding is used to avoid memory corruption caused by some systems.

PR: 234031
Reported by: ultramage and others

MFC r357232:

Merge r357231 from the clang1000-import branch:

Work around assembler error from clang 10.0.0 in gptboot:

stand/i386/gptboot/gptldr.S:141:3: error: value of 36878 is too large for field of 2 bytes.
jmp MEM_JMP # Start BTX
^

Use the same construct as in stand/i386/boot2/boot1.S, which ensures the
jump distance does not become too large.

MFC r357103-r357104: unbreak local.lua, add a modules.loaded hook

r357103:
loader.lua: re-arrange to load local.lua *after* config loading

The major problem with the current ordering is that loader.conf may contain
all of the magic we need to actually setup the console, so loading local.lua
prior to that can make it excessively difficult and annoying to debug
(whoops, sorry Ravi & Warner).

The new ordering has some implications, but I suspect they are a non-issue.
The first is that it's no longer possible for the local module to inject any
logic prior to loading config -- I suspect no one has relied on this. The
second implication is that the config.loaded hook is now useless, as the
local module will always be included after that hook would have fired.

For config.loaded, I will opt to leave it in, just in case we add an early
point for local lua to get injected or in case one wants to schedule some
deferred logic in a custom loader.lua. The overhead of having it if no hooks
will be invoked is relatively minimal.

r357104:
lua: add modules.loaded hook

This may be used for the local module to hook in and load any additional
modules that it wants, since it can't modify the modules table internal to
config. We may consider adding API to do so at a later time, but I suspect
it will be more complicated to use with little return.

status is captured but ignored for the purpose of loading the hook. status
will be false if *any* module failed to load, but we typically don't let
that halt the boot so there's no reason to let it halt hooks. Some vendors
or setups may have expected fails that would be actively thwarted by
checking it.

We may, at a later date, consider adding an API for letting non-config
modules check which modules have successfully (or not) loaded in case an
unexpected failure *should* halt whatever they are doing.

MFC r356538: stand/fdt: Scale blob size better as overlays apply

Currently, our overlay blob will grow to include the size of the complete
overlay blob we're applying. This doesn't scale very well with a lot of
overlays- they tend to include a lot of overhead, and they will generally
only add a fraction of their total size to the blob they're being applied
to.

To combat this, pack the blob as we apply new overlays and keep track of how
many overlays we've applied. Only ubldr has any fixups to be applied after
overlays, so we only need to re-pad the blob in ubldr. Presumably the
allocation won't fail since we just did a lot worse in trying to apply
overlays and succeeded.

I have no intention of removing the padding in make_dtb.sh. There might be
an argument to be had over whether it should be configurable, since ubldr
*is* the only loader that actually has fixups to be applied and we can do
this at runtime, but I'm not too concerned about this.

This diff has been sitting in Phabricator for a year and a half, but I've
decided to flush it as it does make sure that we're scaling the blob
appropriately and leave room at the end for fixups in case of some freak
circumstance where applying overlays leaves us with a blob of insufficient
size.

MFC r356570:
loader: bioscd probe can get sector size 0

With buggy BIOS, it may happen we get sector size reported 0 for cd, and then
the default 512 is used, which is quite wrong.

PR: 238749

stand: liblua: drop default buffer size to 128

Lua allocates LUAL_BUFFERSIZE buffers on the stack for various string
functions (string.format, string.gsub) -- this works out to be somewhat
significant and not necessary, based on how we use string operations.
Dropping it risks having to allocate per call to format/gsub, but this is
not the case for our usage. This simply stops allocating 8K buffers on the
stack when luaL_Buffer is used.

MFC r354247, r355349: lualoader try_include improvement

r354247: lualoader: rewrite try_include using lfs + dofile

Actual modules get require()'d in, rather than try_include(). All instances
of try_include should be provided with proper hooks/API in the rest of
loader to do the work they need to do, since we can't rely on them to exist.
Convert this now to lfs + dofile since we won't really be treating them as
modules.

lfs is required because dofile will properly throw an error if the file
doesn't exist, which is not in the spirit of 'optionally included'.

Getting out of the pcall game allows us to provide a loader.exit() style
call that backs out to the common bits of loader (autoboot sequence unless
disabled with a loader.setenv("autoboot_delay", "NO")). The most ideal way
identified so far to implement loader.exit() is to throw a special
abort-style error that indicates to the caller in interp_lua that we've not
actually errored out, just continue execution. Otherwise, we have to hack in
logic to bubble up and return from loader.lua without continuing further,
which gets kind of ugly depending on the context in which we're aborting.

A compat shim is provided temporarily in case the executing loader doesn't
yet have loader.lua_path, which was just added in r354246.

r355349: lualoader: correct a typo from r354247

r354247 converted try_include to lfs + dofile with the loader.lua_path added
just before. Fortunately, there was a hardcoded /boot/lua fallback in case
loader.lua_path wasn't being set yet- I typo'd it as loader.lua_paths.

Fix the typo.

MFC r354673:

Logging improvements to loader::nfs

Include the server IP address when logging nfs_open(), add a few missing
"\n"s, and correct a typo.

MFC r354246: liblua: add loader.lua_path

As described previously, loader.lua_path is absolute path where scripts are
installed. A future commit will use this to build paths for dofile in
try_include, rather than the current pcall/require setup that makes it more
difficult to coordinate loader aborts from local.lua -- we do not need the
flexibility of require(), and local.lua is in-fact not a 'module-like' file
as we will not be referencing anything from it.

MFC r354245, r354833, r354837: add flua to the base system

r354245: stand: consolidate knowledge of lua path

Multiple places coordinate to 'know' where lua scripts are installed. Knock
this down to being formally defined (and overridable) in exactly one spot,
defs.mk, and spread the knowledge to loaders and liblua alike. A future
commit will expose this to lua as loader.lua_path, so it can build absolute
paths to lua scripts as needed.

r354833: Add flua to the base system, install to /usr/libexec

FreeBSDlua ("flua") is a FreeBSD-private lua, flavored with whatever
extensions we need for base system operations. We currently support a subset
of lfs and lposix that are used in the rewrite of makesyscall.sh into lua,
added in r354786.

flua is intentionally written such that one can install standard lua and
some set of lua modules from ports and achieve the same effect.

linit_flua is a copy of linit.c from contrib/lua with lfs and lposix added
in. This is similar to what we do in stand/. linit.c has been renamed to
make it clear that this has flua-specific bits.

luaconf has been slightly obfuscated to make extensions more difficult. Part
of the problem is that flua is already hard enough to use as a bootstrap
tool because it's not in PATH- attempting to do extension loading would
require a special bootstrap version of flua with paths changed to protect
the innocent.

src.lua.mk has been added to make it easy for in-tree stuff to find flua,
whether it's bootstrap-flua or relying on PATH frobbing by Makefile.inc1.

r354837: flua: newer GCC complains about format-nonliteral at WARNS=2

Disable that one, too.

MFC r353872-r353873: lualoader color handling fixes

r353872:
lualoader: don't botch disabling of color

When colors are disabled, color.escape{fg,bg} would return the passed in
color rather than the proper ANSI sequence for the color.
color.escape{fg,bg} would be wrong.

Instead return '', as the associated reset* functions will also return ''.
This should get rid of the funky '2' and '4' in the kernel selector if
you're booting serial.

r353873:
lualoader: fix setting of loader_color=NO in loader.conf(5)

Previously color.disabled would be calculated at color module load time,
then never touched again. We can detect serial boots beyond just what we're
told by loader.conf(5) so this works out in many cases, but we must
re-evaluate the situation after the config is loaded to make sure we're not
supposed to be forcing it enabled/disabled.

Discovered while trying to test r353872.

MFC r352559: loader: Respect loader_color=YES for serial consoles

It's not uncommon these days for the terminals attached to serial consoles
to support ANSI escape sequences. However, we assume escape sequences may
break some serial consoles and default to not using them when boot_serial or
boot_multicons (or if console contains "comconsole" in the forth loader) for
broader compatibility. We also have loader_color which can be explicitly set
to "NO" to disable the use of ANSI escape sequences.

The problem is that loader_color=YES gets ignored when boot_serial=YES or
boot_multicons=YES (or when console contains "comconsole" in the forth
loader).

To fix, the existing default behavior remains unchanged when loader_color is
unset, loader_color=NO explicitly disables the use of ANSI escape sequences
still, and the change is that loader_color=YES can now be used to explicitly
allow ANSI escapes when a serial console is enabled.

MFC r352314: lualoader: Add reload-conf loader command

This command will trigger a reload of the configuration from disk. This is
useful if you've changed currdev from recovery media to local disk as much
as I have over the past ~2 hours and are tired of the extra keystrokes.

This is really just a glorified shortcut, but reload-conf is likely easier
to remember for other people and does save some keystrokes when reloading
the configuration. It is also resilient to the underlying config method
changing interface, but this is unlikely to happen.

MFC r352194: lualoader: Revert to ASCII menu frame for serial console

The box drawing characters we use aren't necessarily safe with a serial
console; for instance, in the report by npn@, these were causing his xterm
to send back a sequence that lua picked up as input and halted the boot.
This is less than ideal.

Fallback to ASCII frames for console with 'comconsole' in it. This is a
partial revert r338108 by imp@ -- instead of removing the menu entirely and
disabling color/cursor sequences, just reverting the default frame to ASCII
is enough to not break in this setup.

MFC r351865, r351837:
ficl: add xemit word
ficl: add uIsGreater word (needed for xemit)

We need xemit to support userboot.so loading images from CURRENT.

MFC r351408-r351410: reduce pollution from mips machine/regnum.h

r351408:
libsa: mips: use _JB_* from machine/asm.h, remove regnum dep

This brings the libsa/mips _setjmp implementation closer to parity with the
libc version.

r351409:
mips: hide regnum definitions behind _KERNEL/_WANT_MIPS_REGNUM

machine/regnum.h ends up being included by sys/procfs.h and sys/ptrace.h via
machine/reg.h. Many of the regnum definitions are too short and too generic
to be exposing to any userland application including one of these two
headers. Moreover, these actively cause build failures in googletest
(template <typename T1 ...> expanding to template <typename 9 ...>).

Hide the definitions behind _KERNEL or _WANT_MIPS_REGNUM, and patch all of
the userland consumers to define as needed.

r351410:
libsa: mips: fix typo that had slipped into the diff on local machine

MFC r351119, r351135-r351136, r351412: stand xtoolchain-llvm90 fixes

r351119:
stand: push LIBC_SRC up into defs.mk

Other parts of stand/ that don't use libsa will need to grab bits from libc
shortly. Push LIBC_SRC up to defs.mk in advance of this so that they can use
it, and rename it to LIBCSRC to match the convention of the rest of the *SRC
variables in this file.

r351135:
stand: boot2: fix build with xtoolchain-llvm90

ufsread.c grows a dependency on __ashldi3 with llvm90. Grab ashldi3.c out of
compiler-rt rather than trying to link against libsa (for now).
-Wno-missing-prototypes is necessary to compile ashldi3.c standalone.

r351136:
stand: gptboot: fix build with xtoolchain-llvm90

ufsread.c grows a dependency on __ashldi3 with llvm90. For gptboot, just
start pulling in ashldi3.c ashrdi3.c lshrdi3.c into libsa for all archs as
the number of archs requiring one or more of them keeps growing. qdivrem.c
and quad.h can be trivially kicked out of libsa if we start pulling these
from compiler-rt as qdivrem was only used to implement umoddi3, divdi3,
moddi3 (also in qdivrem.c).

r351412:
stand: boot2: fix amd64-xtoolchain-gcc build

-Wno-missing-declarations is the GCC equivalent of
-Wno-missing-prototypes... this was overlooked in r351135.

MFC r351078, r351085, r351088: mostly a nop (two commits + revert of two)

This commit is mostly a nop, but ends up renumbering #4 clause to #3 in one
copy of quad.h... this is OK; stand/ situation in stable/11 is pretty murky
and the commit that renumbered the clause got lost somewhere. quad.h will be
disappearing in a not-so-distant future MFC.

r351078:
stand: kick out quad.h

Use quad.h from libc instead for the time being. This reduces the number of
nearly-identical-quad.h we have in tree to two with only minor changes.

Prototypes for some *sh*di3 have been added to match the copy in libkern.
The differences between the two are likely few enough that they can perhaps
be merged with little additional effort to bring us down to 1.

r351085:
libc quad.h: one last _STANDALONE correction

r351088:
Revert r351078, r351085: stand/quad.h eviction

It did not go well; further examination is required...

MFC r348471: stand: zfs: Free bouncebuf on error path in vdev_read

r344226 inadvertently added this path in which we return from failure on an
lseek and do not free bouncebuf on the way out.

Partial revert r350338: stable/11 part

The stand/libsa changes were not supposed to ride along with this revision;
backing out the stable/11 part and re-applying the MFC.

Pointy hat: kevans

MFC r348328: bectl(8): Address Coverity complaints

CID 1400451: case 0 is missing a break/return and falling through to the
default case. waitpid(0, ...) makes little sense in the child, we likely
wanted to terminate immediately.

CID 1400453: size argument uses sizeof(char **) instead of sizeof(char *)
and is assigned to a char **; sizeof's match but "this isn't a portable
assumption".

CID: 1400451, 1400453

MFC r348569: [zfsboot] Fix boot env back compat (#190)

* Fix boot env back compat

zfsboot must try zfsloader before loader in order to remain compatible
with boot environments created prior to zfs functionality being rolled
into loader proper.

* Improve comments in zfsboot

Explain the significance of the load path order, and put the comment
about looping through the paths in the appropriate scope.

Approved by: re (gjb)

MFC r347244:

Move contrib/zlib to sys/contrib/zlib so that we can use it in kernel.
This is a prerequisite of unifying kernel zlib instances.

Submitted by: Yoshihiro Ota <ota at j.email.ne.jp>
Approved by: re (kib)

MFC r337716:
Add microcode update configuration to the default loader.conf.

Approved by: re (kib)

MFC r346968, r346973

r346968:
Update the manpage text to show the output generated by the first-stage
bootloader these days (x86 instead of i386).

r346973:
Add a paragraph that mentions gptboot having an interactive mode, and
direct the user to the boot(8) manpage, which provides the details on that.

MFC r337715, r337751, r337754, r337758, r337813, r338354, r338687,
r339124, r341821:
Add support for boot-time Intel microcode loading.

MFC r347142:

loader: validate sectorsize argument in disk_open()

The bug and patch is reported against 11.2, but it is good idea to have
the check in place for all versions.

PR: 236585
Submitted by: john@feith.com
Reported by: john@feith.com

MFC r343911, r344238-r344241, r344247, r344254-r344255, r344260, r344268,
r344335, r344839, r345066, r345330

r343911:
Allow reading the UEFI variable size

When loading bigger variables form UEFI it is necessary to know their
size beforehand, so that an appropriate amount of memory can be
allocated. The easiest way to do this is to try to read the variable
with buffer size equal 0, expecting EFI_BUFFER_TOO_SMALL error to be
returned. Allow such possible approach in efi_getenv routine.

Extracted from a bigger patch as suggested by imp.

r344238:
Restore loader(8)'s ability for lsdev to show partitions within a bsd slice.

I'm pretty sure this used to work at one time, perhaps long ago. It has
been failing recently because if you call disk_open() with dev->d_partition
set to -1 when d_slice refers to a bsd slice, it assumes you want it to
open the first partition within that slice. When you then pass that open
dev instance to ptable_open(), it tries to read the start of the 'a'
partition and decides there is no recognizable partition type there.

This restores the old functionality by resetting d_offset to the start
of the raw slice after disk_open() returns. For good measure, d_partition
is also set back to -1, although that doesn't currently affect anything.

I would have preferred to make disk_open() avoid such rude assumptions and
if you ask for partition -1 you get the raw slice. But the commit history
shows that someone already did that once (r239058), and had to revert it
(r239232), so I didn't even try to go down that road.

r344239:
Use a couple local variables to avoid repetitive long expressions that
cause line-wrapping.

r344240:
Make lsdev -v output line up in neat columns by using a fixed width for
the size field and a tab between the partition type and the size.

Changes this

disk devices:
disk0 (MMC)
disk0s1: DOS/Windows 49MB
disk0s2: FreeBSD 14GB
disk0s2a: FreeBSD UFS 14GB
disk0s2b: Unknown 2048KB
disk0s2d: FreeBSD UFS 2040KB

to this

disk devices:
disk0 (MMC)
disk0s1: DOS/Windows 49MB
disk0s2: FreeBSD 14GB
disk0s2a: FreeBSD UFS 14GB
disk0s2b: Unknown 2048KB
disk0s2d: FreeBSD UFS 2040KB

r344241:
Garbage collection no-longer-used constant.

r344247:
Make uboot_devdesc properly alias disk_devdesc, so that parsing the u-boot
loaderdev variable works correctly.

The uboot_devdesc struct is variously cast back and forth between
uboot_devdesc and disk_devdesc as pointers are handed off through various
opaque interfaces. uboot_devdesc attempted to mimic the layout of
disk_devdesc by having a devdesc struct, followed by a union of some
device-specific stuff that included a struct that contains the same fields
as a disk_devdesc. However, one of those fields inside the struct is 64-bit
which causes the entire union to be 64-bit aligned -- 32 bits of padding
is added between the struct devdesc and the union, so the whole mess ends
up NOT properly mimicking a disk_devdesc after all. (In disk_devdesc there
is also 32 bits of padding, but it shows up immediately before the d_offset
field, rather than before the whole collection of d_* fields.)

This fixes the problem by using an anonymous union to overlay the devdesc
field uboot network devices need with the disk_devdesc that uboot storage
devices need. This is a different solution than the one contributed with
the PR (so if anything goes wrong, the blame goes to me), but 95% of the
credit for this fix goes to Pawel Worach and Manuel Stuhn who analyzed the
problem and proposed a fix.

r344254:
Use DEV_TYP_NONE instead of -1 to indicate no device was specified.

DEV_TYP_NONE has a value of zero, which makes more sense since the device
type is a bunch of bits describing the device, crammed into an int.

r344255:
Fix more places to use DEV_TYP_NONE instead of -1 to indicate 'no device'.

r344260:
Allow the u-boot loaderdev env var to be formatted in the "usual" loader(8)
way: device<unit>[s|p]<slice><partition>. E.g., disk0s2a or disk3p12.
The code first tries to parse the variable in this format using the
standard disk_parsedev(). If that fails, it falls back to parsing the
legacy format that has been supported by ubldr for years.

In addition to 'disk', all the valid uboot device names can also be used:
mmc, sata, usb, ide, scsi. The 'disk' device serves as an alias for all
those types and will match the Nth storage-type device found (where N is
the unit number).

r344268:
loader: ptable_close() should check its argument

If the passed in table is NULL, just return.

r344335:
Fix the handling of legacy-format devices in the u-boot loaderdev variable.
When I added support for the standard loader(8) disk0s2a: type formats,
the parsing of legacy format was broken because it also contains a colon,
but it comes before the slice and partition. That would cause disk_parsedev()
to return success with the slice and partition set to wildcard values.

This change examines the string first, and if it contains spaces, dots, or
a colon at any position other than the end, it must be a legacy-format
string and we don't even try to use disk_parsedev() on it.

r344839:
Add retry loop around GetMemoryMap call to fix fragmentation bug

The call to BS->AllocatePages can cause the memory map to become framented,
causing BS->GetMemoryMap to return EFI_BUFFER_TOO_SMALL more than once. For
example this can happen on the MinnowBoard Turbot, causing the boot to stop
with an error. Avoid this by calling GetMemoryMap in a loop.

r345066:
stand: Improve some debugging experience

Some of these files using <FOO>_DEBUG defined a DEBUG() macro to serve as a
debug-printf. -DDEBUG is useful to enable some debugging output across
multiple ELF/common parts, so switch the DEBUG-as-printf macros over to
something more like DPRINTF that is more commonly used for this kind of
thing and less likely to conflict.

userboot/elf64_freebsd debugging also assumed %llx for uint64; use PRIx64
instead.

r345330:
loader: fix loading of kernels with . in path

The loader indended to search the kernel file name (only) for . but
instead searched the entire path, so paths like
"boot/test.elfv2/kernel" would not work.

PR: 233097

MFC r341101, r341231, r341276, r341329, r341433, r341780, r342054-r342055,
r342721, r342742, r342840, r343008, r343225

r341101:
powerpcspe: Don't crash the loader on ubldr with SPE instructions.

-msoft-float seems to be insufficient for disabling the SPE on powerpcspe.
Force it off with -mno-spe as well. This prevents a crash in ubldr on
powerpcspe.

r341231:
loader: command_bcache() should print unsigned values

All bcache counters are unsigned.

r341276:
When handling CMD_CRIT error set command_errmsg to NULL after we dump it out,
so that it does not result in error message printed twice.

OK load doodoo
can't find 'doodoo'
can't find 'doodoo'
OK

r341329:
loader.efi: fix EFI getchar() for multiple consoles

This fix is ported from illumos (issue #9970), the analysis and initial
implementation was done by John Levon.

See also: https://www.illumos.org/issues/9970

Currently, efi_cons_getchar() will wait for a key. While this seems to make
sense, the implementation of getchar() in common/console.c will loop across
getchar() for all consoles without doing ischar() first.

This means that if we've configured multiple consoles, we can't input into
the serial, as getchar() will be sat waiting for input only from efi_console.c

This patch does implement a bit more generic key buffer to support
translation of input keys, and we use generic efi_readkey() to reduce
duplication from calls from getchar() and poll().

r341433:
Move inclusion of src.opts.mk later.

src.opts.mk includes bsd.own.mk. This in turn defines CTFCONVERT_CMD
depending on the MK_CTF value. We then set MK_CTF to no, which has no
real effect. The solution is to set all the MK_foo values before
including src.opts.mk.

This should stop the cdboot binary from exploding in size for releases
built WITH_CTF=yes in src.conf.

r341780:
powerpc/ubldr: Teach powerpc's ubldr to boot 64-bit kernels

This is just a copy of powerpc/ofw's ppc64_elf_freebsd.c modified to fit
ubldr's boot format.

r342054:
Print an error message in efi_main.c if we can't allocate memory for the heap

With the default Qemu parameters, only 128MB RAM gets given to a VM. This causes
the loader to be unable to allocate the 64MB it needs for the heap. This change
makes the cause of the error more obvious.

r342055:
Cast error message in efi_main.c to CHAR16* to avoid build error

r342721:
loader.efi: update memmap command to recognize new attributes

Also move memory type to string translation to libefi for later use.

r342742:
loader.efi: efi variable rework and lsefi command added

This update does add diag and debug capabilities to interpret the efi
variables, configuration and protocols (lsefi).

The side effect is that we add/update bunch of related headers.

r342840:
Create MK_LOADER_VERBOSE and connect it to ELF_VERBOSE in the loader
code.

r343008:
Add Dell Chromebook to the list of devices with E820 extmem quirk enabled

Just like for Acer C270 chromebook the E820 extmem workaround is required for
FreeBSD to boot on Dell chromebook.

r343225:
Unbreak mip64 build after r328437

Add exit and getchar functions to beri/boot2 code. They are required by
panic_action functin introduced in r328437

PR: 18498, 204916

MFC r338262, r339334, r339796, r340240, r340857, r340917, r341007

r338262:
stand: fdt: Drop some write-only assignments/variables and leaked bits

Generally straightforward enough; a copy of argv[1] was being made in
command_fdt_internal, solely used for a comparison within the
handler-search, then promptly leaked.

r339334:
loader.efi: add poweroff command

Add poweroff command to make life a bit easier.

r339796:
Simplify the EFI delay() function by calling BS->Stall()

r340240:
loader: ptable_open() check for ptable_cd9660read result is wrong

The ptable_*read() functions return NULL on read errors (and partition table
closed as an side effect). The ptable_open must check the return value and
act properly.

r340857:
Nuke out buffer overflow safety marker code, it duplicates similar code in
the malloc()/free() as well as having potential of softening the handling
in case error is detected down to a mere warning as compared to hard panic
in free().

r340917:
Update pxeboot(8) manual page to reflect the next-server change in the ISC DHCP v3 server.

r341007:
Bump the date of pxeboot(8) manual page for r340917.

PR: 123484, 232483

MFC r337321, r337435, r337707, r337740, r337834, r337836, r337968

r337321:
Make it possible for init to execute any executable, not just sh(1)
scripts. This means one should be able to eg rewrite their /etc/rc
in Python.

r337435:
Move description of init_shell, init_script, and init_chroot kenv
tunables from loader(8) to init(8), since it's init that actually
uses them. Add .Xrs at their old place.

r337707:
Move around text in loader(8), in particular stuff related to ZFS,
to restore the usual section order.

r337740:
Add init_exec kenv(1) variable, to make init(8) execute a file
after opening the console, replacing init as PID 1.

From the user point of view, it makes it possible to run eg the
shell as PID 1, using 'set init_exec=/bin/sh' at the loader(8)
prompt.

r337834:
Add SECURITY section to loader(8).

r337836:
Improve formatting.

r337968:
Consistently use NULL to terminate the argv; no functional changes.

Relnotes: yes (init_exec kenv(1) variable)

MFC r339292: Fix a minor typo in loader.conf(5).

MFC r337871, r339970, r342151, r342161, r343123-r343124, r344226, r344234,
r344248, r344387

r337871:
pkgfs_init: Initialize pkg

new_package may not set *pp if it errors out, leaving pkg uninitialized.

r339970:
Remove unnecessary include from libstand.

r342151:
loader: zfs reader should not probe partitionless disks

First of all, normal setups can not boot such pools as the tools
do not support installing boot programs.

Secondly, for proper pool configuration detection, we need to checks all
four label copies on disk, 2 from front and 2 from the end of the disk,
but zfs label does not contain the size of the disk - so we depend on
firmware to report the correct disk size or use information from the
partition table.

Without partition table, we only can rely on firmware to report and support
disk IO properly.

There is a specific case: 8TB disks are reported by BIOS to have 4294967295
sectors (0x00000000ffffffff), the sectors reported by OS is 15628053168
(0x00000003a3812ab0), so the reported size is less than actual but is hitting
32-bit max. Unfortuantely the real limit must be even lower because probing
this disk in this system will wnd up with hung system.

UEFI boot of this system seems not to be affected.

r342161:
loader: zfs reader should not probe partitionless disks (UEFI case)

With r342151 I did fix the BIOS version of zfs_probe_dev() from accessing
the whole disk, but the fix was not complete - we actually did not check
if the device name was really for whole disk. Since UEFI version
is only calling the zfs_probe_dev() with partitions and not with whole
disk, the UEFI loader was not able to find the zfs pools.

This update does correct the issue by calling archsw.arch_getdev() to
translate the device name back to dev_desc, and we have whole disk when both
partition and slice values are -1.

r343123:
loader should ignore active multi_vdev_crash_dump feature on zpool

Since the loader zfs reader does not need to read the dump zvol, we can
just enable the feature.

illumos issue #9051 https://www.illumos.org/issues/9051

r343124:
libsa: add asprintf()

asprintf() is a nice tool for string processing.

r344226:
Fix memory corruption bug introduced in r325310

The bug occurred when a bounce buffer was used and the requested read
size was greater than the size of the bounce buffer. This commit also
rewrites the read logic so that it is easier to systematically verify
all alignment and size cases.

r344234:
It turns out r344226 narrowed the overrun bug but did not eliminate it entirely

This commit fixes a remaining output buffer overrun in the
single-sector case when there is a non-zero tail.

r344248:
cd9660: dirmatch fails to unmatch when name is prefix for directory record

Loader does fail to properly match the file name in directory record and
does open file based on prefix match.

For fix, we check the name lengths first.

r344387:
loader: really fix cd9660 dirmatch

The cd9660_open() does pass whole path to dirmatch() and we need to
compare only the current path component, not full path.

Additinally, skip over duplicate / (if any) and check if the last component
in the path was meant to be directory (having trailing /). If it is in fact
a file, error out.

MFC r341253, r341328, r342619, r342626, r342707, r342785, r342865

r341253:
The libstand's panic() appends its own '\n' to the message, so that users of the API
don't need to supply one.

r341328:
loader: create separate lists for fd, cd and hd, merge bioscd with biosdisk

Create unified block IO implementation in BIOS version, like it is done in UEFI
side. Implement fd, disk and cd device lists, this will split floppy devices
from disks and will allow us to have consistent, predictable device naming
(modulo BIOS issues).

r342619:
loader: create bio_alloc and bio_free for bios bounce buffer

We do have 16KB buffer space defined in pxe.c, move it to bio.c and implement
bio_alloc()/bio_free() interface to make it possible to use this space for
other BIOS calls (notably, from biosdisk.c).

r342626:
Add Copyright.

r342707:
i386_parsedev() needs to support fd devices

r342785:
With buggy int13 ah=15, we can mis-identify the floppy devices.

We have no option than trust INT13 ah=08 return code during the init phase.

r342865:
biospci_write_config args were backwards

biospci_write_config args swapped length and value to write. Some
hardware coped just fine, while other hardware had issues.

PR: 155441, 234460

MFC r339658, r339959, r340047, r340049, r340215

r339658:
loader: biosdisk interface should be able to cope with 4k sectors

The 4kn support in current bios specific biosdisk.c is broken, as the code
is only implementing the support for the 512B sector size.

This work is building the support for custom size sectors, we still do assume
the requested data to be multiple of 512B blocks and we only do address the
biosdisk.c interface here.

For reference, see also:
https://www.illumos.org/issues/8303
https://www.illumos.org/rb/r/547

As the GELI is moved above biosdisk "layer", the GELI should just work

r339959:
loader: issue edd probe before legacy ah=08 and detect no media

while probing for drives, use int13 extended info before standard one and
provide workaround for case we are not getting needed information in case
of floppy drive.

In case of INT13 errors, there are (at least) 3 error codes appearing in case
of missin media - 20h, 31h and 80h. Flag the no media and do not print an
error.

r340047:
loader: do not probe floppy devices for zfs

The subject is telling it all.

r340049:
loader: biosdisk should check if the media is present

The bd_print/bd_open/bd_strategy need to make sure the device does have
media, before getting into performing IO operations. Some systems can
hung if the device without a media is accessed.

r340215:
loader: always set media size from partition.

The disk access is validated by using partition table definitions, therefore
we have no need for if statements, just set the disk size.

Of course the partition table itself may be incorrect/inconsistent, but if
so, we are in trouble anyhow.

MFC i386 stand cleanup: r337353-r337354, r337356, r337872, r337878, r337881,
r337890-r337891, r338188

r337353:
loader: cstyle cleanup for biosdisk.c

Also switch u_int to uint32_t. Also replace "write" by "dowrite".
No functional changes intended.

r337354:
loader: 337353 did miss to rename 2 write instances

2 write instances got somehow missed.

r337356:
loader: bd_open() should cleanup from disk_open() error

Since bd_open() does early increment for reference counter and bcache
allocation, it also should undo those in case of the error.

Also remove unused variables rdev, g_err.

r337872:
libi386: remove BD_SUPPORT_FRAGS

BD_SUPPORT_FRAGS is preprocessor knob to allow partial reads in bioscd/biosdisk
level. However, we already have support for partial reads in bcache, and there
is no need to have duplication via preprocessor controls.

Note that bioscd/biosdisk interface is assumed to perform IO in 512B blocks,
so the only translation we have to do is 512 <-> native block size.

r337878:
libi386: remove bd_read() and bd_write() wrappers

Those wroappers are nice, but do not really add much value.

r337881:
libi386: use BD_RD and BR_WR constants

Use BD_RD and BD_WR instead of 0 and 1.

r337890:
libi386: small style updates in biosdisk

Use break instead of return in for loop, as done earlier. Insert and remove
some blank lines. No functional changes intended.

r337891:
libi386: bd_io_workaround() is to be called for reads only

bd_io() can perform either reads or writes, we only need bd_io_workaround()
for reads.

r338188:
loader: bios loader should allow to chain load a file

The current chain command does accept only device, allow also a file to be used,
such as /boot/pmbr or /boot/mbr (or stored third party MBR/VBR block).

Also fix file descriptor leak.

MFC r337271, r337317: stand: i386: sector calculation fixes

r337271:
Some drives report a geometry that is inconsisetent with the total
number of sectors reported through the BIOS. Cylinders * heads *
sectors may not necessarily be equal to the total number of sectors
reported through int13h function 48h.

An example of this is when a Mediasonic HD3-U2B PATA to USB enclosure
with a 80 GB disk is attached. Loader hangs at line 506 of
stand/i386/libi386/biosdisk.c while attempting to read sectors beyond
the end of the disk, sector 156906855. I discovered that the Mediasonic
enclosure was reporting the disk with 9767 cylinders, 255 heads, 63
sectors/track. That's 156906855 sectors. However camcontrol and
Windows 10 both report report the disk having 156301488 sectors, not
the calculated value. At line 280 biosdisk.c sets the sectors to the
higher of either bd->bd_sectors or the total calculated at line 276
(156906855) instead of the lower and correct value of 156301488 reported
by int 13h 48h.

This was tested on all three of my Mediasonic HD3-U2B PATA to USB
enclosures.

Instead of using the higher of bd_sectors (returned by int13h) or the
calculated value, this patch uses the lower and safer of the values.

r337317:
In r337271, we limited the sector number to the lower of calculated
number and CHS based number. However, on some systems, BIOS would
report 0 in CHS fields, making the system to think there is 0 sectors.

Add a check before comparing the calculated total with bd_sectors.

MFC r336424-r336425: loader command typos

r336424:
Fix typo in the command summary.

Of course, I can't get the command to work, but it's a start...

r336425:
More typos

MFC r333662: Clarify that boot_mute / boot -m mutes kernel console only

Perhaps RB_MUTE could mute user startup (rc) output as well, but right
now it mutes only kernel console output, so make the documentation match
reality.

PR: 228193

MFC r346132: stand: refactor overlay loading a little bit

It was pointed out that manually loading a .dtb to be used rather than
relying on platform-specific method for loading .dtb will result in overlays
not being applied. This was true because overlay loading was hacked into
fdt_platform_load_dtb, rather than done in a way more independent from how
the .dtb is loaded.

Instead, push overlay loading (for now) out into an
fdt_platform_load_overlays. This method easily allows ubldr to pull in any
fdt_overlays specified in the ub env, and omits overlay-checking on
platforms where they're not tested and/or not desired (e.g. powerpc). If we
eventually stop caring about fdt_overlays from ubenv (if we ever cared),
this method should get chopped out in favor of just calling
fdt_load_dtb_overlays() directly.

MFC r344161: stand: dev_net: correct net_open's interpretation of params

net_open previously casted the first vararg to a char * and this was
half-OK: at first, it is passed to netif_open, which would cast it back to
the struct devdesc * that it really is and use it properly. It is then
strdup()d and used as the netdev_name, which is objectively wrong.

Correct it so that the first vararg is properly casted to a struct devdesc *
and the netdev_name gets set properly to make it more clear at a glance that
it's not doing something horribly wrong.

MFC r344243, r345517-r345518: lualoader: More intelligent screen clearing

r344243:
lualoader: only clear the screen before first password prompt

This was previously an unconditional screen clear, regardless of whether or
not we would be prompting for any passwords. This is pointless, given that
the screen clear is only there to put our screen into a consistent state
before we draw the prompts and do cursor manipulation.

This is also the only screen clear besides that to draw the menu. One can
now see early pre-loader and loader output with the menu disabled, which may
be useful for diagnostics.

r345517:
lualoader: Clear the screen before prompting for password

Assuming that the autoboot sequence was interrupted, we've done enough
cursor manipulation that the prompt for the password will be sufficiently
obscured a couple of lines up. Clear the screen and reset the cursor
position here, too.

r345518:
lualoader: Fix up some luacheck concerns

- Garbage collect an unused (removed because it was useless) constant
- Don't bother with vararg notation if args will not be used

Direct commit for some clarification/correction

The notice in UPDATING stated the wrong direction for the link.

Additionally, it's decidedly not a bad idea to throw in a comment mandating
that the ordering of LOADER_DEFAULT_INTERP bits remains the same in this
branch. This makes it even more obvious from diff review (against head) that
this difference is intentional and creates merge conflicts if someone tries
to apply the original commit manually. I suspect this won't be a problem
given the set of people that maintain loader bits in stable/11, but it does
offer a peace of mind.

Reported by: rgrimes

Direct commit to catch pc98 up to recent changes

I have no means of actually testing the new features on pc98, so simply mark
them as BROKEN_OPTIONS on pc98 and interested parties can test/report back
as to the functionality. This is done out of an abundance of caution.

pc98 also needed catching up on the inflate.c -> subr_inflate.c rename.

MFC Loader Interpreter Coexistance: r337806, r337868, r337914, r337927,
r337989, r338052, r338064, r338407, r338418, r338474

r337806:
Create a loader for each interpreter for x86 BIOS and all EFI

Create loader_{4th,lua,simp}{,.efi}. All of these are installed by
default. Create LOADER_DEFAULT_INTERP to specify the default
interpreter when no other is specified. LOADER_INTERP is the current
interpreter language building. Turn building of lua on by default to
match 4th. simploader is a simplified loader build w/o any interpreter
language (but with a simple loader). This is the historic behavir you
got with WITHOUT_FORTH. Make a hard link to the default loader. This
has to be a hard link rather than the more desirable soft link because
older zfsboot blocks don't support symlinks.

r337868:
stand: Use -Oz/-Os for all loader/stand builds.

While we're not super size constrained, the x86 BIOS /boot/loader has
to be less than about 520k-530k to be reliable. The LUA loader is at
this size today. -Oz saves 15-20% on the size, keeping us safely small
enough (comparable to where we were with the 4th loader). This will
also help with sjg's work on bringing in bearssl, though we may again
be looking for space in the LUA loader.

Size table for clang 6.0.0:
default -O1 -Os -Oz
4th 442368 417792 389120 376832
lua 524288 479232 446464 430080

r337914:
Install links for loader.efi.

r337927:
Add ashldi3 and ashrdi3 to mips.

Now that we're using -Os, mips needs these routines.

r337989:
Copy the boot loader from the new location for the co-existing
loaders.

r338052:
libsa: Add lshrdi3.c for powerpc* and mips

r338064:
Turn back the clock just a little: make userboot.so always be 4th

Turns out there was a hidden dependency we hasn't counted upon. The
host load /boot/userboot.so to boot the VMs it runs. This means that
the change to lua meant suddently that nobody could run their older
VMs because LUA wasn't in 10.0, last month's HardenedBSD, 11.2 or
whatever. Even more than for the /boot/loader* binaries, we need a
good coexistance strategy for this. While that's being designed and
implemented, drop back to always 4th for userboot.so. This will fail
safe in all but the most extreme environments (but lua-only hacks
to .lua files won't be processes in VMs until we fix it).

r338407:
lualoader: Print error messages from command failures at the prompt

Previously lualoader would remain silent, rather than printing
command_errmsg or noting that a command had failed or was not found.

r338418:
userboot: handle guest interpreter mismatches more intelligently

The switch to lualoader creates a problem with userboot: the host is
inclined to build userboot with Lua, but the host userboot's interpreter
must match what's available on the guest. For almost all FreeBSD guests in
the wild, Lua is not yet available and a Lua-based userboot will fail.

This revision updates userboot protocol to version 5, which adds a
swap_interpreter callback to request a different interpreter, and tries to
determine the proper interpreter to be used based on how the guest
/boot/loader is compiled. This is still a bit of a guess, but it's likely
the best possible guess we can make in order to get it right. The
interpreter is now embedded in the resulting executable, so we can open
/boot/loader on the guest and hunt that down to derive the interpreter it
was built with.

Using -l with bhyveload will not allow an intepreter swap, even if the
loader specified happens to be a userboot with the wrong interpreter. We'll
simply complain about the mismatch and bail out.

For legacy guests without the interpreter marker, we assume they're 4th.
For new guests with the interpreter marker, we'll read it and swap over
to the proper interpreter if it doesn't match what the userboot we're using
was compiled with.

Both flavors of userboot are installed by default, userboot_4th.so and
userboot_lua.so. This fixes the build WITHOUT_FORTH as a coincidence, which
was broken by userboot being forced to 4th.

r338474:
Be a little conservative about when to force size optimizations.

Reports have come in that there's issue with powerpc and sparc64 since
we've switched to using -Oz / -Os. We don't strictly need them for
!x86, so be conservative about when we enable them.

MFC r336837: Be more conservative about setting hw.uart.console

Note when we've found a 8250 PNP node. Only try to set hw.uart.console
if we see one (otherwise ignore serial hints). The 8250 is the only
one known to have I/O ports, so limit the guessing to when we've
positively seen one. And limit this to x86 since that's the only
platform where we have I/O ports. Otherwise, we'd set the serial port
to something crazy for the platform and fall off the cliff early in
boot.

MFC r336431: stand: Fix UEFI console scrolling

Remove bogus attempt to simulate scrolling. It's not needed and messes
up serial output. Setting the cursor position after every character is
inefficient, and causes all lines to be over-printed in the serial
console for the boot loader. Allow the terminal to do the emulation.

This isn't completely perfect when the size of the terminal attached
to the serial port isn't the same as 80x25 to match the viedoe console
(or whatever the video console is). While imperfect still, these
changes make it much better.

This makes the serial port useful with UEFI.

MFC r337131, r337231: stand: efipart fixes

r337131:
loader.efi: efipart_inithandles() failure should not prevent devsw initialization

The efipart_inithandles() can return error code when there are no block devices,
however, we still should walk and initialize devsw.

r337231:
loader.efi: clean up misleading noise from missing block devices

If there are no block devices, there is no need to printout
error (ENOENT).

In case of netboot, our image path has no block device, no need to make
noise about it.

MFC r338337: Fix lualoader on arm64

Lua has a few places where it allocates a large buffer on the stack. This
is normally fine, except there are a few places where there can be multiple
frames with this buffer. This can cause a stack overflow on some arm64 SoCs.

Fix this by allocating our own stack in loader.efi large enough for these
objects. The required size has been found by tracing how the stack pointer
changes in a virtual machine and found to be no larger than 50kB. A
larger stack is allocated to reduce the likelihood of overflow from future
changes.

MFC various libsa fixes: r337037-r337039, r337065, r337412-r337413, r337874,
r338535, r338540, r339651, r339992-r339993, r340026

r337037:
libsa: pointer differs in signedness

A small cleanup, fix the argument type and while there, replace (char *)0 with
NULL.

r337038:
libsa: bootp is using pointers with different sign

Just change bp_file to char and same for variable s.

r337039:
libsa: assignment to char * from u_char *

Cast to char * instead of u_char *

r337065:
libsa: dereferencing type-punned pointer in cd9660

The warning is given by gcc build, but it is good to fix anyhow.
use bcopy instead of direct assignment.

r337412:
libsa: dos_checksum() should take unsigned chars

Fix pointers to integers with different sign issue.

r337413:
libsa: gzipfs.c converts pointers to integer types with different sign

Signed versus unsigned char.

r337874:
libsa: zfs_probe() needs to set spa to NULL

Silence the warning about possibly uninitialized use of spa.

r338535:
libsa: memory leak in tftp_open()

tftpfile is allocated just above and needs to be freed.

r338540:
libsa: validate tftp_makereq() after we did reset the read

The name check referred in the comment is not the only possible error source,
we need to validate the result.

r339651:
libsa: re-send ACK for older data packets in tftp

In current tftp code we drop out-of-order packets; however, we should play
nice and re-send ACK for older data packets we are receiving. This will
hopefully stop server repeating those packets we already have received.
Note we do not answer duplicates from "previous" session (that is, session
with different port number), those will eventually time out.

r339992:
libsa: tftp should not read past file end

When we have the file size via tsize option, use it to make sure we
will not attempt to read past file end.

r339993:
libsa: tftp should use calloc

instead of malloc() memset(), use calloc().

r340026:
libsa: cstyle cleanup tftp.c

No functinal changes intended.

MFC r336651-r336655: stand: libefi: various boot protocol aux impl.

r336651:
Implement efi_devpath_to_media_path

Takes a generic device path as its input. Scans through it to find the
first media_path node in it and returns a pointer to it. If none is
found, NULL is returned.

r336652:
Store the number of handles we get back in efipart_nhandles rather
than the number of bytes. Don't divide by the element size every time
we have to iterate. Eliminate now-unused variables.

r336653:
Implement efi_devpath_match_node

Returns true if the first node pointed to by devpath1 is identical to
the first node pointed to by devpath2, with care taken to not read
past the end of the valid parts of either devpath1 or
devpath2. Otherwise, returns false.

r336654:
Implement efi_devpath_length

Return the total length, in bytes, of the device path (including the
terminating node at the end).

r336655:
Implement efiblk_get_pdinfo_by_device_path

Lookup a block device by it's device path. We use a 'loose' lookup
whereby we scan forward to the first Media Path portion of the device
path, then look at all our handles for one whose first Media Path
matches. This will also work if the device path pointed to has a
following file path (or paths) as that's ignored. It assumes that
there's only one media path node that describes the entire device,
which is true as of the latest UEFI spec (2.7 Errata A) as far as I've
been able to determine.

MFC r336535: Hoist EFI_TARGET and SOURCE_DATE_EPOCH up into efi/Makefile.inc

MFC r336463: If rootdev is set, use it as currdev.

Setting rootdev in the enviornment should specify things
completely. If it is set, then have it override everything else.

MFC r336264, r336270-r336272, r336464, r336627: UEFI Console Improvements

r336264:
Define ADR subtype of ACPI type for a device path.

r336270:
uefi stand: Guess the console better

For server machines, ComOut is set to the set of devices that the efi
console suppots. Parse it to see if we have serial, video or both.
Make that take precidence over the command line args. boot1.efi parses
them, but loader.efi doesn't. It's not clear where to read boot.conf
from, so we don't do that. The command line args can still be set via
efibootmgr, which is more inline with the UEFI boot manager to replace
that. These args are typically used only to set serial vs video and
the com speed line. We can infer that from ComOut, so do so.
Remember the com speed and hw.uart.console to match.

r336271:
Add reporting of whether or not a keyboard is detected. In addition,
note that r336270's commit message was slightly incorrect. It changed
the default setting of the console to honor the ConOut
variable. Overrides via the command line are still possible, and we
use the devices in ConOut to set the proper console. If, for example,
serial cosnole is specified, we'll set console to "efi" if ConOut has
a serial port list and to either "efi comconsole" or "comconsole efi"
if not depending on whether -D or -D -h was specified.

r336272:
Minor adjustments:

o Fix the parsing of the device path. a last minute change terminated
it too soon.
o Kill setting LINES. We don't need to do it, and even if we did hard
coding it to 24 is wrong.
o Now that the console is working again for the loader, adjust the
printfs to be more in line with other platforms.

r336464:
If the console is already set, don't override it.

If console=X is specified on the command line, it's effectively
overridden by the current code. It shouldn't do that.

r336627:
Fix the attempt to see if we're overriding the console in the command
line args. I had thought console would be NULL, but it's efi. Set it
to efi (as a clue) before we initialize the console, then test it to
see if it changed on the command line to do the automatic
override. This gets my serial console back.

RelNotes: yes

MFC GELI Loader Improvements: r336252, r336254, r336256, r336354,
r336532-r336534, r336537, r336626, r337326, r337349, r341071, r341160,
r341420, r341473, r341651, r342793

Note that this MFC contains some seemingly unrelated zfsloader bits -- this
was needed in order to pull in some later fixes for GELI hand-off w/ ZFS
bits included.

r336252:
Extend loader(8) geli support to all architectures and all disk-like devices.

This moves the bulk of the geli support from lib386/biosdisk.c into a new
geli/gelidev.c which implements a devsw-type device whose dv_strategy()
function handles geli decryption. Support for all arches comes from moving
the taste-and-attach code to the devopen() function in libsa.

After opening any DEVT_DISK device, devopen() calls the new function
geli_probe_and_attach(), which will "attach" the geli code to the open_file
struct by creating a geli_devdesc instance to replace the disk_devdesc
instance in the open_file. That routes all IO for the device through the
geli code.

A new public geli_add_key() function is added, to allow arch/vendor-specific
code to add keys obtained from custom hardware or other sources.

With these changes, geli support will be compiled into all variations of
loader(8) on all arches because the default is WITH_LOADER_GELI.

r336254:
Use if rather than case for a simple boolean. gcc thinks blks is
undefined sometimes with the case, but enc is always 0 or 1, so
and if / else is better anyway.

r336256:
Fix glitched indentation (and rewrap as needed due to deeper indent).
No functional changes.

r336354:
zfsboot: fix build with WITHOUT_LOADER_GELI

r336532:
Collapse zfsloader functionality back down into loader.

We no longer really need a separate zfsloader. It was useful when we
were first supporting ZFS and had limited ability to properly boot off
of ZFS without the special boot loader. Now that the boot loader has
matured, go the way loader.efi pioneered and just build one
binary. Change the name of the loader to load in the secondary boot
blocks to be just /boot/loader. Provide a symbolic link from zfsloader
to loader so people who have not upgraded their boot blocks are not
affected. This has the happy benefit of making coexistence easier as
well (fewer binaries in the matrix).

r336533:
Eliminate zfsloader man page.

Remove all cross references to zfsloader.8 and /boot/zfsloader.
Move ZFS specific info into loader.8.

r336534:
NM and OBJCOPY are already defined for all builds. There's no need to
conditionally define them here.

r336537:
Mention zfsloader being folded into loader in UPDATING.

r336626:
Older zfs boot blocks don't support symlinks. install the link to
zfsloader as a hard link. While newer ones do, the whole point of the
link was to transition to the new world order smoothly. A hard link is
less flexible, but it works and will result in fewer bumps. Adjust
UPDATING entry to match.

r337326:
loader: biosdisk.c has leftover geli header.

A small cleanup, remove unneeded #include.

r337349:
zfsboot: Fix startup crash

On a FreeNAS mini XL, with geli encrypted drives the loader crashed in
geli_read().

When we iterate over the list of disks and allocate the zfsdsk structures we
don’t zero out the gdev pointer. In one case that resulted in geli_read()
(called on the bogus pointer) dividing by zero.

Use calloc() to ensure the zfsdsk structure is always zeroed, so the pointer is
initialised to NULL. As a side benefit it gets rid of one #ifdef
LOADER_GELI_SUPPORT.

r341071:
Restore the ability to override the disk unit/partition at the boot: prompt
in gptboot.

When arch-independent geli support was added, a new static 'gdsk' struct
was added, but there was still a static 'dsk' struct, and when you typed
in an alternate disk/partition, the string was parsed into that struct,
which was then never used for anything. Now the string gets parsed into
gdsk.dsk, the struct that's actually used.

r341160:
Add comments describing the bootargs handoff between loader(8) and gptboot
or zfsboot, when loader(8) is the BTX loader. No functional changes.

r341420:
Eliminate duplicated code and struct member definitions in the handoff
of args data between gptboot/zfsboot and loader(8).

Despite what seems like a lot of changes here, there are no actual
changes in behavior, or in the data layout in the structures involved.
This is just eliminating identical code pasted into multiple locations.

In detail, the changes are...

- Move struct zfs_boot_args definition from libsa/zfs/libzfs.h to
i386/common/bootargs.h because it is specific to x86 booting and the
handoff between zfsboot and loader, and has no relation to the zfs
library code in general.

- The geli_boot_args and zfs_boot_args structs both contain an identical
set of member variables containing geli information. Extract this out
to a new geli_boot_data struct, and embed it in the arg-passing structs.

- Provide new routines geli_import_boot_data() and geli_export_boot_data()
that can be shared between gptboot, zfsboot, and loader instead of
pasting identical code into several different .c files.

- Remove some checks for a NULL pointer that can never be true because the
pointer being tested was set using pointer math (kargs + 1) and that can
never result in NULL in this code.

r341473:
Fix args cross-threading between gptboot(8) and loader(8) with zfs support.

When loader(8) is built with zfs support enabled, it assumes that any extarg
data present is a zfs_boot_args struct, but if the first-stage loader was
gptboot(8) the extarg data is actually a geli_boot_args struct. Luckily,
zfsboot(8) and gptzfsboot(8) have always passed KARGS_FLAGS_ZFS along with
KARGS_FLAGS_EXTARG, so we can use KARGS_FLAGS_ZFS to decide whether the
extarg data is a zfs_boot_args struct.

To avoid similar problems in the future, gptboot(8) now passes a new
KARGS_FLAGS_GELI to indicate that extarg data is geli_boot_args. In
loader(8), if the neither KARGS_FLAGS_ZFS nor KARGS_FLAGS_GELI is set but
extarg data is present (which will be the case for gptboot compiled before
this change), we now check for the known size of the geli_boot_args struct
passed by the older versions of gptboot as a way of confirming what type of
extarg data is present.

In a semi-related tidying up, since loader's main() has already decided
what type of extarg data is present and set the global 'zargs' var
accordingly, don't repeat the check in extract_currdev, just check whether
zargs is NULL or not.

r341651:
Don't reference zfs-specific variables if LOADER_ZFS_SUPPORT is undefined
because the variables will be undefined too.

r342793:
MK_ZFS -> {MK_ZFS|MK_LOADER_ZFS}, this is so we can diable userland / kernel
ZFS but keep the boot-loaders when using ZoL port.

Relnotes: yes (GELI support extended)
Relnotes: yes (zfsloader has been collapsed into loader and may be
removed after boot blocks have been updated)

MFC r316312, r332361, r333438-r333439, r339804: GELI dry-run

r316312:
sys/geom/eli: Switch bzero() to explicit_bzero() for sensitive data

In GELI, anywhere we are zeroing out possibly sensitive data, like
the metadata struct, the metadata sector (both contain the encrypted
master key), the user key, or the master key, use explicit_bzero.

Didn't touch the bzero() used to initialize structs.

r332361:
Introduce dry run option for attaching the device.
This will allow us to verify if passphrase and key is valid without
decrypting whole device.

r333438:
Change option dry-run from 'n' to 'C' in geli attach command.

'n' is used in other commands to define the key index.
We should be consistent with that.
'C' option is used by patch(1) to perform dryrun so lets use that.

r333439:
Introduce the 'n' flag for the geli attach command.

If the 'n' flag is provided the provided key number will be used to
decrypt device. This can be used combined with dryrun to verify if the key
is set correctly. This can be also used to determine which key slot we want to
change on already attached device.

r339804:
Restore backward compatibility for "attach" verb.

In r332361 and r333439, two new parameters were added to geli attach
verb using gctl_get_paraml, which requires the value to be present.
This would prevent old geli(8) binary from attaching geli(4) device
as they have no knowledge about the new parameters.

Restore backward compatibility by treating the absense of these two
values as seeing the default value supplied by userland.

MFC r306681, r336249-r336250, r336261: remove inflate conflict

r306681:
ANSIfy inflate.c

Reviewed by: kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8143

r336249:
Fix sparc64 builds

gcc is complaining about struct infate being defined in a parameter
list. It's inclear how long this has been broken, but the fix is
simple enough.

r336250:
There's two files in the sys tree named inflate.c, in addition
to it being a common name elsewhere. Rename the old kzip one
to subr_inflate.c.

This actually fixes the build issues on sparc64 that my inclusion of
.PATH ${SYSDIR}/kern created in r336244, so also revert the broken
workaround I committed in r336249.

This slipped passed me because apparently, I never did a clean build.

r336261:
Catch up to the inflate renaming.

MFC r336244, r336246-r336247: Standardize boot arg parsing

r336244:
Create helper functions for parsing boot args.

boot_parse_arg to parse a single arg
boot_parse_cmdline to parse a command line string
boot_parse_args to parse all the args in a vector
boot_howto_to_env Convert howto bits to env vars
boot_env_to_howto Return howto mask mased on what's set in the environment.

All these routines return an int that's the bitmask of the args
translated to RB_* flags. As a special case, the 'S' flag sets the
comconsole_speed env var. Any arg that looks like a=b will set the env
key 'a' to value 'b'. If =b is omitted, 'a' is set to '1'. This
should help us reduce the number of redundant copies of these routines
in the tree. It should also give a more uniform experience between
platforms.

Also, invent a new flag RB_PROBE that's set when 'P' is parsed. On
x86 + BIOS, this means 'probe for the keyboard, and if it's not there
set both RB_MULTIPLE and RB_SERIAL (which means show the output on
both video and serial consoles, but make serial primary). Others it
may be some similar concept of probing, but it's loader dependent
what, exactly, it means.

These routines are suitable for /boot/loader and/or the kernel,
though they may not be suitable for the tightly hand-rolled-for-space
environments like boot2.

r336246:
Eliminate boot loader copies of boot arg parsing.

Eliminate 4 of the copies of the arg parsing in /boot/laoder
by using boot_parse_cmdline.

r336247:
Transition to boot_env_to_howto and boot_howto_to_env in the boot
loader.

MFC r336084-r336085, r336118: Move zfs impl into libsa

r336084:
Move ZFS files into libsa

Move the libzfs stuff into libsa. There's no need for it to be a
separate library. The separate library adds to the issues of build
ordering that we see from time to time. Move the filesystem support
into libsa, like all the other filesystem support rather than making
zfs the odd-duck out.

Discussed with: allanjude@

r336085:
Stop using ../zfs/libzfs.h but instead use libzfs.h.

While ../zfs/libzfs.h mostly works, there are a few situations where
it does not. Eliminate the problem by using plain libzfs.h, like we do
for ufs support. This fixes the weird cases, and is easier to
understand. It also follows the general style convetion of avoiding
../ in #includes.

r336118:
Remove dirs that git svn should have removed but didn't.

MFC r304850, r305480, r324550-r324551, r324655, r324684: correct mis-merge

Some of these commits were improperly MFC'd in the sys/boot => stand
mega-MFC, others were simply missed. Correct that mistake now by manually
merging the few that were missed and record-only merge on the others.

r304850:
Unused variables and cstyle fix for loader dosfs

r305480:
Renumber the advertising clause.

r324550:
Add $FreeBSD$ to ancient sources that it's missing from.

r324551:
Move lib/libstand to sys/boot/libsa

Move the sources to sys/boot. Make adjustments related to the
move. Kill LIBSTAND_SRC since it's no longer needed.

r324655:
Remove the libstand directory which is now empty.

r324684:
Remove lib/libstand again, accidentally readded in r324683

MFC r335869, r335933: stand: uboot behavioral fixes

r335869:
stand: uboot: Do not panic if we can't find a boot device

It is really anoying to panic when there is no boot device as you
cannot see the availables ones.

r335933:
loader: fdt: Try to load every possible DTB from u-boot

U-Boot setup a few variables :

- fdt_addr which is the board static dtb (most of the time loaded before
u-boot or coming from some hardware like a ROM)
- fdt_addr_r which is a location in RAM that holds the DTB loaded by
u-boot or before u-boot

In the case of u-boot + rpi firmware the DTB is loaded in RAM but the location
still end up in the fdt_addr variable and the fdt_addr_r variable exist.

Change the behavior so we test that a DTB exists for every possible variable :

- fdt_addr_r is checked first as if u-boot needed to modify it the
correct DTB will live there.
- fdt_addr is checked second as if we run on a hardware with DTB in ROM
it means that we what/need to run that
- fdtaddr looks like a FreeBSD-ism but since I'm not sure leave it.

MFC r335773: Revert preference to be an int.

While in base we use it as a boolean (of the wrong spelling), there's
at least one out of tree user that needs it to be int since priorirty
is a small int, not a 0/1. In deference to the time it's wasted me and
my team, push this up into FreeBSD for whatever short life boot1 may
have in the tree.

MFC r335321, r335336: stand: move libgeliboot into libsa

r335321:
stand: move libgeliboot into libsa.

Reduce by 1 the number of crazy libraries we need in stand by moving
geli into libsa (where architecturally it belonged all along). This
just moves things around without any code changes.

r335336:
Remove now-empty geli directory

MFC r335337: Remove stray debugging line.

MFC r335228-r335231, r335272: stand: efiloader fixes

r335228:
Migrate has_keyboard to bool.

r335229:
Provide a more direct interface to tell ZFS what the preferred handle
is. We tell the ZFS code now, and it checks rather than having a
callback to do the checks.

This will allow us to have a more graceful fallback code. In the
future, it's anticipated that we may fallback to a more global search
(or implement a command to do so) when reqeusted by the user, or we
detect a violation of the UEFI Boot Manager protocol severe enough to
warrant this backstop. For now, it just allows us to get rid of img as
a global.

r335230:
Move arg parsing into its own routine for possible later reuse.

r335231:
There's no need to walk through the tables looking for the smbios
table if we're just going to ignore it on arm, so expand, slightly,
the reach of the ifdef. Move the buffer to the inner block so we
don't have a separate #ifdef far away from these lines.

The issue on arm is that smbios_detect does unaligned accesses, which
in the u-boot implementing EFI context causes a crash.

r335272:
Many netboot scenarios don't have /boot/defaults/loader.conf. As
a fallback, also check /boot/kernel/kernel existing as well, since
that's the fallback behavior of the loader.

MFC r334412: userboot: Fix bad type-punning

Pass a struct devdesc to the format commands. Use proper type rather
than doing weird type-punning that happened to work because the size
was right. We copied a zfs devdesc into a disk_devdesc and then after
passing through a NULL pointer reinterpreted it as a
zfs_devdesc. Instead, pass the base devdesc around and 'upcase' when
we know the types are right.

This has the happy side effect of fixing a gcc warning about bad
type punning.

MFC r335245-r335248, r335254, r335276, r335298, r335398, r335868, r335883

r335245:
Correct logic error in biosdisk.c:bd_realstrategy()

The wrong condition is used when evaluating the return of disk_ioctl()
This results in reaching the 'We should not get here' branch in most casts

r335246:
biosdisk.c remove redundant variable

`rdev` and `disk` serve the same purpose, read the partition table without
the `d_offset` or `d_slice` set, so the read is relative to the start of
the disk. Reuse the already initialized `disk` instead of making another
copy later.

r335247:
biosdisk.c: add missing \n to error message

r335248:
biosdisk.c: fix type in debug printf

r335254:
Avoid reading past the end of the disk in zfsboot.c and biosdisk.c

The GELI boot code rounds reads up to 4k, since the encrypted sectors are
4k, and must be decrypted as a unit. With oddball sized disks (almost
always virtual), this can lead to reading past the end of the disk.

r335276:
gptboot, zfsboot, gptzfsboot: Enable the video and serial consoles early

Normally the serial console is not enabled until /boot.config is read and
we know how the serial console should be configured. Initialize the
consoles early in 'dual' mode (serial & keyboard) with a default serial
rate of 115200. Then serial is re-initialized once the disk is decrypted
and the /boot.config file can be read.

This allows the GELIBoot passphrase to be provided via the serial console.

r335298:
stand/common/disk.c: Update debug printf

This was missed in r330809 because it is compiled out by default

r335398:
Revert r335276

This was causing issues for people booting.
I will likely bring this back as an optional feature, similar to
boot0sio, like gptboot-serial or something.

r335868:
stand/common/disk.c: Read partition table relative to the start of the disk

If a disk is of an oddball size, like the 200mb + 512b used in rootgen.sh,
when disk_open() is called on a GELI encrypted partition, attempts to read
the partition table fail, as they pass through the decryption process which
turns the already plaintext data into jibberish.

When reading the partition table, always pass a slice and partition setting
of -1, and an offset of 0. Setting the slice to -1 prevents a false
positive when checking the slice against the cache of GELI encrypted
slices.

r335883:
stand/common/disk.c: dev->d_offset still needs to be set to 0

With r335868, I thought this was no longer necessary. I was wrong.

MFC r335125-r335127, r335148, r335224

r335125:
libsa: cstyle cleanup for open/close/read/write sources

r335126:
libsa: f_rabuf could be NULL

It is possible that we will not get RA buffer from open(), therefore
we must validate RA.

r335127:
libsa: remobe unneeded check before free()

free() is checking for NULL, remove duplicate check.

r335148:
libsa: open() should use NULL instead of typecasted 0

r335224:
Remove empty directory

MFC r335117: stand bootprog_info consolidation

bootprog_info is generated in vers.c. Move it's definition to
bootstrap.h and remove all the redundant copies.

MFC r335069-r335071, r335110, r335116: stand housecleaning

r335069:
Remove fail: label. It's unused.

r335070:
Remove unused variables.

[kevans@: removed ufs.c modifications; those depend on ufs rewrite bits]

r335071:
Initialize variables we later free so they aren't used
uninitialized in the error path.
Remove unused variables.

r335110:
Remove redundant defs.mk includes. They aren't needed.

r335116:
Use bool for vargood, since it's a boolean.

MFC r334868: Add st_mtime, st_ino and st_dev for ufs_stat

MFC r332557-r332558, r332560, r332565: loader command cleanup-lite

r332557:
loader: make sure we use snprintf() in commands.c

Safeguard against memory corruptions.

r332558:
loader: command_errmsg should be const

Use const char * for command_errmsg.

r332560:
loader: make sure we do not return garbage from help_getnext

Since we do free subtopic and desc in help_getnext(), we need to set them also
NULL, so we make sure we dont get double free().

r332565:
loader: cstyle cleanup of command.c

just clean it up. no functional changes intended.

MFC r332555: loader: provide values in help_getnext()

With r328289 we attempt to make sure we free the resources allocated in
help_getnext(), however, it is possible that we get no resources allocated
and help_getnext() will return early.

Make sure we have pointers set to NULL early in help_getnext().

MFC r330757: stand/interp_lua: correct errorr => error

MFC r332170, r332173: light cd9660 cleanup

r332170:
libsa: cd9660: warning: 'lenskip' may be used uninitialized in this function

We better provide value for lenskip in both instances.

r332173:
libsa: name is not used in dirmatch()

Seems like variable 'name' is leftover.

MFC r329727-r329728: lualoader ZFS BE Support

r329727:
stand/zfs: Add all bootenvs to environment

For the benefit of lualoader, add all bootenvs to environment when
init_zfs_bootenv is invoked. All of the boot environment logic can then be
implemented in pure lua, rather than going back and forth with C to
implement paging.

This stores all boot environments in bootenvs[idx] and the final count of
bootenvs in bootenvs_count.

While here, make a copy of currdev for init_zfs_bootenv since it will be
modifying it and the caller may not necessarily want that. Some of the logic
was shifted around so that the 'currdev' pointer remains at the beginning of
the string and 'beroot' is moved around as needed to modify it or ultimately
store it in zfs_be_root.

The original zfs_bootenv that this was copied from will be able to go away
only if/when forth eventually goes away.

r329728:
stand/zfs: Unbreak build, 'truct' ~= 'struct'

MFC lualoader: r326353, r328440, r328443, r329166-r329167, r329274, r329329,
r329349-r329352, r329355-r329359, r329366-r329369, r329386-r329387, r329393,
r329413-r329415, r329417, r329424-r329436, r329457, r329473-r329474,
r329496-r329501, r329503-r329504, r329543, r329547-r329551, r329576-r329578,
r329580, r329583, r329585-r329590, r329592-r329596, r329609-r329611,
r329614, r329619, r329621-r329622, r329624, r329626-r329627,
r329629-r329632, r329640-r329641, r329643-r329646, r329649-r329650, r329654,
r329656, r329662, r329669-r329671, r329673-r329674, r329680,
r329684-r329689, r329692-r329693, r329696-r329700, r329709, r329716,
r329731, r329733-r329734, r329747-r329748, r329756, r329779, r329782,
r329784, r329786, r329804, r329806, r329809, r329811, r329836,
r329850-r329852, r329854, r329856-r329858, r329861, r329895,
r329897-r329899, r329901-r329903, r329922-r329924, r329927-r329928,
r329944-r329949, r329986-r329987, r330008-r330010, r330012, r330020,
r330082-r330084, r330087-r330088, r330098-r330101, r330138-r330139,
r330261-r330263, r330267-r330269, r330281-r330284, r330287, r330339-r330342,
r330345-r330346, r330369-r330370, r330434-r330435, r330564, r330616-r330618,
r330620, r330625, r330690, r330701, r330703, r330825, r331211, r331257,
r331259, r331281-r331282, r331304, r331314, r331476-r331477,
r331563-r331564, r331854-r331857, r331859-r331860, r332106, r334723,
r334879, r334891, r334907, r334912, r334939, r334986, r335009, r335371,
r336759, r337711, r337807-r337810, r338054, r338063, r338065-r338067,
r338083, r338085-r338086, r338108, r338167-r338168, r338173, r338203,
r338255, r338259, r338309, r338394, r338438, r338886, r338893, r339173,
r339200, r339218, r339222, r339301, r339307, r339677-r339678, r339702,
r339805, r339831, r339849, r340040, r340152

Some notes for this MFC:
- This is still pre-forth/lua coexistance; that will come shortly-ish so
that forth/lua may be installed together (with forth remaining the default)

- module_blacklist support for lualoader has been MFC'd, but the drm modules
are not blacklisted in this stable branch.

r326353: Import lua 5.3.4 to contrib
r328440: Preserve the original luaconf.h in a convenient place. Clients will
r328443: Gross hack to omit printing hex floating point when the lua number
r329166: Add Lua as a scripting langauge to /boot/loader
r329167: Add the lua scripts from the lua-bootloader SoC
r329274: stand/lua: Exit sub-menus on backspace
r329329: stand/lua: Always boot on 'enter' keypress in menus
r329349: stand/lua: Reduce magic numbers
r329350: stand/lua: Don't descend into an empty kernels submenu
r329351: stand/lua: Set reasonable ACPI default based on presence
r329352: stand/lua: Consistently use semicolons for line endings
r329355: stand/lua: Move kernel selection into main menu
r329356: stand/lua: Allow menu items to not have explicit aliases
r329357: stand/lua: Remove explicit alias from "Back to main menu"
r329358: stand/lua: Say "loader prompt" instead of "lua interpreter"
r329359: stand/lua: Remove a magic number/string (not a trivial literal)
r329366: stand/lua: Set ACPI's default the proper way (setACPI)
r329367: stand/lua: Create a "carousel" menu entry type
r329368: stand/lua: Create/use some MENU_ constants where applicable
r329369: stand/lua: Remove sneaky kernel assignment
r329386: stand/lua: Correct usage and acceptance of BACKSPACE/DELETE keys
r329387: stand/lua: Chop off the decimal for numbers passed to setcursor
r329393: stand/lua: Use escaped dot instead of single character class
r329413: stand/lua: Allow MENU_RETURN items to have func, fix esc. to prompt
r329414: stand/lua: Don't try to divide by 0; do nothing
r329415: stand/lua: Don't reload kernel config if we only have one kernel
r329417: stand/lua: Make CAROUSEL_ENTRY func parameters consistent with name
r329424: stand/lua: Don't set autoboot_delay=NO in menu autoboot sequence
r329425: stand/lua: Enable menu autoboot; it seems to work
r329426: stand/lua: Correct interpretation of autoboot_delay
r329427: stand/lua: Color non-default kernels blue
r329428: stand/lua: Correct some trivial errors in config
r329429: stand/lua: Add debug method to dump modules
r329430: stand/lua: Address some nits
r329431: stand/lua: Correct test sense, this should have been 'not nil'
r329432: stand/lua: Try to load alternate kernels as directories first
r329433: stand/lua: Add optional GELI passphrase prompt
r329434: stand/lua: Check for nil (GELI prompt)
r329435: stand/lua: Style pass
r329436: stand/lua: Debugging string snuck in...
r329457: stand/lua: dumpModules => lsModules
r329473: liblua: Clean up io/loader C module registration
r329474: liblua: Emulate DIR, opendir, fdopendir, closedir
r329496: stand/lua: Fix verbiage and some typos
r329497: stand/lua: Fix module_path handling with multiple kernels
r329498: stand/lua: Remove some debugging bits that snuck in... gr...
r329499: interp_lua: Register io/loader with regular Lua module system
r329500: Lua loader: Add barebones "lfs" module
r329501: lua loader: Auto detect eligible list of kernels to boot
r329503: liblua: Fix missing '}' in lutil.c after r329499
r329504: stand/lua: More style nits, config.lua
r329543: Create style.lua(9)
r329547: stand/lua: Allow menu items to be conditionally (in)visible
r329548: stand/lua: Addres style.lua(9) concern
r329549: stand/lua: Clear the screen before prompting for passwords
r329550: stand/lua: Store the loaded kernel as config.kernel_loaded
r329551: stand/lua: reload previously loaded kernel at config-load/reload
r329576: stand/lua: Defer kernel/module loading until boot or menu escape
r329577: stand/lua: Rename bootserial for clarity
r329578: stand/lua: Menu style.lua(9) nits
r329580: stand/lua: Remove some unused local declarations
r329583: stand/lua: Store menu entries in an "entries" table
r329585: stand/lua: Add core.isSingleUserBoot
r329586: stand/lua: Call menu_entries if it's a function
r329587: stand/lua: Swap single-/multi- user boot entries as needed
r329588: stand/lua: Re-wrap menu.lua now that I've added indentation...
r329589: stand/lua: Track env changes that come in via loader.conf(5)
r329590: stand/lua: Restore environment upon config reload
r329592: stand/lua: Remove inaccurate comment after r329590
r329593: stand/lua: Change boot menu items' names when swapped
r329594: stand/lua: Round up some more style.lua(9) concerns
r329595: stand/lua: Re-order locals after copyright notice; require first
r329596: stand/lua: Add copyright notice in places
r329609: stand/lua: Cache swapped menu, and don't create locals for swapping
r329610: style.lua(9): Note that wrapping at 80-columns is not rigid
r329611: stand/lua: Wrap tuple assignment earlier for readability
r329614: stand/lua: Don't set ACPI off just because we can't detect it.
r329619: stand/lua: Extract menu handlers out into menu.handlers table
r329621: stand/lua: Add and use drawer.menu_name_handlers
r329622: stand/lua: Move drawer.menu_name_handlers further up
r329624: stand/lua: Reduce exposure of the drawer module
r329626: stand/lua: Refactor logos into drawer.logodefs table
r329627: stand/lua: Refactor brands into drawer.branddefs
r329629: stand/lua: Use 'graphic' instead of 'logo' for depicting graphics
r329630: stand/lua: Stick a copyright notice on drawer.lua
r329631: stand/lua: Insert helpful comment for drawer.branddefs
r329632: style.lua(9): Clarify local variable guideline
r329640: stand/lua: Consistently declare local functions at module scope
r329641: stand/lua: Consistently organize modules
r329643: Implement loader.command
r329644: lualoader: Add ability to intercept cli commands
r329645: lualoader: Move carousel storage out into config
r329646: lualoader: Eliminate global namespace pollution in loader.lua
r329649: Lua lfs.attributes: Provide a more consistent error return
r329650: liblua: Add loader.machine and loader.machine_arch properties
r329654: lualoader: Ignore ACPI bits on !i386
r329656: loader.lua: Expose errno table to lua
r329662: lualoader: Replace invalid construct with valid construct
r329669: lualoader: Prefer selected kernel to currently loaded
r329670: lualoader: Don't try so hard to load a kernel
r329671: lualoader: Prepare for interception of "boot" CLI cmd
r329673: lualoader: Intercept boot cli command
r329674: lualoader: Intercept the 'autoboot' cli command
r329680: lualoader: When restoring environment, only restore unchanged vars
r329684: lualoader: Drop terminating semicolons
r329685: lualoader: Drop excessive parenthesizing
r329686: style.lua(9): Drop notes about semicolons
r329687: lualoader: Drop explicit boolean tests; b or not b
r329688: lualoader: Don't return false for failure to open config on silent
r329689: lualoader: Output "Failed to parse" messages
r329692: lualoader: Bring in local.lua module if it exists
r329693: lualoader: Return only argstr if with_kernel not requested
r329696: lualoader: Add "menu.default", initialized to menu.welcome
r329697: lualoader: Drop name requirement for menu separators
r329698: lualoader: Directly reference submenu definition with submenu key
r329699: lualoader: Simplify menu definitions a little further
r329700: lualoader: Allow carousel 'items' to be a table as well
r329709: lualoader: Don't autodetect kernels if 'kernels' is explicitly set
r329716: lualoader: Use the key that interrupts autoboot as a menu choice
r329731: lualoader: Add boot environment support
r329733: lualoader: Make kernel autodetect. contingent on loader.conf(5) var
r329734: lualoader: Don't execute menu.autoboot() for every opened menu
r329747: lualoader: Replace 8-space indentation with a single tab
r329748: lualoader: Drop password length restrictions
r329756: lualoader: Remove nasty hack for not printing out ".0"
r329779: lualoader: Split cli bits out into a cli module
r329782: lualoader: Unbreak 'boot [kernel]' by including config
r329784: lualoader: Pull argument extraction for cli funcs to cli.arguments
r329786: lualoader: Attach cli command functions to cli module
r329804: lualoader: Eliminate some unused locals
r329806: lualoader: Consistently use double quotes
r329809: lualoader: Address some 'luacheck' concerns
r329811: lualoader: Clear up an empty conditional branch
r329836: lualoader: Attend to some 80-col issues, pointed out by luacheck
r329850: lualoader: Drop unused return values; we'll only use the first
r329851: Add SPDX tags to lua files
r329852: Add copyright notice to core.lua
r329854: lualoader: shallowCopyTable => deepCopyTable
r329856: lualoader: Use "local function x()" instead "local x = function()"
r329857: Centralize lua defines
r329858: When the LUA_FLOAT_TYPE != LUA_FLOAT_INT64, we can't ref. float
r329861: lualoader: Track effective line number, use it for drawing
r329895: liblua: Implement write support
r329897: lualoader: Add nextboot support
r329898: lualoader: Plug file handle not properly closed
r329899: lualoader: Correct test and name
r329901: lualoader: Add comment on trailing space, don't operate on nil
r329902: lualoader: Remove unused variable; we now use effective line number
r329903: lualoader: Explain nextboot stuff a little bit more
r329922: lualoader: Split config file I/O out into a separate function
r329923: lualoader: Strip config.parse of its I/O privileges
r329924: lualoader: throw out nextboot's usage of standard config processing
r329927: lualoader: Clean up naming conventions a little bit
r329928: lualoader: Remove inaccurate part of comment
r329944: lualoader: Don't explicitly index tables without reason
r329945: lualoader: menu: Terminate final values in tables with a comma
r329946: lualoader: Clean up menu handling a little bit
r329947: lualoader: Pull menu redrawing specifics out of menu.process
r329948: lualoader: Pull autoboot handling out into menu.run()
r329949: lualoader: Explain deviation from naming guidelines
r329986: lualoader: Invalidate the screen from menu perspective mnu exit
r329987: lualoader: Track the menu currently drawn, instead of validity
r330008: lualoader: A little more general menu cleanup
r330009: lualoader: More argument name expansion, part 2
r330010: lualoader: screen argument fixes
r330012: style.lua(9): Add some additional notes about naming and commas
r330020: lualoader: Re-work menu skipping bits
r330082: lualoader: Add a twiddle at password prompt
r330083: lualoader: Remove remnants of testing...
r330084: lualoader: Replace instances of \027 with KEYSTR_ESCAPE
r330087: lualoader: Convert instances of KEYSTR_ESCAPE .. "[" -> KEYSTR_CSI
r330088: lualoader: Correct test sense, comments, and add some more comments
r330098: lualoader: Re-do twiddle
r330099: lualoader: Further screen cleanup
r330100: lualoader: Remove debug function
r330101: lualoader: Add note that \027 is a decimal representation
r330138: lualoader: Dedup these "Return to main menu" entries
r330139: lualoader: config: Pull some messages out into constants
r330261: lualoader: Fix some lint-mentioned errors
r330262: lualoader: Use #str instead of tracking length with 'n'
r330263: lualoader: Use string literal \xNN instead of string.char()
r330267: Add core.lua(8), but do not add to distribution
r330268: Add menu.lua(8), but do not add to distribution
r330269: core.lua(8): Add missing note about core.KEYSTR_CSI
r330281: lualoader: Steamroll the box-drawing
r330282: lualoader: Register loader.printc as global printc
r330283: lualoader: Use global printc instead of loader.printc
r330284: liblua: Use putc instead of printf for printc
r330287: lualoader: Reset the cursor position after the menu is drawn
r330339: liblua: Add loader.interpret
r330340: lualoader: Execute menu_timeout_command at the end of menu autoboot
r330341: lualoader: Respect loader_menu_title, prepare for align
r330342: lualoader: Respect loader_menu_title_align
r330345: lualoader: Tweak positioning and fix an off-by-one
r330346: lualoader: Shift menu+brand even for logo=none with customized pos
r330369: lualoader: Return meaningful value in cli_execute
r330370: lualoader: logdef -> logodef typo
r330434: lualoader: Add note about importance of including cli module early
r330435: lualoader: Use FILESDIR instead of BINDIR
r330564: lualoader: Only loadelf before boot/autoboot if no kernel loaded
r330616: lualoader: Expose loader.parse and add cli_execute_unparsed
r330617: lualoader: Fix name, cli.execute_unparsed -> cli_execute_unparsed
r330618: lualoader: Use cli_execute_unparsed instead of loader.interpret
r330620: lualoader: Use cli_execute_unparsed for commands via loader.conf
r330625: lualoader: Return status in cli_execute_unparsed properly
r330690: stand: Fix copy-paste-o, unbreaks libi386 lualoader build
r330701: lualoader: Don't redraw the autoboot message every .05s
r330703: lualoader: Cache kernel list
r330825: lualoader: Sprinkle some verbose_loading salt
r331211: lualoader: Setup default color scheme if we're using colors
r331257: lualoader: Reset attributes and color scheme with color.highlight()
r331259: lualoader: Use less atomic options for resetting colors/attributes
r331281: lualoader: Add primitive hook module to untangle bogus reference
r331282: core.lua(8): Update to reflect recently added function
r331304: lualoader: Clear up some possible naming confusion
r331314: lualoader: Use printc when we expect ANSI escape sequences
r331476: lualoader: Make config env-related bits private API
r331477: lualoader: Privatize some more config.lua bits
r331563: lualoader: Implement try_include and use it for including local
r331564: lualoader: Actually re-raise error in try_include
r331854: lualoader: Do case-insensitive comparison of "yes"
r331855: lualoader: Don't assume that {module}_load is set
r331856: lualoader: revert whitespace change that snuck in
r331857: lualoader: Simplify some expressions
r331859: lualoader: Split logodefs out into logo-* files
r331860: lualoader: Don't try to lookup a nil logo
r332106: lualoader: Fix menu skipping with loader.conf(5) vars
r334723: lualoader: Add a loaded hook for others to execute upon config load
r334879: lualoader: Add hook.lua(8) to tree
r334891: lualoader: Add cli.lua(8) to the tree
r334907: lualoader: Process loader_conf_files properly
r334912: lualoader: Support variable substitution in env var settings
r334939: lualoader: Allow brand-*.lua for adding new brands
r334986: lualoader: More black-on-white fixes
r335009: lualoader: Match Forth module-loading behavior w.r.t flags
r335371: lualoader: Correct kernel_options handling
r336759: lualoader: "nextboot_file" should be spelled "nextboot_conf"
r337711: lualoader: Fix parsing of negative number loader.conf(5) variables
r337807: Prevent a wanring about checkdp being unused.
r337808: When the LUA floating point model is INT64, we don't need to do the
r337809: For our INT64 implementation, we can compare integers and numbers
r337810: MFV r337586: lua: Update to 5.3.5
r338054: Add config.lua(8) to the tree
r338063: lualoader: Stop exporting drawer.draw
r338065: lualoader: Hide most of the internal drawing functions
r338066: lualoader: Hide the rest of the private interfaces
r338067: lualoader: Add drawer-exported variables for default logodefs
r338083: Add drawer.lua(8)
r338085: Add color.lua(8), password.lua(8), and screen.lua(8)
r338086: lualoader: Install all manpages
r338108: Serial console menus for lua.
r338167: lualoader: Just compare expression directly
r338168: lualoader: Refactor config line expressions
r338173: lualoader: Fix loader.conf(5) EOL validation for 'exec' lines
r338203: Turn off LOADER_GELI and LOADER_LUA for sparc64, until functional
r338255: lualoader: Fix (add) Xen support
r338259: lualoader: Accept that people use unquoted values in loader.conf
r338309: lualoader: Fix override of module_path on loader prompt
r338394: lualoader: fix color usage
r338438: lualoader: Handle comma-separated kernels as well
r338886: Improve loader passwords:
r338893: Set the default loader for powerpc(32- and 64-bit) back to to forth
r339173: Set the default loader for powerpc64 back to to forth too.
r339200: lualoader: Don't draw loader menu with autoboot_delay=-1
r339218: lualoader: Create a module blacklist, add DRM modules to it
r339222: lualoader: Honor boot_* variables at lua init
r339301: Loader GELI support, like lua loader, seems to be broken on PowerPC
r339307: lualoader: Provide a 'menu' command to redraw the menu at prompt
r339677: lualoader: unload upon kernel change if a kernel was loaded
r339678: menu.lua: Abort autoboot sequence on failed command
r339702: lualoader: Improve module loading diagnostics
r339805: lualoader: Always return a proper dictionary for blacklist
r339831: Move LUA_ROOT to /boot/lua
r339849: lualoader: Fix try_include error handling
r340040: lualoader: Implement boot-conf
r340152: lualoader: Add chainload menu entry

Relnotes: yes ("lualoader has been merged to stable/11, off by
default, to facilitate testing")

MFC r339509: Fix loader.conf(5) "password" feature

Restore the ability to prevent the user from interrupting the boot process
without first entering the password stored in loader.conf(5).

PR: kern/207069
Reported by: david@dcrosstech.com
Sponsored by: Smule, Inc.

Direct commit since these files have gone away in head

Move pc98's biosdisk.c to the new style disk access. This is missing
support in common/part.c, however, for pc98 partitions, so it's
unlikely to actually work. Lack of a pc98 machine that's in sevice
limits my ability to test, but this allows pc98 to compile again.

Direct commit to stable, file not present in current

Catch up to r332154: Fix d_dev removal of d_type.

MFC 338022: Fix casts between 64-bit physical addresses and pointers in EFI.

Compiling FreeBSD/i386 with modern GCC triggers warnings for various
places that convert 64-bit EFI_ADDRs to pointers and vice versa.
- Cast pointers to uintptr_t rather than to uint64_t when assigning
to a 64-bit integer.
- Cast 64-bit integers to uintptr_t before a cast to a pointer.

MFC r338219, r338250: FDT in Loader fixes

r338219:
fdt_fixups: relocate the /chosen node after applying fixups

As indicated by the comment, any fixups applied (which might include
overlays) can invalidate the previously located node by adding nodes or
setting/adding properties. The later fdt_setprop of fixup-applied property
would then fail because of the bad/wrong node offset.

This would have generally been harmless, but potentially caused multiple
applications of fixups and caused a little bit of bloat.

r338250:
efiloader: Setup FDT in autoload to fix overlays clobbering kenv

manu found in the noted PR that overlays seemed to be clobbering the kenv
and killing the boot. Further inspection revealed that one can `fdt ls` at
the loader prompt for a successful boot, but autoboot breaks it.

In the autoboot case, first setup of FDT is happening in the middle of
bi_load, which triggers loading of the DTBO from /boot.

This is bad, bad, bad. Files in the loader are loaded somewhere in the
middle of the address space one after another. bi_load starts building the
needed kernel bootinfo immediately after the highest-addr loaded file. File
loads in the middle of bi_load suddenly clobber bootinfo and everything goes
off the rails.

The solution to this is to use take advantage of arch_autoload to setup FDT
in efiloader compiled with LOADER_FDT_SUPPORT. This matches how it works in
ubldr land, and is how it should have worked when overlay support was added
to efiloader since fdt_setup_fdtp now has the potential to load files
(courtesy of fdt_platform_load_dtb).

MFC r338201: increase heap size during "loader" on sparc64

Increase the size of the heap size available on sparc64 during
operation of "loader". The dramatic increase in size of
SPA_MAXBLOCKSIZE in r304321 causes the heap space to be exhausted,
so malloc() fails, ultimately leading to a memcpy() with a
destination of 0x0.

MFC r337524: libi386: Fix typo in pxe.h

PR: 207337

MFC r337523: libsa: exit on EOF in ngets

It was possible in some rare circumstances for ngets to behave terribly with
bhyveload and some form of redirecting user input over a pipe.

PR: 198706

MFC r337520: Fix WITHOUT_LOADER_GELI (gptboot) and isoboot in general

gptboot was broken when r316078 added the LOADER_GELI_SUPPORT #ifdef to
not pass geliargs via __exec. KARGS_FLAGS_EXTARG must not be used if we're
not going to pass an additional argument to __exec.

PR: 228151

ubldr: Bump heap size, 1MB -> 2MB

1MB was leaving very little margin in some of the worse-case scenarios with
lualoader. 2MB is still low enough that we shouldn't have any problems with
UBoot-supported boards.

MFC r335733:

Don't use CCACHE for linking.

MFC r336919, r336924

r336919:
efirt: Add tunable to allow disabling EFI Runtime Services

Leading up to enabling EFIRT in GENERIC, allow runtime services to be
disabled with a new tunable: efi.rt_disabled. This makes it so that EFIRT
can be disabled easily in case we run into some buggy UEFI implementation
and fail to boot.

r336924:
Follow up to r336919 and r336921: s/efi.rt_disabled/efi.rt.disabled/

The latter matches the rest of the tree better [0]. The UPDATING entry has
been updated to reflect this, and the new tunable is now documented in
loader(8) [1].

Reported by: imp [0], Shawn Webb [1]

MFC r335874: Cut currdev dev and path fields for ZFS on the last colon.

Previously the code cut those fields on second colon, that prevented
boot from boot environments with colon in their names. This change
moves the limitation from dev field to path, which is empty by default.

MFC r334882, r334884-r334885: loader(8) boot flag <-> environment fixes

r334882:
stand: Consolidate checking for boot flags driven by environment vars

e.g. boot_mute, boot_single, boot_verbose, and friends; we checked for these
in multiple places, consolidate into common/ and allow a setting of "NO" for
any of these to turn them off. This allows systems with multiple
loader.conf(5) or loader.conf(5) overlay systems to easily turn off
variables in later processed files by setting it to NO.

Reported by: Nick Wolff @ iXsystems
Reviewed by: imp

r334884:
stand: Fix build after r334882

Not sure how this was not caught in Universe.

r334885:
stand: One more trivial consolidation (setting environment from howto)

MFC r334878: libsa(3): Correct statement about FS Write-support, name change

- jhb implemented UFS write support a little over 16 years ago.
- Update the library name while we're here.

MFC r335607: check-password.4th(8): Fix manual [in]accuracy

SVN r280384 updated the maximum password length from 16 bytes to 255. The
manual was not updated to reflect this.

Sponsored by: Smule, Inc.

MFC r334656, r334665, r334695

r334656:
Add vsnprintf() to libsa. Alpha-sort the printf prototypes in stand.h.

r334665:
Make the v*printf() functions in libsa return int instead of void.
This makes them compatible with the C standard signatures, avoiding
spurious mismatch errors in the places where the oddball requirements
of standalone code end up putting two declarations of the same function
in play.

r334695:
Remove comments and assertions that are no longer valid after r330809.

r330809 replaced duplication of devdesc struct fields with an embedded copy
of the devdesc struct, to avoid fragility. That means all the scattered
comments indicating that structs must match are no longer valid. Likewise
asserts that attempted to mitigate some of the old fragility.

Reviewed by: imp@

MFC r334432:

Fix build of stand with base gcc

* Make autoboot() a static function in stand/common/boot.c, so it does
not shadow local variables in gptboot.c and zfsboot.c.
* Remove -Winline from the Makefiles for gptboot, gptzfsboot and
zfsboot, as gcc will always fail to inline some functions, and there
is nothing we can do about it.
* For gcc <= 4.2.1, silence -Wuninitialized for isoboot, as it produces
a false positive warning.
* Remove deprecated and unnecessary -mcpu=i386 flag from stand/defs.mk,
as there is already a -march=i386 flag further in the file.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D15628

MFC r334310, r334337:

r334310 (imp):
Teach ufs_module.c about bsd labels and probe 'a' partition.

If the check for a UFS partition at offset 0 on the disk fails, check
to see if there's a BSD disklabel at block 1 (standard) or at offset
512 (install images assume 512 sector size). If found, probe for UFS
on the 'a' partition.

This fixes UEFI booting images from a BSD labeled MBR slice when the
'a' partiton isn't at offset 0. This is a stop-gap fix since we plan
on removing boot1.efi in FreeBSD 12. We can't easily do that for 11.2,
however, hence the short MFC window.

r334337 (emaste):
switch amd64 memstick installer images to MBR

A good number of BIOSes have trouble booting from GPT in non-UEFI
mode.

With this change amd64 memsticks remain dual-mode (booting from either
UEFI or CSM); the partitioning type is just switched from GPT to MBR.

PR: 227954

Note, there are two changes specific to stable/11 where there is code
that had diverged from head and never merged back. The two changes are
an include in stand/efi/boot1/ufs_module.c, replacing sys/disk/bsd.h
with sys/disklabel.h and replacing BSD_MAGIC with DISKMAGIC in the
same file. The latter two are direct commits to stable/11 in order to
avoid unexpected regressions at this point of the 11.2 cycle. Thank
you to imp@ for pointing out what changes needed to be made.

Approved by: re (marius)

Sponsored by: The FreeBSD Foundation

MFC: r333955

- Unbreak booting sparc64 kernels after the metadata unification in
r329190 (MFCed to stable/11 in r332150); sparc64 kernels are always
64-bit but with that revision in place, the loader was treating them
as 32-bit ones.
- In order to reduce the likelihood of this kind of breakage in the
future, #ifdef out md_load() on sparc64 and make md_load_dual() -
which is currently local to metadata.c anyway - static.
- Make md_getboothowto() - also local to metadata.c - static.
- Get rid of the unused DTB pointer on sparc64.

Approved by: re (kib)

MFC r333079 (imp):
No need to make objects here.

This fixes polluting the src checkout when building arm64/aarch64.

Approved by: re (marius)
Sponsored by: The FreeBSD Foundation

whitespace changes to reduce diffs from i386.
still broken pc98 boot.

MFC r332090: stand: pass --no-rosegment for i386 bits when linking with lld

btxld does not correctly handle input with other than 2 PT_LOAD
segments. Passing --no-rosegment lets lld produce output eqivalent to
ld.bfd: 2 PT_LOAD segments and no PT_GNU_RELRO.

PR: 225775
Sponsored by: The FreeBSD Foundation

MFC r332085

Add an ISO9660 "partition table" type to loader.

When booted via isoboot(8) loader will be handed a disk that simply contains
an ISO9660 image. Currently this confuses it greatly. Teach it how to spot
that it's in this situation and that ISO9660 has one "partition" covering
the whole disk.

Sponsored by: iXsystems, Inc.

MFC r332084

Add isoboot(8) for booting BIOS systems from HDDs containing ISO images.

This is part of a project for adding the ability to create hybrid CD/USB boot
images. In the BIOS case when booting from something that isn't a CD we need
some extra boot code to actually find our next stage (loader) within an
ISO9660 filesystem. This code will reside in a GPT partition (similar to
gptboot(8) from which it is derived) and looks for /boot/loader in an
ISO9660 filesystem on the image.

Sponsored by: iXsystems, Inc.

MFC r332083:

Various style(9) fixes.

Sponsored by: iXsystems, Inc.

MFC r332082

Use consistent whitespace.

Sponsored by: iXsystems, Inc.

MFC r332416: Refactor currdev setting

Refactor the currdev setting to find the device we booted from. Limit
searching when we don't already have a reasonable currdev from that to
the same device only. Search a little harder for ZFS volumes as that's
needed for loader.efi to live on an ESP.

MFC r332561,r332573: Rename volume label for ESP

MFC r332561: Rename volume label for ESP

Harry Schmalzbauer reports that some firmware, in his experience, trips
over the ESP we install due to the volume label. It has been theorized that
this is due to some confusion with the label and the path on the ESP to
boot1.efi.

Regardless, Harry found that renaming the label seems to fix it.

MFC r332573: Regenerate FAT templates after r332561

PR: 214282

MFC 328101,328911: Require SHF_ALLOC for kernel object module sections.

328101:
Require the SHF_ALLOC flag for program sections from kernel object modules.

ELF object files can contain program sections which are not supposed
to be loaded into memory (e.g. .comment). Normally the static linker
uses these flags to decide which sections are allocated to loadable
program segments in ELF binaries and shared objects (including kernels
on all architectures and kernel modules on architectures other than
amd64).

Mapping ELF object files (such as amd64 kernel modules) into memory
directly is a bit of a grey area. ELF object files are intended to be
used as inputs to the static linker. As a result, there is not a
standardized definition for what the memory layout of an ELF object
should be (none of the section headers have valid virtual memory
addresses for example).

The kernel and loader were not checking the SHF_ALLOC flag but loading
any program sections with certain types such as SHT_PROGBITS. As a
result, the kernel and loader would load into RAM some sections that
weren't marked with SHF_ALLOC such as .comment that are not loaded
into RAM for kernel modules on other architectures (which are
implemented as ELF shared objects). Aside from possibly requiring
slightly more RAM to hold a kernel module this does not affect runtime
correctness as the kernel relocates symbols based on the layout it
uses.

Debuggers such as gdb and lldb do not extract symbol tables from a
running process or kernel. Instead, they replicate the memory layout
of ELF executables and shared objects and use that to construct their
own symbol tables. For executables and shared objects this works
fine. For ELF objects the current logic in kgdb (and probably lldb
based on a simple reading) assumes that only sections with SHF_ALLOC
are memory resident when constructing a memory layout. If the
debugger constructs a different memory layout than the kernel, then it
will compute different addresses for symbols causing symbols in the
debugger to appear to have the wrong values (though the kernel itself
is working fine). The current port of mdb does not check SHF_ALLOC as
it replicates the kernel's logic in its existing kernel support.

The bfd linker sorts the sections in ELF object files such that all of
the allocated sections (sections with SHF_ALLOCATED) are placed first
followed by unallocated sections. As a result, when kgdb composed a
memory layout using only the allocated sections, this layout happened
to match the layout used by the kernel and loader. The lld linker
does not sort the sections in ELF object files and mixed allocated and
unallocated sections. This resulted in kgdb composing a different
memory layout than the kernel and loader.

We could either patch kgdb (and possibly in the future lldb) to use
custom handling when generating memory layouts for kernel modules that
are ELF objects, or we could change the kernel and loader to check
SHF_ALLOCATED. I chose the latter as I feel we shouldn't be loading
things into RAM that the module won't use. This should mostly be a
NOP when linking with bfd but will allow the existing kgdb to work
with amd64 kernel modules linked with lld.

Note that we only require SHF_ALLOC for "program" sections for types
like SHT_PROGBITS and SHT_NOBITS. Other section types such as symbol
tables, string tables, and relocations must also be loaded and are not
marked with SHF_ALLOC.

328911:
Ignore relocation tables for non-memory-resident sections.

As a followup to r328101, ignore relocation tables for ELF object
sections that are not memory resident. For modules loaded by the
loader, ignore relocation tables whose associated section was not
loaded by the loader (sh_addr is zero). For modules loaded at runtime
via kldload(2), ignore relocation tables whose associated section is
not marked with SHF_ALLOC.

MFC r330970, r331067, r331767, r331852, r331858

r330970:
libefi: UEFI_BOOT_VAR_GUID duplicates EFI_GLOBAL_VARIABLE

Drop UEFI_BOOT_VAR_GUID and use EFI_GLOBAL_VARIABLE.

r331067:
Only print boot order / boot current if we can get the variables from
the loader. Some UEFI implementations don't return all of them.

Sponsored by: Netflix

r331767:
efinet: Do not return only if ReceiveFilter fails

If the network interface or the uefi implementation do not support the
ReceiveFilter interface do not return only and just print a message.
U-Boot doesn't support is and likely never will. Also even if this fails
it doesn't mean that network in EFI isn't supported.

r331852:
fwohcireg.h is 99% the same between the boot loader and the
kernel. Delete it and fix up the 1% difference because there's no need
for them to be different.

r331858:
The Uninorth ID was really for Uninorth 2.

MFC r331331: Apply r228478 (CTASSERT => _Static_assert()) to bootstrap.h

MFC r330806-r330815, r330837, r330864, r330883

pc98 changes associated with the named commits are also included in this
commit, despite not having been made with the original commits due to its
removal in head.

r330806:
Minor cosmetic changes.

Make sure { on the same line as struct for all struct *devdesc. Move
some type definitions to next to the dv_type define, since that's what
sets the d_type.

r330807:
We can't use d_opendata for blkio storage.

open_disk uses d_opendata for it's own purpse. We can't store blkio
there. Fortunately, blkio is stored elsewhere and we never actually
retrieve blkio from d_opendata. Eliminate it as a source of confusion.
Eliminate all stores of d_opendata in efi since this layer doesn't own
that field.

r330808:
Make struct libi386_devdesc match the struct devdesc better

Move data to top and call it d_opendata.

r330809:
Use the actual struct devdesc at the start of all *_devdesc structs

The current system is fragile and requires very careful layout of all
*_devdesc structures. It also makes it hard to change the base
devdesc. Take a page from CAM and put the 'header' in all the derived
classes and adjust the code to match.

For OFW, move the iHandle h_handle out of a slot conflicting with
d_opendata. Due to quirks in the alignment rules, this worked.
However changing the code to use d_opendata storage now that it's a
pointer is hard, so just have a separate field for it.

All other cleanups were to make the *_devdesc structures match where
they'd taken some liberties that were none-the-less compatible enough
to work.

r330810:
Remove d_type from devdesc. It's not needed as we can fetch it from
d_dev->dv_type when we need it.

r330811:
GC unused routines.

Sponsored by: Netflix

r330812:
Use the one-line-per-file pattern here, and sort the file names.

Sponsored by: Netflix

r330813:
Move the env convenience routines out of boot1.c.

These routines are more generally useful. Even though boot1 is on its
way out, it's better to make these common during the transition than
copy them.

r330814:
Star BootCurrent entry when booting.

Sponsored by: Netflix

r330815:
Print the load and device path as well as BootCurrent and BootOrder

Sponsored by: Netflix

r330837:
biosdisk.c should not set d_opendata.

Same as 330807, d_opendata is owned by open_disk and we should not
set it.

M stand/i386/libi386/biosdisk.c

r330864:
Prefer uintXX_t to u_intXX_t

A foolish consistency is the hobgoblin of little minds, adored by
little statesmen and philosophers and divines. With consistency a
great soul has simply nothing to do. -- Ralph Waldo Emerson

r330883:
Fix typo that misteriously passes compilation.

MFC r330365, r330371: OFW changes

r330365:
Move "common" Open Firmware parts of the loader used only on PowerPC to
the powerpc/ subdirectory. These have never used by SPARC and we have
no other (and almost certainly will have no other) Open Firmware platforms.
This makes the directory structure simpler and lets us avoid some
cargo-cult MI patterns on code that is, and always was,
architecture-specific.

r330371:
Where we can, pass the kernel an FDT facsimile of the OF device tree rather
than a pointer to Open Firmware by default. This eliminates a number of
potentially unsafe calls to firmware from the kernel and provides better
performance.

This feature is meant to be expanded until it is on by default
unconditionally and, ideally, we can then garbage-collect the
nightmare pile of hacks required to call into Open Firmware from a live
kernel.

MFC r329190, r329315, r330131: metadata load file unification

r329190: Unify metadata load files for arm, mips, powerpc, sparc64

Summary:
All metadata.c files are very similar, with only trivial changes. Unify them
into a single common file, with minor special-casing where needed.

r329315: stand: Fix ubldr after r329190

metadata load files were consolidated in r329190, and these relocation fixup
bits were inadvertently dropped in the process. Re-add them to fix boot with
ubldr.

r330131: Fix module loading on arm after the metadata.c unification in
r329190. Arm modules need an additional address fixup not needed by other
platforms.

MFC r330788: beri loader: Replace getc/putc with beri_ prefixed versions

This matches a convention that we use, at least in ubldr, to prefix
getc/putc with a loader-specific prefix to avoid collisions. This was
encountered while trying to build the beri loader with MK_LOADER_LUA=yes.

MFC r330249-r330250

r330249:
stand: Makefile SUBDIR cleanup

Use SUBDIR.${MK_*} where appropriate. r330248 eliminated most of the
offenders, sweep the rest under the rug.

Differential Revision: https://reviews.freebsd.org/D14545

r330250:
stand: Fix build after r330249

One does not simply convert to SUBDIR.yes in stand without making everything
else in the affected files SUBDIR.yes -- there are better ways to do this.

Revert lua bits that snuck in with r332144

These will get re-MFC'd differently when we import lualoader from head. This
is a direct commit to stable/11

MFC r330077, r330248

r330077:
Move EFI up to common makefile. There's no need for all these .if's
based on architecture.

Sponsored by: Netflix

r330248:
Create LOADER_UBOOT, and LOADER_OFW. Move these options out of
Makefile.${MACHINE_ARCH} and remove the now-empty files. Generate the
*32 directories on the necessary architectures (well, currently only
amd64) on the fly. Remove LOADER_EFI variable and co-locate it with
EFI.

MFC r330056: libsa: replace remaining _write callbacks by null_write

There are some _write callbacks left only returning EROFS, replace them
by null_write. return EROFS from null_write().

MFC r330026: libsa: Move MAXWAIT from net.h to net.c

It's not a setting that has any effect or use outside of the net.c context.

MFC r329879, r329892

r329879:
libsa: Const-ify buffer argument of write(2) analog

r329892:
libsa: Change write(2)-alike prototype to match definition

Broken in r329879.

Apparently old GCC detects this, but modern GCC didn't. Mea culpa.

MFC r329859,r329860: Float protection in stand

r329859: Do not include float interfaces when using libsa.

We don't support float in the boot loaders, so don't include
interfaces for float or double in systems headers. In addition, take
the unusual step of spiking double and float to prevent any more
accidental seepage.

r329860: Floaty McFloatface is funnier...

MFC r329758: Fix compilation with LOADER_DEBUG defined after r329725.

MFC r329745: load_elf.c: Use consistent indentation

As noted in D14267 load_elf.c has a variety of indentation styles. Move
to standard 8 column hard tab indents, 4 space second level indents.
Also includes some whitespace cleanups found by clang-format.

MFC r329345: Eliminate bsd.stand.mk and -fPIC 32-bit intel builds

OK. We don't really need a bsd.stand.mk, and it was causing a -fPIC
for the toolchain to be added (bogusly) when building on amd64. Pull
all relevant defs back into defs.mk and delete bsd.stand.mk.

This saves about 15-20k on i386 loader and zfsloader which when
combined with Lua give us a lot more stack space in those constrained
environments.

MFC r329726, r329737, r329740

r329726:
Honor settings for including / excluding cd9660, ufs, ext2fs and msdos.

The Makefile gives the impression that ext2fs and msdos were excluded
(they weren't) and that you could exclude cd9660 and ufs support (you
couldn't). Allow those to be excluded.

We need to look, in the future, at trimming the number of supported
filesystems, and this will make that easier.

r329737:
Purely whitespace changes bringing this file closer to style(9).

Curiously, changing whitespace seems to cause the md5 of the .o files to differ
these days hence the following testing strategy:

Tested by: objdump -d | md5 (both in-tree clang and lang/gcc6)

r329740:
Further style(9) changes.

Tested by: objdump -d | md5 (both in-tree clang and lang/gcc6)

MFC r329725, r329831

r329725:
Consolidate three copies of ZFS commands into a central location.

There's no reason to have multiple copies of lszfs and
reloadbe. Consolidate them into one location. Also ldi_get_size is the
same everywhere (except sparc64). Make it the same everywhere as the
common definition is more general and will work on spar64.

r329831:
Fix userboot w/ ZFS after r329725

r329725 cleaned up ZFS commands duplicated in multiple places, but userboot
was not setting HAVE_ZFS when MK_ZFS != "no". This resulted in a failure to
boot (as seen in PR 226118) in bhyve, with the following message:

/boot/userboot.so: Undefined symbol "ldi_get_size"

MFC r329268, r329517

r329268:
efi: Only scan the BLKIO MEDIA once

Scan only the BLOCK IO MEDIA once instead of each time for each type of
device (fd, cd and hdd).
Leave the mechanism to free and reprobe all devices if one day we want
to implement a "dev rescan" thing.

r329517:
efi: Do not pad the efi devpath structure

This solve problem when booting with efi on armv7

MFC r331212,r331213: Move /boot/overlays to /boot/dtb/overlays

r331212: Move /boot/overlays to /boot/dtb/overlays

The former is fairly vague; these are FDT overlays to be applied to the
running system, so /boot/dtb is a sensible location to put it without
cluttering up /boot/dtb even further if desired.

r331213: Amend missed reference to /boot/overlays

MFC r330005-r330007, r330021, r330029, r330622, r331207: Solo loader.conf(5)

r330005:
Go back to one loader.conf

We really only need one loader.conf. The other loader.conf was created
because the current one took forever to parse in FORTH. That will be
fixed in the next commit.

r330006:
Take a meat cleaver to defaults/loader.conf

Remove almost all of the _load=XXX options (kept only those relevant
to splash screens, since there were other settings).
Remove the excessively cutesy comment blocks.
Remove excessive comments and replace with similar content
Remove gratuitous blank lines (while leaving some)

We have too many modules to list them all here. There's no purpose in
doing so and it's a giant hassle to maintain. In addition the extra
~500 lines slow this down on small platforms. It slowed it down
so much small platforms forked, which caused other issues...
This is a compromise between those two extremes.

r330007:
loader.conf is loader agnostic, so remove 4th references.

r330021:
These two directories build man pages, so it's incorrect to tag them
NO_OBJ. Also, make sure the loader.conf.5 man gets built and installed.

r330029:
Fix a typo: "now" -> "no".

r330622:
loader.conf(5): Document some other settings

These tend to have less coverage in other places and they don't have
defaults as of yet, so mention them here:
- fdt_overlays
- kernels_autodetect (lualoader only)

r331207:
loader.conf: remove obsolete non-x86 beastie menu statement

As of r330005 the same loader.conf defaults are used on all platforms.

MFC r331718: stand: Add workaround for HP BIOS issues

hrs@ and kuriyama@ have found that on some HP BIOS, a system will fail to
boot immediately after installation with the claim that it can't work out
which disk they are booting from.

They tracked it down to a buffer overrun, and found that it could be
alleviated by doing a dummy read before-hand.

MFC r330868, r331241, r331361, r331365: EFIRT Fixes

r330868:
EFIRT: SetVirtualAddressMap with 1:1 mapping after exiting boot services

This fixes a problem encountered on the Lenovo Thinkpad X220/Yoga 11e where
runtime services would try to inexplicably jump to other parts of memory
where it shouldn't be when attempting to enumerate EFI vars, causing a
panic.

The virtual mapping is enabled by default and can be disabled by setting
efi_disable_vmap in loader.conf(5).

r331241:
Check if the gettime runtime service is valid.

The U-Boot efi runtime service expects us to set the address map before
calling any runtime services. It will then remap a few functions to their
runtime version. One of these is the gettime function. If we call into
this without having set a runtime map we get a page fault.

Add a check to see if this is valid in efi_init() so we don't try to use
the possibly invalid pointer.

r331361:
Enter into the EFI environment before dereferencing the runtime services
pointer. This may be within the EFI address space and not the FreeBSD
kernel address space.

r331365:
Re-work efidev ordering to fix efirt preloaded by loader on amd64

On amd64, efi_enter calls fpu_kern_enter(). This may not be called until
fpuinitstate has been invoked, resulting in a kernel panic with
efirt_load="YES" in loader.conf(5).

Move fpuinitstate a little earlier in SI_SUB_DRIVERS so that we can squeeze
efirt between it and efirtc at SI_SUB_DRIVERS, SI_ORDER_ANY. efidev must be
after efirt and doesn't really need to be at SI_SUB_DEVFS, so drop it at
SI_SUB_DRIVER, SI_ORDER_ANY.

The not immediately obvious dependency of fpuinitstate by efirt has been
noted in both places.

MFC r331630: Fix instructions in the zfsboot manual page.

zfsloader(8) fails to probe a slice containing ZFS pool if its second sector
contains traces of BSD label (DISKMAGIC == 0x82564557).
Fix manual page to show working example erasing such traces.

PR: 226714
Approved by: avg (mentor)

_M .
M stand/i386/zfsboot/zfsboot.8

MFC r331475: loader consoles: Implement SGR 24, 25

Mostly for completeness sake- implement 24 (no underline) and 25 (no blink)

MFC r331416, r331440: Loader consoles: Implement SGR 22, reste intensity

r331416:
Loader consoles: Implement SGR 22, reset intensity

r331440:
Match broken style of vidconsole.c

This particular function uses a broken mix of indentation styles. Match it
for the newly added SGR 22 bits.

MFC r330891: ubldr: Bump heap size from 512K to 1M

lualoader in itself only uses another ~200K, but there seems to be no reason
not to bump it a little higher to give us some more wiggle room.

With this, I can boot using a menu-enabled lualoader, no problem and
reasonably fast. Some heap usage datapoints from the review:

forthloader, no menus, kernel loaded:
heap base at 0x1203d5b0, top at 0x1208e000, used 330320

lualoader, no menus, kernel loaded:
heap base at 0x42050028, top at 0x420ab000, used 372696

lualoader, menus, kernel loaded:
heap base at 0x42050028, top at 0x420d5000, used 544728

Since then, the no menu case for lualoader should have decreased slightly as
I've made some changes to make sure that it no longer loads any of th emenu
bits with beastie disabled.

While here, split heap size out into a HEAP_SIZE macro.

MFC r330023, r330028: Add MAXWAIT for configuring pxeboot timeout

r330023: libsa: Add MAXWAIT to net for establishing max total timeout

Current timeout behavior is to progress in timeout values from MINTMO to
MAXTMO in MINTMO steps before finally timing out. This results in a fairly
long time before operations finally timeout, which may not be ideal for some
use-cases.

Add MAXWAIT that may be configured along with MINTMO/MAXTMO. If we attempt
to start our send/recv cycle over again but MAXWAIT > 0 and MAXWAIT seconds
have already passed, then go ahead and timeout.

This is intended for those that just want to say "timeout after 180 seconds"
rather than calculate and tweak MINTMO/MAXTMO to get their desired timeout.
The default is 0, or "progress from MINTMO to MAXTMO with no exception."

This has been modified since review to allow for it to be defined via CFLAGS
and doing appropriate error checking. Future work may add some Makefile foo
to respect LOADER_NET_MAXWAIT if it's specified in the environment and pass
it in as MAXWAIT accordingly.

r330028: libsa: Partially revert r330023

The removal of tmo >= MAXTMO check should not have been done; this is
specifically what handles timeout if MAXWAIT == 0.

MFC r330612: stand/ficl: Fix testmain

testmain is a userland application intended to be built with standard
headers and whatnot, which we broke.

Fix it by having the testmain build clobber cflags, reducing it to just the
set of defines/includes it needs to build.

MFC: r331140

Merge ahead of schedule to fix pc98 build issues.

Direct commit to stable

Remove libstand32 here. pc98 is a 32-bit platform, so it shouldn't compile
the extra 32-bit copy of libsa. The copy built in libstand is already 32-bit.
Add a comment saying we need an empty Makefile.pc98 since otherwise it would
pull in Makefile.i386 and there is no EFI on pc98, and the machines are too
old to have ZFS or GELI be a viable option (and besides, those don't compile).

Note: We also need r331140 to be MFC'd for pc98 build to work in all cases.

MFC r329339: libsa: Consolidate tftp sendrecv into net.c sendrecv

bootp/arp/rarp/rpc all use the sendrecv implementation in net.c. tftp has
its own implementation because it passes an extra parameter into the recv
callback for the received payload type to be held.

These sendrecv implementations are otherwise equivalent, so consolidate
them. The other users of sendrecv won't be using the extra argument to recv,
but this gives us only one place to worry about respecting timeouts and one
consistent timeout behavior.

stand: Make pc98 build

This is a direct commit to stable/11 due to pc98 removal from head.

Several improvements in head that were MFC'd left the pc98 build broken in
stable/11. Massage the pc98 bits to get it to at least build.

Revert part of r330123

The SUBDIR+= efi removal was part of local testing; revert that part.

This is a direct commit to stable/11.

MFC r330004: Add NO_OBJ to those directories that don't make anything.

For directories that don't many anything, add NO_OBJ=t just before we
include bsd.init.mk. This prevents them from creating an OBJ
directory. In addition, prevent defs.mk from creating the machine
related links in these cases. They aren't needed and break, at least
on stable, the read-only src tree build.

PR: 226074

MFC r323067, r323184, r323185, r323195, r323196 (by cem)

------------------------------------------------------------------------
r323067 | cem | 2017-08-31 11:39:18 -0700 (Thu, 31 Aug 2017) | 4 lines

amdtemp.4: Update BKDG URL to current location

Sponsored by: Dell EMC Isilon

------------------------------------------------------------------------
r323184 | cem | 2017-09-05 08:13:41 -0700 (Tue, 05 Sep 2017) | 10 lines

Add smn(4) driver for AMD System Management Network

AMD Family 17h CPUs have an internal network used to communicate between
the host CPU and the PSP and SMU coprocessors. It exposes a simple
32-bit register space.

Reviewed by: avg (no +1), mjoras, truckman
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12217

------------------------------------------------------------------------
r323185 | cem | 2017-09-05 08:19:14 -0700 (Tue, 05 Sep 2017) | 13 lines

amdtemp(4): Add support for Family 17h temperature sensor

The sensor value is formatted similarly to previous models (same
bitfield sizes, same units), but must be read off of the internal
System Management Network (SMN) from the System Management Unit (SMU)
co-processor.

PR: 218264
Reported and tested by: Nils Beyer <nbe AT renzel.net>
Reviewed by: avg (no +1), mjoras, truckman
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12217

------------------------------------------------------------------------
r323195 | cem | 2017-09-05 13:35:25 -0700 (Tue, 05 Sep 2017) | 8 lines

amdtemp(4): Do not probe not matching hostbridges

Some systems have hostbs that do not match our PCI device id criteria.
Detect and ignore these devices in probe.

PR: 218264
Sponsored by: Dell EMC Isilon

------------------------------------------------------------------------
r323196 | cem | 2017-09-05 14:00:33 -0700 (Tue, 05 Sep 2017) | 8 lines

amdsmn(4): Do not probe not matching hostbridges

Similar to r323195, but for amdsmn(4) driver (which borrowed some design).

Ignore hostbs that do not match our PCI device id criteria.

Sponsored by: Dell EMC Isilon

PR: 218264
Differential Revision: https://reviews.freebsd.org/D12217

MFC r329264: libsa: Fix IP recv timeout

[This is slightly modified to not set `t` in the middle of the loop so that
the connection will eventually timeout after MAXTMO]

readip() doesn't, at the moment, properly indicate to callers that it has
timed out. One can tell that it's timed out if errno == EAGAIN when it
returns, but this is not ideal. Restructure it a little bit to explicitly
set errno to ETIMEDOUT if we've exhausted tleft.

I found two places that care about where it timed out or not: sendrecv in
net.c and sendrecv_tftp. Both are structured to pass smaller timeout values
to readip while tracking a larger timeout. Neither of them were able to do
this properly with readip not indicating ETIMEDOUT, so fix it.

While here, straighten out the time (t/t1) usage in sendrecv_tftp.

This would have manifested itself in periodic failures to NFS/TFTP boot for
no apparent reason because MINTMO/MAXTMO were not actually being respected
properly. Problems were not reported with NFS, only TFTP.

stand: Clean up some unintentional inconsistencies

This is a direct commit to stable/11 to address the following final
unintended inconsistencies between stable/11 and head:

- Some unused LIBSTAND= cruft left in efi/loader/Makefie
- A comment that flew in with unrelated changes
- An #include that may go away now that stand is basically self-contained

Revert MFC r328911 from r329183

jhb@ had specified an MFC timer of 1-month, and I clobbered his efforts by
MFC'ing it way prematurely. Back out that part of r329183 so it may be MFC'd
when jhb is ready.

MFC Loader Fixes Final: r327612,r327703,r327704,r327878,r327879,r327881,
r328007,r328029,r328030,r328031,r328061,r328156,r328169,r328288,r328289,
r328290,r328291,r328292,r328411,r328536,r328603,r328614,r328642,r328769,
r328779,r328780,r328781,r328782,r328783,r328806,r328808,r328826,r328835,
r328911,r328986,r328987,r328990,r328999,r329000,r329019,r329050,r329054,
r329060

r327612: Invent new #defines for the biospci_{read,write}_config function

r327703: Define __dmadat after #include'ing ufsread.c.

r327704: Fix printf missing format variables warnings.

r327878: Add GUID for UEFI boot manager variables.

r327879: Report the boot order and where we are in that boot order.

r327881: Allow this file to be included

r328007: Fix booting on some arm64 systems after r327879 by fixing the call
to utf8_to_ucs2

r328029: When returning an error and freeing allocated memory from
ucs2_to_utf8, NULL the return pointer.

r328030: Check the return value from utf8_to_ucs2 instead of whether or not
uv is NULL.

r328031: Need to free uv after we're done using it.

r328061: utf8_to_ucs2() should check for malloc failure

r328156: stand: Move sections around to fix stand/ build with ld.lld on
armv7

r328169: Remove extra copy of bootinfo.c. It's a bit rotted copy of the one
in efi/loader.

r328288: Fix some resource leaks.

r328289: Don't leak memory when displaying help.

r328290: On malloc failure, be sure to close the include file that triggered
it.

r328291: getenv does not return tainted data in the boot loader. Attempt to
clue Coverity into that fact.

r328292: There's no tainted data here, tag it as such to avoid false
positives.

r328411: loader.efi: add missing EFI GUIDs

r328536: loader: support for mixed-endianness ELF/loader and POWER8

r328603: Add missing non-POWERPC case to give the scr value something
non-zero.

r328614: Move libstand.3 to libsa.3. Update libsa.3 to include functions

r328642: Break out the interpreters (simple and forth) w/o ifdefs.

r328769: Centralize several variables.

r328779: Retire pnp.4th and the code needed only for 4th words used here.

r328780: These 4th words were an attempt to allow integration into the boot
loader scripts. However, that path won't be taken after all it
seems.

r328781: Remove pcibios forth support.

r328782: Now that we no longer conditionally compile some files outside of
ficl

r328783: Invent new LDR_INTERP for the loader interpreter to use.

r328806: We need more heap space to properly load newer powerpc kernels.

r328808: Implement strcoll as strcmp.

r328826: Make cross-endian loader changes apply only to powerpc

r328835: Fix regression introduced in r328806, preventing boot on many
platforms.

r328911: Ignore relocation tables for non-memory-resident sections.

r328986: Fix relative location of USB sources after recent move.

r328987: A more definitions to kernel emulation shim in order to build
stand/usb.

r328990: Move the stand/usb test loader into its own directory.

r328999: Fix indentation to FreeBSD standard for interp files

r329000: Move simple interpreter 'perform' into interp.c and call it

r329019: Move to tabs for indentation and to 8-space notches, per style(9).

r329050: Fix build of userboot.so

r329054: Set script.lang in the environment to either 'forth' or 'simple' to
reflect what scripting language was compiled into the loader.

r329060: loader: fix endianness conversion

PR: 225323

MFC r327705,r328447: Fix BERI bootloader build

r327705: Use <stand.h> instead of <inttypes.h> and <stdio.h> in boot code.
r328447: BERI isn't BTX, so we don't have to provide exit().

MFC Loader Fixes 2017q4p7: r324844,r326089,r326926,r326440,r326484,r326494,
r326588,r326708,r326784,r326914,r327390,r328446,r326090,r326143,r326144,
r326182,r326384,r326421,r326440,r326441,r326442,r326443,r326444,r326445,
r326446,r326447,r326448,r326484,r326485,r326486,r326487,r326488,r326490,
r326491,r326492,r326493,r326494,r326495,r326504,r326507,r326509,r326584,
r326585,r326586,r326587,r326588,r326589,r326590,r326591,r326592,r326593,
r326594,r326600,r326616,r326671,r326707,r326708,r326709,r326710,r326711,
r326712,r326714,r326720,r326768,r326772,r326784,r326792,r326812,r326854,
r326855,r326856,r326858,r326886,r326887,r326914,r326926,r326927,r326960,
r326961,r326962,r326963,r327351,r327453,r327390,r327523,r327524,r326489,
r327880,r328437,r328438,r328439,r328441,r328446,r328448,r328449,r328612,
r328613,r328615

While here, undo our libfdt hack of not including <stdlib.h> if we're
compiling _STANDALONE.

r324844: When building standalone, don't define errno. Let the definition from
stand.h override. This is similar to what we do in the kernel.

r326089: loader.efi: efipart does not recognize partitionless disks

r326090: net_parse_rootpath() has no parameters

r326143: Fix theoretical integer overflow issues. If the product here is

r326144: Mark the func pointer as __dead2. It looks up loader_main, which

r326182: Modify all FreeBSD bootloaders on PowerPC AIM (Book-S) systems

r326384: Use const pointers to avoid casting away constness.

r326421: loader.efi: efipart should exclude iPXE stub block protocol

r326440: Remove stale dependency on ufsread.c

r326441: Minor flags cleanup

r326442: Cleanup CFALGS usage here

r326443: We don't need both _STAND and _STANDALONE, use the latter.

r326444: Move geli to common DO32 stuff

r326445: Fix random() and srandom() prototypes to match the standard.

r326446: Undefine _STANDALONE since this is test code.

r326447: Tweaks to the beri boot loader so that it builds w/o warnings.

r326448: Fix all warnings related to geli and ZFS support on x86.

r326484: Const poison the propname.

r326485: Delcare md_load in libofw.h. Make all prototypes match for ofw

r326486: Include machine/md_var to pick up __syncicache prototype.

r326487: Cast mdp (a vm_offset_t) to void * to match prototype.

r326488: e_entry can be smaller than a pointer. Cast it to an intptr_t

r326490: Declare our strange brand of main().

r326491: Disconnet ps3 from the build. There's too many warnings to fix.

r326492: Cast void * pointer to char * so the arg matches the %s format.

r326493: Provide a md_load64 prototype.

r326494: Mark two things as unused (since they are only sometimes used)

r326495: Now it's safe to bump WARNS to 1.

r326504: Switch to proper MK_LOADER_GELI tests.

r326507: increase maximum size of zfsboot

r326509: loader.efi: add note about iPXE into the efipart.c

r326584: When building standalone, include stand.h rather than the kernel

r326585: Include ficl.h before anything else

r326586: No need to include the userland md5.h, the kernel one is just fine.

r326587: Use the kernel relative paths, rather than the userland relative
paths

r326588: Need to include skein in the include path

r326589: Make sure we include the right path for skein.h

r326590: Prefer stdint.h to inttypes.h

r326591: This isn't NetBSD specific code. Include these for any kernel /

r326592: Don't inherit CFLAGS. This a specialized test program.

r326593: Stop building with the standard system headers.

r326594: Now that we offer a semi-sane standards-ish set of #include files,
stop hacking includes with sed.

r326600: Since this is contrib code, create an upstreamable version of my

r326616: dhcp_try_rfc1048() is not used any more

r326671: Avoid setting -Wno-tentative-definition-incomplete-type with gcc.

r326707: Add partial support signal.h functioanlity. Pull in
machine/signal.h

r326708: Remove _KERNEL hack now that errno.h does the right thing when
built standalone.

r326709: Provide implementations for iscntrl, ispunct and isgraph.

r326710: Put the files we're copying over into a few variables and clean
hings up.

r326711: Const poison a couple of interfaces.

r326712: Create interp class.

r326714: boot1.c needs EFI_ZFS_BOOT too, so add it globally.

r326720: This path belongs in ficl/Makefile, not the common defines for
users

r326768: Fix a comment to be more accurate

r326772: Fix regression with lua import

r326784: Revert part of 362772. It was causing problems for includes

r326792: Attempt to unbreak buildworld

r326812: Revert r326792, r326784, r326772, r326712

r326854: libefi: make efichar.h more usable in stand code

r326855: Cargo cut a fix for the regressions r326585 caused.

r326856: Fix comments after bump in size.

r326858: Revert r326855: Cargo cut a fix for the regressions r326585 caused.

r326886: Panic in sbrk if setheap hasn't been called yet. This is preferable
o a mysterious crash

r326887: Remove the 'mini libstand in libstand' that util.[ch] provided.

r326914: Move loader-only defines to loader.mk from defs.mk

r326926: Move loader help file definitions to being 100% inside of
loader.mk.

r326927: libficl is only ever used in a loader (never a boot) program. Move
it.

r326960: Simplify things a little. The RETURN macro isn't required.

r326961: Interact is always called with NULL. Simplify code a little

r326962: Hoist btx include stuff to i386/Makefile.inc

r326963: No need to use relative paths like this here.

r327351: Fix ubldr. uboot/lib uses defines for the loader.

r327453: Add a validbcd() routine that uses the bcd2bin_data[] array

r327390: Garbage-collect loader.ps3. It is currently disconnected from the
build and kboot replaces.

r327523: Don't clobber system LDFLAGS for beri boot loaders.

r327524: Use 'extern uint8_t' instead of 'extern void' for external symbols.

r326489: Allow this file to be used in libsa without warning...

r327880: Move getsecs() prototype to stand.h from net.h so it can be used

r328437: Split panic routine

r328438: Implement abort() as a call to panic.

r328439: Provide abs form stdlib.h.

r328441: abort() should be marked __dead2 since it won't return.

r328446: Now that exit is __dead2, we need to tag ub_exit() as __dead2.

r328448: Make exit() never return until host_exit can be written.

r328449: Tag unreachable places as such. I left the while (1); in place

r328612: Move strtold wrapper from strtol.c to its own strtold.c.

r328613: Kill copies of strtol and strtoul.

r328615: Update stand.h for changes for strto*l

PR: 223969