MFC r359688: stand: -fno-common fixes for !x86 loaders

- beriloader: archsw is declared extern and defined elsewhere
- ofwloader: ofw_elf{,64} are defined in elf_freebsd.c and
ppc64_elf_freebsd.c respectively
- ubldr: syscall_ptr is defined in start.S for whichever ubldr platform is
building

-fno-common will become the default in GCC10/LLVM11.

MFC r356538: stand/fdt: Scale blob size better as overlays apply

Currently, our overlay blob will grow to include the size of the complete
overlay blob we're applying. This doesn't scale very well with a lot of
overlays- they tend to include a lot of overhead, and they will generally
only add a fraction of their total size to the blob they're being applied
to.

To combat this, pack the blob as we apply new overlays and keep track of how
many overlays we've applied. Only ubldr has any fixups to be applied after
overlays, so we only need to re-pad the blob in ubldr. Presumably the
allocation won't fail since we just did a lot worse in trying to apply
overlays and succeeded.

I have no intention of removing the padding in make_dtb.sh. There might be
an argument to be had over whether it should be configurable, since ubldr
*is* the only loader that actually has fixups to be applied and we can do
this at runtime, but I'm not too concerned about this.

This diff has been sitting in Phabricator for a year and a half, but I've
decided to flush it as it does make sure that we're scaling the blob
appropriately and leave room at the end for fixups in case of some freak
circumstance where applying overlays leaves us with a blob of insufficient
size.

MFC r343911, r344238-r344241, r344247, r344254-r344255, r344260, r344268,
r344335, r344839, r345066, r345330

r343911:
Allow reading the UEFI variable size

When loading bigger variables form UEFI it is necessary to know their
size beforehand, so that an appropriate amount of memory can be
allocated. The easiest way to do this is to try to read the variable
with buffer size equal 0, expecting EFI_BUFFER_TOO_SMALL error to be
returned. Allow such possible approach in efi_getenv routine.

Extracted from a bigger patch as suggested by imp.

r344238:
Restore loader(8)'s ability for lsdev to show partitions within a bsd slice.

I'm pretty sure this used to work at one time, perhaps long ago. It has
been failing recently because if you call disk_open() with dev->d_partition
set to -1 when d_slice refers to a bsd slice, it assumes you want it to
open the first partition within that slice. When you then pass that open
dev instance to ptable_open(), it tries to read the start of the 'a'
partition and decides there is no recognizable partition type there.

This restores the old functionality by resetting d_offset to the start
of the raw slice after disk_open() returns. For good measure, d_partition
is also set back to -1, although that doesn't currently affect anything.

I would have preferred to make disk_open() avoid such rude assumptions and
if you ask for partition -1 you get the raw slice. But the commit history
shows that someone already did that once (r239058), and had to revert it
(r239232), so I didn't even try to go down that road.

r344239:
Use a couple local variables to avoid repetitive long expressions that
cause line-wrapping.

r344240:
Make lsdev -v output line up in neat columns by using a fixed width for
the size field and a tab between the partition type and the size.

Changes this

disk devices:
disk0 (MMC)
disk0s1: DOS/Windows 49MB
disk0s2: FreeBSD 14GB
disk0s2a: FreeBSD UFS 14GB
disk0s2b: Unknown 2048KB
disk0s2d: FreeBSD UFS 2040KB

to this

disk devices:
disk0 (MMC)
disk0s1: DOS/Windows 49MB
disk0s2: FreeBSD 14GB
disk0s2a: FreeBSD UFS 14GB
disk0s2b: Unknown 2048KB
disk0s2d: FreeBSD UFS 2040KB

r344241:
Garbage collection no-longer-used constant.

r344247:
Make uboot_devdesc properly alias disk_devdesc, so that parsing the u-boot
loaderdev variable works correctly.

The uboot_devdesc struct is variously cast back and forth between
uboot_devdesc and disk_devdesc as pointers are handed off through various
opaque interfaces. uboot_devdesc attempted to mimic the layout of
disk_devdesc by having a devdesc struct, followed by a union of some
device-specific stuff that included a struct that contains the same fields
as a disk_devdesc. However, one of those fields inside the struct is 64-bit
which causes the entire union to be 64-bit aligned -- 32 bits of padding
is added between the struct devdesc and the union, so the whole mess ends
up NOT properly mimicking a disk_devdesc after all. (In disk_devdesc there
is also 32 bits of padding, but it shows up immediately before the d_offset
field, rather than before the whole collection of d_* fields.)

This fixes the problem by using an anonymous union to overlay the devdesc
field uboot network devices need with the disk_devdesc that uboot storage
devices need. This is a different solution than the one contributed with
the PR (so if anything goes wrong, the blame goes to me), but 95% of the
credit for this fix goes to Pawel Worach and Manuel Stuhn who analyzed the
problem and proposed a fix.

r344254:
Use DEV_TYP_NONE instead of -1 to indicate no device was specified.

DEV_TYP_NONE has a value of zero, which makes more sense since the device
type is a bunch of bits describing the device, crammed into an int.

r344255:
Fix more places to use DEV_TYP_NONE instead of -1 to indicate 'no device'.

r344260:
Allow the u-boot loaderdev env var to be formatted in the "usual" loader(8)
way: device<unit>[s|p]<slice><partition>. E.g., disk0s2a or disk3p12.
The code first tries to parse the variable in this format using the
standard disk_parsedev(). If that fails, it falls back to parsing the
legacy format that has been supported by ubldr for years.

In addition to 'disk', all the valid uboot device names can also be used:
mmc, sata, usb, ide, scsi. The 'disk' device serves as an alias for all
those types and will match the Nth storage-type device found (where N is
the unit number).

r344268:
loader: ptable_close() should check its argument

If the passed in table is NULL, just return.

r344335:
Fix the handling of legacy-format devices in the u-boot loaderdev variable.
When I added support for the standard loader(8) disk0s2a: type formats,
the parsing of legacy format was broken because it also contains a colon,
but it comes before the slice and partition. That would cause disk_parsedev()
to return success with the slice and partition set to wildcard values.

This change examines the string first, and if it contains spaces, dots, or
a colon at any position other than the end, it must be a legacy-format
string and we don't even try to use disk_parsedev() on it.

r344839:
Add retry loop around GetMemoryMap call to fix fragmentation bug

The call to BS->AllocatePages can cause the memory map to become framented,
causing BS->GetMemoryMap to return EFI_BUFFER_TOO_SMALL more than once. For
example this can happen on the MinnowBoard Turbot, causing the boot to stop
with an error. Avoid this by calling GetMemoryMap in a loop.

r345066:
stand: Improve some debugging experience

Some of these files using <FOO>_DEBUG defined a DEBUG() macro to serve as a
debug-printf. -DDEBUG is useful to enable some debugging output across
multiple ELF/common parts, so switch the DEBUG-as-printf macros over to
something more like DPRINTF that is more commonly used for this kind of
thing and less likely to conflict.

userboot/elf64_freebsd debugging also assumed %llx for uint64; use PRIx64
instead.

r345330:
loader: fix loading of kernels with . in path

The loader indended to search the kernel file name (only) for . but
instead searched the entire path, so paths like
"boot/test.elfv2/kernel" would not work.

PR: 233097

MFC r341253, r341328, r342619, r342626, r342707, r342785, r342865

r341253:
The libstand's panic() appends its own '\n' to the message, so that users of the API
don't need to supply one.

r341328:
loader: create separate lists for fd, cd and hd, merge bioscd with biosdisk

Create unified block IO implementation in BIOS version, like it is done in UEFI
side. Implement fd, disk and cd device lists, this will split floppy devices
from disks and will allow us to have consistent, predictable device naming
(modulo BIOS issues).

r342619:
loader: create bio_alloc and bio_free for bios bounce buffer

We do have 16KB buffer space defined in pxe.c, move it to bio.c and implement
bio_alloc()/bio_free() interface to make it possible to use this space for
other BIOS calls (notably, from biosdisk.c).

r342626:
Add Copyright.

r342707:
i386_parsedev() needs to support fd devices

r342785:
With buggy int13 ah=15, we can mis-identify the floppy devices.

We have no option than trust INT13 ah=08 return code during the init phase.

r342865:
biospci_write_config args were backwards

biospci_write_config args swapped length and value to write. Some
hardware coped just fine, while other hardware had issues.

PR: 155441, 234460

MFC r346132: stand: refactor overlay loading a little bit

It was pointed out that manually loading a .dtb to be used rather than
relying on platform-specific method for loading .dtb will result in overlays
not being applied. This was true because overlay loading was hacked into
fdt_platform_load_dtb, rather than done in a way more independent from how
the .dtb is loaded.

Instead, push overlay loading (for now) out into an
fdt_platform_load_overlays. This method easily allows ubldr to pull in any
fdt_overlays specified in the ub env, and omits overlay-checking on
platforms where they're not tested and/or not desired (e.g. powerpc). If we
eventually stop caring about fdt_overlays from ubenv (if we ever cared),
this method should get chopped out in favor of just calling
fdt_load_dtb_overlays() directly.

MFC r335869, r335933: stand: uboot behavioral fixes

r335869:
stand: uboot: Do not panic if we can't find a boot device

It is really anoying to panic when there is no boot device as you
cannot see the availables ones.

r335933:
loader: fdt: Try to load every possible DTB from u-boot

U-Boot setup a few variables :

- fdt_addr which is the board static dtb (most of the time loaded before
u-boot or coming from some hardware like a ROM)
- fdt_addr_r which is a location in RAM that holds the DTB loaded by
u-boot or before u-boot

In the case of u-boot + rpi firmware the DTB is loaded in RAM but the location
still end up in the fdt_addr variable and the fdt_addr_r variable exist.

Change the behavior so we test that a DTB exists for every possible variable :

- fdt_addr_r is checked first as if u-boot needed to modify it the
correct DTB will live there.
- fdt_addr is checked second as if we run on a hardware with DTB in ROM
it means that we what/need to run that
- fdtaddr looks like a FreeBSD-ism but since I'm not sure leave it.

MFC r335117: stand bootprog_info consolidation

bootprog_info is generated in vers.c. Move it's definition to
bootstrap.h and remove all the redundant copies.

ubldr: Bump heap size, 1MB -> 2MB

1MB was leaving very little margin in some of the worse-case scenarios with
lualoader. 2MB is still low enough that we shouldn't have any problems with
UBoot-supported boards.

MFC r334656, r334665, r334695

r334656:
Add vsnprintf() to libsa. Alpha-sort the printf prototypes in stand.h.

r334665:
Make the v*printf() functions in libsa return int instead of void.
This makes them compatible with the C standard signatures, avoiding
spurious mismatch errors in the places where the oddball requirements
of standalone code end up putting two declarations of the same function
in play.

r334695:
Remove comments and assertions that are no longer valid after r330809.

r330809 replaced duplication of devdesc struct fields with an embedded copy
of the devdesc struct, to avoid fragility. That means all the scattered
comments indicating that structs must match are no longer valid. Likewise
asserts that attempted to mitigate some of the old fragility.

Reviewed by: imp@

MFC r330806-r330815, r330837, r330864, r330883

pc98 changes associated with the named commits are also included in this
commit, despite not having been made with the original commits due to its
removal in head.

r330806:
Minor cosmetic changes.

Make sure { on the same line as struct for all struct *devdesc. Move
some type definitions to next to the dv_type define, since that's what
sets the d_type.

r330807:
We can't use d_opendata for blkio storage.

open_disk uses d_opendata for it's own purpse. We can't store blkio
there. Fortunately, blkio is stored elsewhere and we never actually
retrieve blkio from d_opendata. Eliminate it as a source of confusion.
Eliminate all stores of d_opendata in efi since this layer doesn't own
that field.

r330808:
Make struct libi386_devdesc match the struct devdesc better

Move data to top and call it d_opendata.

r330809:
Use the actual struct devdesc at the start of all *_devdesc structs

The current system is fragile and requires very careful layout of all
*_devdesc structures. It also makes it hard to change the base
devdesc. Take a page from CAM and put the 'header' in all the derived
classes and adjust the code to match.

For OFW, move the iHandle h_handle out of a slot conflicting with
d_opendata. Due to quirks in the alignment rules, this worked.
However changing the code to use d_opendata storage now that it's a
pointer is hard, so just have a separate field for it.

All other cleanups were to make the *_devdesc structures match where
they'd taken some liberties that were none-the-less compatible enough
to work.

r330810:
Remove d_type from devdesc. It's not needed as we can fetch it from
d_dev->dv_type when we need it.

r330811:
GC unused routines.

Sponsored by: Netflix

r330812:
Use the one-line-per-file pattern here, and sort the file names.

Sponsored by: Netflix

r330813:
Move the env convenience routines out of boot1.c.

These routines are more generally useful. Even though boot1 is on its
way out, it's better to make these common during the transition than
copy them.

r330814:
Star BootCurrent entry when booting.

Sponsored by: Netflix

r330815:
Print the load and device path as well as BootCurrent and BootOrder

Sponsored by: Netflix

r330837:
biosdisk.c should not set d_opendata.

Same as 330807, d_opendata is owned by open_disk and we should not
set it.

M stand/i386/libi386/biosdisk.c

r330864:
Prefer uintXX_t to u_intXX_t

A foolish consistency is the hobgoblin of little minds, adored by
little statesmen and philosophers and divines. With consistency a
great soul has simply nothing to do. -- Ralph Waldo Emerson

r330883:
Fix typo that misteriously passes compilation.

MFC r329190, r329315, r330131: metadata load file unification

r329190: Unify metadata load files for arm, mips, powerpc, sparc64

Summary:
All metadata.c files are very similar, with only trivial changes. Unify them
into a single common file, with minor special-casing where needed.

r329315: stand: Fix ubldr after r329190

metadata load files were consolidated in r329190, and these relocation fixup
bits were inadvertently dropped in the process. Re-add them to fix boot with
ubldr.

r330131: Fix module loading on arm after the metadata.c unification in
r329190. Arm modules need an additional address fixup not needed by other
platforms.

MFC r330249-r330250

r330249:
stand: Makefile SUBDIR cleanup

Use SUBDIR.${MK_*} where appropriate. r330248 eliminated most of the
offenders, sweep the rest under the rug.

Differential Revision: https://reviews.freebsd.org/D14545

r330250:
stand: Fix build after r330249

One does not simply convert to SUBDIR.yes in stand without making everything
else in the affected files SUBDIR.yes -- there are better ways to do this.

MFC r329345: Eliminate bsd.stand.mk and -fPIC 32-bit intel builds

OK. We don't really need a bsd.stand.mk, and it was causing a -fPIC
for the toolchain to be added (bogusly) when building on amd64. Pull
all relevant defs back into defs.mk and delete bsd.stand.mk.

This saves about 15-20k on i386 loader and zfsloader which when
combined with Lua give us a lot more stack space in those constrained
environments.

MFC r330891: ubldr: Bump heap size from 512K to 1M

lualoader in itself only uses another ~200K, but there seems to be no reason
not to bump it a little higher to give us some more wiggle room.

With this, I can boot using a menu-enabled lualoader, no problem and
reasonably fast. Some heap usage datapoints from the review:

forthloader, no menus, kernel loaded:
heap base at 0x1203d5b0, top at 0x1208e000, used 330320

lualoader, no menus, kernel loaded:
heap base at 0x42050028, top at 0x420ab000, used 372696

lualoader, menus, kernel loaded:
heap base at 0x42050028, top at 0x420d5000, used 544728

Since then, the no menu case for lualoader should have decreased slightly as
I've made some changes to make sure that it no longer loads any of th emenu
bits with beastie disabled.

While here, split heap size out into a HEAP_SIZE macro.

MFC Loader Fixes Final: r327612,r327703,r327704,r327878,r327879,r327881,
r328007,r328029,r328030,r328031,r328061,r328156,r328169,r328288,r328289,
r328290,r328291,r328292,r328411,r328536,r328603,r328614,r328642,r328769,
r328779,r328780,r328781,r328782,r328783,r328806,r328808,r328826,r328835,
r328911,r328986,r328987,r328990,r328999,r329000,r329019,r329050,r329054,
r329060

r327612: Invent new #defines for the biospci_{read,write}_config function

r327703: Define __dmadat after #include'ing ufsread.c.

r327704: Fix printf missing format variables warnings.

r327878: Add GUID for UEFI boot manager variables.

r327879: Report the boot order and where we are in that boot order.

r327881: Allow this file to be included

r328007: Fix booting on some arm64 systems after r327879 by fixing the call
to utf8_to_ucs2

r328029: When returning an error and freeing allocated memory from
ucs2_to_utf8, NULL the return pointer.

r328030: Check the return value from utf8_to_ucs2 instead of whether or not
uv is NULL.

r328031: Need to free uv after we're done using it.

r328061: utf8_to_ucs2() should check for malloc failure

r328156: stand: Move sections around to fix stand/ build with ld.lld on
armv7

r328169: Remove extra copy of bootinfo.c. It's a bit rotted copy of the one
in efi/loader.

r328288: Fix some resource leaks.

r328289: Don't leak memory when displaying help.

r328290: On malloc failure, be sure to close the include file that triggered
it.

r328291: getenv does not return tainted data in the boot loader. Attempt to
clue Coverity into that fact.

r328292: There's no tainted data here, tag it as such to avoid false
positives.

r328411: loader.efi: add missing EFI GUIDs

r328536: loader: support for mixed-endianness ELF/loader and POWER8

r328603: Add missing non-POWERPC case to give the scr value something
non-zero.

r328614: Move libstand.3 to libsa.3. Update libsa.3 to include functions

r328642: Break out the interpreters (simple and forth) w/o ifdefs.

r328769: Centralize several variables.

r328779: Retire pnp.4th and the code needed only for 4th words used here.

r328780: These 4th words were an attempt to allow integration into the boot
loader scripts. However, that path won't be taken after all it
seems.

r328781: Remove pcibios forth support.

r328782: Now that we no longer conditionally compile some files outside of
ficl

r328783: Invent new LDR_INTERP for the loader interpreter to use.

r328806: We need more heap space to properly load newer powerpc kernels.

r328808: Implement strcoll as strcmp.

r328826: Make cross-endian loader changes apply only to powerpc

r328835: Fix regression introduced in r328806, preventing boot on many
platforms.

r328911: Ignore relocation tables for non-memory-resident sections.

r328986: Fix relative location of USB sources after recent move.

r328987: A more definitions to kernel emulation shim in order to build
stand/usb.

r328990: Move the stand/usb test loader into its own directory.

r328999: Fix indentation to FreeBSD standard for interp files

r329000: Move simple interpreter 'perform' into interp.c and call it

r329019: Move to tabs for indentation and to 8-space notches, per style(9).

r329050: Fix build of userboot.so

r329054: Set script.lang in the environment to either 'forth' or 'simple' to
reflect what scripting language was compiled into the loader.

r329060: loader: fix endianness conversion

PR: 225323

MFC Loader Fixes 2017q4p7: r324844,r326089,r326926,r326440,r326484,r326494,
r326588,r326708,r326784,r326914,r327390,r328446,r326090,r326143,r326144,
r326182,r326384,r326421,r326440,r326441,r326442,r326443,r326444,r326445,
r326446,r326447,r326448,r326484,r326485,r326486,r326487,r326488,r326490,
r326491,r326492,r326493,r326494,r326495,r326504,r326507,r326509,r326584,
r326585,r326586,r326587,r326588,r326589,r326590,r326591,r326592,r326593,
r326594,r326600,r326616,r326671,r326707,r326708,r326709,r326710,r326711,
r326712,r326714,r326720,r326768,r326772,r326784,r326792,r326812,r326854,
r326855,r326856,r326858,r326886,r326887,r326914,r326926,r326927,r326960,
r326961,r326962,r326963,r327351,r327453,r327390,r327523,r327524,r326489,
r327880,r328437,r328438,r328439,r328441,r328446,r328448,r328449,r328612,
r328613,r328615

While here, undo our libfdt hack of not including <stdlib.h> if we're
compiling _STANDALONE.

r324844: When building standalone, don't define errno. Let the definition from
stand.h override. This is similar to what we do in the kernel.

r326089: loader.efi: efipart does not recognize partitionless disks

r326090: net_parse_rootpath() has no parameters

r326143: Fix theoretical integer overflow issues. If the product here is

r326144: Mark the func pointer as __dead2. It looks up loader_main, which

r326182: Modify all FreeBSD bootloaders on PowerPC AIM (Book-S) systems

r326384: Use const pointers to avoid casting away constness.

r326421: loader.efi: efipart should exclude iPXE stub block protocol

r326440: Remove stale dependency on ufsread.c

r326441: Minor flags cleanup

r326442: Cleanup CFALGS usage here

r326443: We don't need both _STAND and _STANDALONE, use the latter.

r326444: Move geli to common DO32 stuff

r326445: Fix random() and srandom() prototypes to match the standard.

r326446: Undefine _STANDALONE since this is test code.

r326447: Tweaks to the beri boot loader so that it builds w/o warnings.

r326448: Fix all warnings related to geli and ZFS support on x86.

r326484: Const poison the propname.

r326485: Delcare md_load in libofw.h. Make all prototypes match for ofw

r326486: Include machine/md_var to pick up __syncicache prototype.

r326487: Cast mdp (a vm_offset_t) to void * to match prototype.

r326488: e_entry can be smaller than a pointer. Cast it to an intptr_t

r326490: Declare our strange brand of main().

r326491: Disconnet ps3 from the build. There's too many warnings to fix.

r326492: Cast void * pointer to char * so the arg matches the %s format.

r326493: Provide a md_load64 prototype.

r326494: Mark two things as unused (since they are only sometimes used)

r326495: Now it's safe to bump WARNS to 1.

r326504: Switch to proper MK_LOADER_GELI tests.

r326507: increase maximum size of zfsboot

r326509: loader.efi: add note about iPXE into the efipart.c

r326584: When building standalone, include stand.h rather than the kernel

r326585: Include ficl.h before anything else

r326586: No need to include the userland md5.h, the kernel one is just fine.

r326587: Use the kernel relative paths, rather than the userland relative
paths

r326588: Need to include skein in the include path

r326589: Make sure we include the right path for skein.h

r326590: Prefer stdint.h to inttypes.h

r326591: This isn't NetBSD specific code. Include these for any kernel /

r326592: Don't inherit CFLAGS. This a specialized test program.

r326593: Stop building with the standard system headers.

r326594: Now that we offer a semi-sane standards-ish set of #include files,
stop hacking includes with sed.

r326600: Since this is contrib code, create an upstreamable version of my

r326616: dhcp_try_rfc1048() is not used any more

r326671: Avoid setting -Wno-tentative-definition-incomplete-type with gcc.

r326707: Add partial support signal.h functioanlity. Pull in
machine/signal.h

r326708: Remove _KERNEL hack now that errno.h does the right thing when
built standalone.

r326709: Provide implementations for iscntrl, ispunct and isgraph.

r326710: Put the files we're copying over into a few variables and clean
hings up.

r326711: Const poison a couple of interfaces.

r326712: Create interp class.

r326714: boot1.c needs EFI_ZFS_BOOT too, so add it globally.

r326720: This path belongs in ficl/Makefile, not the common defines for
users

r326768: Fix a comment to be more accurate

r326772: Fix regression with lua import

r326784: Revert part of 362772. It was causing problems for includes

r326792: Attempt to unbreak buildworld

r326812: Revert r326792, r326784, r326772, r326712

r326854: libefi: make efichar.h more usable in stand code

r326855: Cargo cut a fix for the regressions r326585 caused.

r326856: Fix comments after bump in size.

r326858: Revert r326855: Cargo cut a fix for the regressions r326585 caused.

r326886: Panic in sbrk if setheap hasn't been called yet. This is preferable
o a mysterious crash

r326887: Remove the 'mini libstand in libstand' that util.[ch] provided.

r326914: Move loader-only defines to loader.mk from defs.mk

r326926: Move loader help file definitions to being 100% inside of
loader.mk.

r326927: libficl is only ever used in a loader (never a boot) program. Move
it.

r326960: Simplify things a little. The RETURN macro isn't required.

r326961: Interact is always called with NULL. Simplify code a little

r326962: Hoist btx include stuff to i386/Makefile.inc

r326963: No need to use relative paths like this here.

r327351: Fix ubldr. uboot/lib uses defines for the loader.

r327453: Add a validbcd() routine that uses the bcd2bin_data[] array

r327390: Garbage-collect loader.ps3. It is currently disconnected from the
build and kboot replaces.

r327523: Don't clobber system LDFLAGS for beri boot loaders.

r327524: Use 'extern uint8_t' instead of 'extern void' for external symbols.

r326489: Allow this file to be used in libsa without warning...

r327880: Move getsecs() prototype to stand.h from net.h so it can be used

r328437: Split panic routine

r328438: Implement abort() as a call to panic.

r328439: Provide abs form stdlib.h.

r328441: abort() should be marked __dead2 since it won't return.

r328446: Now that exit is __dead2, we need to tag ub_exit() as __dead2.

r328448: Make exit() never return until host_exit can be written.

r328449: Tag unreachable places as such. I left the while (1); in place

r328612: Move strtold wrapper from strtol.c to its own strtold.c.

r328613: Kill copies of strtol and strtoul.

r328615: Update stand.h for changes for strto*l

PR: 223969

MFC r326038,r326069,r326072: More stand simplification and fixes

r326038: Move some more common stuff up to Makefile.inc.

r326069: Fix gptzfsboot for cases with GELI.

r326072: Unbreak riscv build in universe.

MFC r325834,r325997,326502: Move sys/boot to stand/

This is effectively a direct commit to stable/11, due to differences between
stable/11 and head. Changes to DTS in sys/boot/fdt/dts were often
accompanied by kernel changes. Many of these were also risc-v updates that
likely had many more dependencies to MFC.

Because of this, sys/boot/fdt/dts remains as-is while everything else in
sys/boot relocates to stand/.

r325834: Move sys/boot to stand. Fix all references to new location

r325997: Remove empty directories.

r326502: Document the sys/boot -> stand move in hier.7 and the top-level README.

MFC Loader Fixes 2017q4p6: r324649,r324650,r324651,r324652,r324653,r324654,
r324700,r324702,r324709,r324717,r324719,r324841,r324842,r324843,r324845,
r324850,r324876,r324877,r324878,r324879,r324880,r324881,r324883,r324980,
r324981,r324982,r324995,r325014,r325093,r325094,r325114,r325170,r325171,
r325172,r325173,r325174,r325175,r325176,r325248,r325286,r325310,r325332,
r325338,r325339,r325376,r325377,r325379,r325380,r325382,r325478,r325479,
r325480,r325482,r325483,r325484,r325485,r325556,r325641,r325681,r325685,
r325686,r325687,r325688,r325689,r325690,r325691,r325692,r325693,r325694,
r325743,r325744,r325748,r325775,r325779,r325780

r324649: Move common/Makefile.inc to sys/boot/loader.mk.

r324650: tweak style

r324651: create defs.mk for common definitions

r324652: Move all the ficl common code into ficl.mk

r324653: LOADER_foo_SUPPORTED

r324654: Explicitly inlcude SYSDIR in the include path

r324700: loader: initialize dv_cleanup in md.c to eliminate clang warning

r324702: We need to include disk.o in libuboot.a when we're building with
support for disk access.

r324709: Revert "Unify boot1 with loader" change r324646

r324717: libsa/ip.c: misplaced comment, ip_v is half char, not ip_p

r324719: libsa/ip: stop read loop on bad fragments

r324841: Use BOOTDIR more consistently in defs.mk rather than repeat
sys/boot.

r324842: Introduce BOOTOBJ: The top level object directory for the boot tree

r324843: Stopgap fix to the mistmatch between LOADER_GELI_SUPPORT and
LOADER_NO_GELI_SUPPORT.

r324845: Use BOOTOBJ and BOOTDIR to find geli includes and libraries.

r324850: Define LIBSA32 to LIBSA on i386 to fix build.

r324876: Move fdt and uboot defines into common uboot.mk.

r324877: End source directories with SRC rather than a hodgepodge of names

r324878: Make at91 boot loader compile again.

r324879: Prefer SRCTOP paths for bits we're grabbing from libc.

r324880: Use BOOTSRC here.

r324881: Use SYSDIR instead of ${.CURDIR}/../..<etc>/sys.

r324883: Use preferred defined paths, rather than relative paths in fdt.

r324980: Use BOOTDIR consistently.

r324981: Move BINDIR definition to defs.mk, and override where it isn't
/boot

r324982: Remove sys/boot/arm/at91 and ixp425

r324995: loader.efi: Make framebuffer commands available for arm64

r325014: Add a 'place holder' arm struct efi_fb until a real one comes

r325093: Define new EFI variables

r325094: Cleanup non-arch Makefiles

r325114: Use defs.mk defins in most MD code

r325170: Use defs.mk values for userboot

r325171: Use defs.mk name and prefer bsd.init.mk

r325172: Remove the -nostdlib stuff I added. Instead, fix LDFLAGS to be
honored correctly with the new Makefile.inc include order.

r325173: We don't need to build a special ficl for userboot.

r325174: Minor cleanup

r325175: For amd64, compile both zfs and zfs32 libraries.

r325176: Actually add zfs32/Makefile

r325248: loader ptblread() is broken with >512B sectors

r325286: efipart_strategy is using wrong offset with >512B sectors

r325310: zfs.c:vdev_read() needs to be careful about large sectors

r325332: loader: re-enable gzip support for x86

r325338: loader: fix BOOTSRC -> BOOTOBJ in a library path

r325339: This used to have bzip2 support too.

r325376: WIP: centralize machine links

r325377: mostly libsa

r325379: Revert "mostly libsa"

r325380: Revert "WIP: centralize machine links"

r325382: Cleanup stray libstand names to be libsa names.

r325478: Powerpc is a 32-bit boot loader.

r325479: Define LIBFICL32 to be libficl.a on i386 and libficl32.a on amd64.

r325480: Use DO32 for all the places that we need to flag we're building a
32-bit version of a library.

r325482: Move machine and other link creation to defs.mk

r325483: MACHINE can never be powerpc64, so cleanup code that thinks it can.

r325484: Prefer bsd.init.mk to src.opts.mk

r325485: Centralize all 32-bit builds on 64-bit platform stuff.

r325556: loader: set options before including bsd.init.mk

r325641: loader.efi: efi_devpath_is_prefix should return bool

r325681: boot1: avoid using NULL device path

r325685: libsa32 isn't needed for i386. It's already a 32-bit platform.

r325686: Simplify this if to a direct assignment.

r325687: Remove all the empty help files from the powerpc build.

r325688: FDT support doesn't make sense for ps3, remove it.

r325689: Remove LOADER_FDT_SUPPORT as a Makefile variable.

r325690: Remove LOADER_ZFS_SUPPORT as a Makefile variable

r325691: Remove useless PNP define here.

r325692: Replace LOADER_FIREWIRE_SUPPORT variable

r325693: Move LOADER_{NO,}_GELI_SUPPORT to MK_LOADER_GELI

r325694: Install the 4th files in sys/boot/forth instead of each loader

r325743: Make sure the proper loader.rc gets installed.

r325744: boot1: also check for NULL device

r325748: Use proper include file.

r325775: Add loader.conf to the list of files that are MD.

r325779: Add /boot/dts to the list of default modules.

r325780: Don't add /boot/dt*s* but /boot/dt*b*. Stupid think-o.

MFC Loader Fixes 2017q4p4: r324552,r324553,r324554,r324555,r324556

r324552: Kill the userboot copy of libstand.

r324553: Rename libstand.a to libsa.a and libstand32.a to libsa32.a

r324554: Move crc32.c, util.c and gpt.c over to libsa.

r324555: Honor CFLAGS modifications in Makefile.inc by using += here.

r324556: Move ufsread.c

MFC Loader Fixes 2017q4p3: r324460,r324471

r324460: Rather than laying whack-a-mole with including the path to stand.h,
always include it.

r324471: Fix typos: `stand alon{e,g}` should be spelled like `standalone`

MFC Loader Fixes 2017q4p1: r324321,r324359,r324360,r324388,r324449,r324450,
r324451,r324452

r324321: This README file was quite relevant for FreeBSD 3 or 4. No more.

r324359: Add efi_devpath_is_prefix

r324360: Encapsulate ZFS preferences into efi_zfs_is_preferred

r324388: Unbreak building efiboot with MK_ZFS == no

r324449: Prefer ${LIBSTAND} to -lstand

r324450: Define SASRC and use it

r324451: Add missing -I${SASRC} in a couple of places so that stand.h
doesn't accidentally come in via host pollution.

r324452: Define LIBSA* and use them instead of overloaded LIBSTAND

MFC Loader Fixes 2017q2: r316437,r316577,r316578,r316585,r316590,r316612,
r316625,r316628,r316654,r316682,r316704,r316771,r317092,r317097,r317099,
r317652,r317785,r317886,r317887,r318142,r318320,r318356,r318678,r318754,
r318982,r318986,r318987,r318988,r318989,r318990,r318991,r318992,r318993,
r318994,r318999,r319083,r319084,r319085,r320011,r320234,r320288,r320304,
r320467,r320482

r316437: Small cleanup to make i386/loader match efi/loader boot environment
code

r316577: loader: part.c cstyle cleanup

r316578: loader: want mechanism to avoid RA with bcache

r316585: loader: zfs reader should check all labels

r316590: libstand/dosfs: cache FAT32 in 128 Kb blocks to save loader memory

r316612: In r298230 the value of HEAP_MIN was changed from 3MB to 64MB.
Correct a comment.

r316625: Do not use -msoft-float with intention of disabling FP on aarch64

r316628: Silence GCC warning by initializing the local variable.

r316654: loader: r316585 did miss userboot update

r316682: loader: r316585 did miss sparc/ofw

r316704: loader.efi: only fetch zfs pool guid for the actual boot device

r316771: loader: Avoid possible overflow via environment variable

r317092: loader: zfs reader vdev_probe should check for minimum device size

r317097: loader: F_READ/F_WRITE should be checked against masked flag

r317099: loader: uboot disk ioctl should call disk_ioctl

r317652: loader.efi: ResetSystem does not use data with EFI_SUCCESS

r317785: zfsboot: drvsize() may be unusable on some systems

r317886: distinguish NFS versus TFTP boot by rootpath

r317887: loader: network read rework

r318142: libstand: NULL pointer dereference in rarp

r318320: loader: add ip layer code into libstand

r318356: libstand: increase nfs max read size to 16k

r318678: Replacing iterating over rootpath by strsep(3).

r318754: Pass -N directly to ld via -Wl rather than passing it to the
compiler driver.

r318982: Pass a "FREEBSD" user-class in PXE dhcp request

r318986: add a comment on vendor index 19 and 20 to avoid confusion

r318987: Support URI scheme for root-path in netbooting

r318988: Always build tftpfs support along with nfs for pxeboot

r318989: Always issue the pxe request

r318990: Partially revert r314948

r318991: Document recent changes on pxeboot

r318992: Capitalize DHCP

r318993: Use the usual FreeBSD spelling for the DHCP user class

r318994: Catch with the change in the user class

r318999: Update the comments concerning net_parse_rootpath to reflect what
it is now

r319083: Followup on the user-class changes

r319084: Small cleanup in dev_net.c

r319085: use the same option list for dhcp discovery and request

r320011: Add chain loader support for loader

r320234: Make structure padding explicit in EFI_MEMORY_DESCRIPTOR

r320288: Allow Clang's integrated assembler to assemble boot0

r320304: loader.efi: Disable smbios for arm

r320467: loader: chain load relocate data declaration is bad

r320482: As with arm64 mark the EFI PE header as allocated on arm.

PR: 218473

MFC Loader Fixes 2017q1: r311458,r312237,r312314,r312374,r312947,r313042,
r313047,r313166,r313328,r313332,r313333,r313337,r313348,r313349,r313389,
r313442,r313451,r313575,r313645,r313710,r314114,r314213,r314275,r314945,
r314948,r315008,r315408,r315427,r315645,r315646,r315648,r315653,r315850,
r316064,r316078,r316079,r316100,r316104,r316111,r316112,r316171,r316279,
r316280,r316287,r316311,r316343,r316424,r316436

r311458: Use compiler driver to link BERI boot loaders

r312237: loader.efi: find_currdev() can leak memory

r312314: loader: move device path definitions to include/efidevp.h

r312374: loader: efi devpath api usage should be more aware of NULL pointers

r312947: Remove "-Xassembler -G0" from CFLAGS.

r313042: loader.efi environment related cleanups

r313047: loader: disk/part api needs to use uint64_t offsets

r313166: loader: libefi/env.c warnings in arm build

r313328: loader: Implement disk_ioctl() to support DIOCGSECTORSIZE and
DIOCGMEDIASIZE.

r313332: loader: bcache read ahead block count should take account the large
sectors

r313333: loader: Replace EFI part devices.

r313337: loader: 313329 missed ZFS guard in loader/main.c

r313348: loader: biosdisk fix for 2+TB disks

r313349: loader: disk io should not use alloca()

r313389: efipart is also using the '%S' printf format, add -Wno-format for
it.

r313442: loader: possible NULL pointer dereference in efipart.c

r313451: loader: possible NULL pointer dereference in bcache.c

r313575: makefs: make the buffer functions look exactly like the kernel ones

r313645: loader: implement MEDIA_FILEPATH_DP support in efipart

r313710: loader: cstyle fixes and DIOCGMEDIASIZE should use uint64_t

r314114: Use LDFLAGS with CC instead of _LDFLAGS.

r314213: Remove control+r handling from geliboot's pwgets()

r314275: Remove unused macro from common/drv.c.

r314945: Some style(9) fixes. No functional changes.

r314948: Try to extract the RFC1048 data from PXE.

r315008: r314948 seems to be missing a variable or two that will break

r315408: loader: remove open_disk cache

r315427: loader: biosdisk should report IO error from INT13

r315645: loader: disk_cleanup was left in userboot_disk.c

r315646: loader: pxe.h constants have wrong values

r315648: libstand: verify value provided by nfs.read_size

r315653: loader: verify the value from dhcp.interface-mtu and use snprintf
o set mtu

r315850: The original author abused Nd (one-line description, used by
makewhatis)

r316064: Fix build with path names with 'align' or 'nop' in them.

r316078: gpt*boot: Save a bit more memory when LOADER_NO_GELI_SUPPORT is
specified

r316079: Simply retire the sedification of the boot2.s file.

r316100: Remove -fno-guess-branch-probability and -fno-unit-at-a-time.

r316104: Use `NO_WCAST_ALIGN` instead of spelling it out as -Wno-cast-align
in CFLAGS

r316111: loader: move bios getsecs into time.c

r316112: loader: ls command should display file types properly

r316171: xfsread inlined uses more space, so remove the inline tag.

r316279: loader: efipart should check disk size from partition table

r316280: loader: simplify efi_zfs_probe and avoid double probing for zfs.

r316287: Remove OLD_NFSV2 from loader and libstand

r316311: Add explicit_bzero() to libstand, and switch GELIBoot to using it

r316343: Implement boot-time encryption key passing (keybuf)

r316424: Fix sparc64 build broken by r316343 and r316076

r316436: Restore EFI boot environment functionality broken in r313333

PR: 216940 217298 217935

MFC r303555,r303556,r303936,r303962,r304317,r304532,r305026,r305107,r305132,
r305178,r305353,r305814,r306159,r306380,r306504: Loader fixes, 2016q3

r303555: bcache should support reads shorter than sector size

r303556: Improve boot loader quote parsing

r303936: Add kernel environment variables under smbios.system

r303962: Add the missing space between .asciz directive and opening quote
for some lines with #ifdef BTXLDR_VERBOSE/#endif

r304317: boot1.efi Free() should check for NULL to provide consistent
behavior

r304532: Replace sprintf -> snprintf for command_errbuf provisioned from
dynamic content.

r305026: Emulate efi_cons_poll when WaitForKey is not available

r305107: Create a hook 'post-initialize' for people that want to define
something to read in .conf files after all other .conf files for the purpose
of overriding.

r305132: Remove accidentally committed stray comment.

r305178: bd_int13probe() should check extended info if sector info is bad

r305353: Don't use -N to set the OMAGIC with data and text writeable and
data not page aligned.

r305814: ufsread: Do not cast struct direct from void *

r306159: Consistently declare getsecs(void) with proper return type and void
when no arguments are present.

r306380: loader command interpreter should reset command_errmsg

r306504: Fix a cluster of bugs in list EFI environment variables

PR: 204602, 211958, 211958
Relnotes: yes ("Improve boot loader quote parsing")

MFC r308434, 308827

MFC r308434: Loader paged/pageable data is not always paged.
MFC r308827: lsdev device name section headers should be printed by dv_print
callback.

MFC r328504: stand/fdt: Consolidate overlay handling a little further

[This is effectively a direct commit to stable/11 due to path restructuring
in HEAD. The diff against HEAD has simply been applied to the old path]

This should have been done as part of r327350, but due to lack of foresight
it came later. In the different places we apply overlays, we duplicate the
bits that check for fdt_overlays in the environment and supplement that with
any other places we need to check for overlays to load. These "other places"
will be loader specific and are not candidates for consolidation.

Provide an fdt_load_dtb_overlays to capture the common logic, allow passing
in an additional list of overlays to be loaded. This additional list of
overlays is used in practice for ubldr to pull in any fdt_overlays passed to
it from U-Boot environment, but it can be used for any other source of
overlays.

These additional overlays supplement loader.conf(5) fdt_overlays, rather
than replace, so that we're not restricted to specifying overlays in only
one place. This is a change from previous behavior where loader.conf(5)
supplied fdt_overlays would cause us to ignore U-Boot environment, and this
seems nonsensical- user should have sufficient control over both of these
aspects, or lack of control for good reasons.

A knob could be considered in the future to ignore U-Boot supplied overlays,
but the supplemental treatment seems like a good start.

MFC (conceptually) r327350: Consistently apply fdt_overlays

This is a direct commit to stable/11 due to restructuring of sys/boot =>
stand in -HEAD. The diff remains the same and it is simply applied to the
previous location.

MFC r327350: stand/fdt: Consistently apply fdt_overlays

Overlays were previously not applied when U-Boot provides FDT or EFI
provides FDT, only when we load FDT from /boot/dtb given name from U-Boot.

Make all three paths lead to loading fdt_overlays and applying them, so that
fdt_overlays can be expected to Just Work.

MFC (conceptually) r326752, r326754:

This is a direct commit to 11-stable, because the code has moved and
the directories have been restructured in 12-current, but it just hand-
applies the same patches to the corresponding files in their old locations.

r326752:
Save and restore r9 register in arm ubldr. In old gcc 4.2, r9 was a callee-
saved register, but in arm EABI it may be either callee-saved or dedicated
to some special purpose (such as a TLS pointer). It appears clang does not
treat it as a callee-saved register (instead using it as another work
register, similar to r12).

Another important side effect of these changes is that saving an extra
register in the push/pop statements keeps the stack aligned to an 8-byte
boundary during the self_reloc() call, as it always should have been.

As stated in the PR...

Essentially the important caller-saved registers are pushed (r0, r1, r9, lr)
before the relocation call, and popped after. Then r8/r9 are saved as usual
for the syscall trampoline, and lr is stored in r8 (now free) as a
callee-saved value before calling into `main`.

The call to `main` can no longer be a tail call because we must restore r9
especially after main returns (although since we have used r8 to hold lr we
must also restore this).

r326754:
When building for arm arches, set PKTALIGN to the max cache line size
supported by the arch, to meet u-boot's requirement that I/O be done
in cache-aligned chunks.

PR: 223977 224008

MFC r316374, r316377:

Preserve the registers containing argc, argv, and return address values
passed in from u-boot across the call to self_reloc and any other early-init
code, and restore them before calling main().

Correct a comment... the stack used by ubldr is the same stack u-boot was
running on when it jumped to the ubldr entry point. None of the arches
that use this code set up a different stack in their start.S routines.

MFC r309369,310850,310853:

libstand: dosfs cstyle cleanup for return keyword.
dosfs support in libstand is broken since r298230

PR: 214423
Submitted by: Mikhail Kupchik
Reported by: Mikhail Kupchik
Approved by: imp (mentor)

MFC r310225: Reduce boot loader version string duplication

Instead of repeating "%s, Revision %s" "(%s %s)" in each loader, just
create the full version string in vers.c

MFC r302465:

Remove extra 0x before %p format strings introduced by r296182

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Fix several instances where the boot loader ignored pager_output
return value when it could return 1 (indicating we should stop).
Fix a few instances of pager_open() / pager_close() not being called.
Actually use these routines for the environment variable printing code
I just committed.

sys/boot: spelling fixes in comments.

No functional change.

Add FDT overlays support to ubldr

FDT overlays is de-facto standard for describing expansion boards like
Beaglebone capes or Raspberry Pi shields. The ides is to have basic
DTB for base board and overlays DTB for shields/capes and to construct
final DTB either using human-readable configuration or some
self-discovery mechanism. I believe this approach can also be expanded
to support dynamically loadable FPGA bitstreams on systems like
Zedboard/Zybo.

Overlaying process is simmilar to executable link process for
binaries: each DTB has "exported" symbols and "undefined" symbols, the
latter are resolved using information for the former obtained from
base DTB or one of the overlays applied earlier (more rare case).

This symbols information is not generated by standard dtc that FreeBSD
has in base system, patched[1] version required to produces
overlay-compatible blobs. So although DTB files generated by
buildkernel do not support overlays there are enough
vendor/community-provided DTB blobs ciruclating around to justify
committing this change to ubldr.

This commit introduces handler for "fdt_overlays" variable that can be
defined either as a loader env variable or U-Boot env variable.
fdt_overlays is comma-separated list of .dtbo files located in
/boot/dtb/ directory along with base .dtb. ubldr loads files and
applies them one-by-one to base .dtb and then passes result blob to
the kernel.

[1] https://github.com/RobertCNelson/dtc/commit/dd6a0533e846e8d5e690a618fa35cc15a6103efb

Differential Revision: https://reviews.freebsd.org/D3180

sys/boot: use our nitems() macro when it is available through param.h.

No functional change, only trivial cases are done in this sweep,

Discussed in: freebsd-current

A new implementation of the loader block cache

The block cache implementation in loader has proven to be almost useless, and in worst case even slowing down the disk reads due to insufficient cache size and extra memory copy.
Also the current cache implementation does not cache reads from CDs, or work with zfs built on top of multiple disks.
Instead of an LRU, this code uses a simple hash (O(1) read from cache), and instead of a single global cache, a separate cache per block device.
The cache also implements limited read-ahead to increase performance.
To simplify read ahead management, the read ahead will not wrap over bcache end, so in worst case, single block physical read will be performed to fill the last block in bcache.

Booting from a virtual CD over IPMI:
0ms latency, before: 27 second, after: 7 seconds
60ms latency, before: over 12 minutes, after: under 5 minutes.

Submitted by: Toomas Soome <tsoome@me.com>
Reviewed by: delphij (previous version), emaste (previous version)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D4713

Add more .NOMETA missed in r291320

Sponsored by: EMC / Isilon Storage Division

Improve U-Boot API detection

Until now, ubldr has been trying to locate the U-Boot API using a hint
address (U-Boot's current stack pointer), aligning it to 1MiB and going
over a 3MiB (or 1MiB in case of MIPS) memory region searching for a
valid API signature.

This change introduces an alternative way of doing this, namely the
following:
- both U-Boot's bootelf and go commands actually pass argc and argv to
the entry point (e.g., ubldr's start function, but they should also
be passed over to main() transparently)
- so, instead of trying to go and look for a valid API signature, we
look at the parameters passed to main()
- if there's an option '-a' with argument, which is a valid hexadecimal
unsigned long number (x), we try to verify whether we have a valid
API signature at address x. If so - we use it. If not - we fallback
to the original way of locating the API signature.

The U-Boot change, which causes the API structure address to be
exported as an environment variable, was committed to mainline U-Boot
as commit 22aa61f707574dd569296f521fcfc46a05f51c48

Reviewed by: andrew, adrian
Approved by: adrian (mentor)
Sponsored by: Smartcom - Bulgaria AD
Differential Revision: https://reviews.freebsd.org/D5492

These changes attempt to put things in order before the introduction of MIPS
ubldr.

The changes are mostly dealing with removing unnecessary casts from the U-Boot
API (we're passing only pointers, no obvious reason to cast them to uint32_t),
cleaning up some compiler warnings and using the proper printf format
specifiers in order to be able to compile cleanly for both 32-bit and 64-bit
MIPS targets.

Reviewed by: imp
Approved by: adrian (mentor)
Sponsored by: Smartcom - Bulgaria AD
Differential Revision: https://reviews.freebsd.org/D5312

So far bsd.stand.mk wasn't included in the following files:
sys/boot/fdt/Makefile
sys/boot/uboot/fdt/Makefile
sys/boot/uboot/lib/Makefile

This causes compilation issues on MIPS due to trying to link PIC with non-PIC
code. This revision includes bsd.stand.mk in the above files.

Reviewed by: imp
Approved by: adrian (mentor)
Sponsored by: Smartcom - Bulgaria AD
Differential Revision: https://reviews.freebsd.org/D5311

If the user has set a u-boot env var named rootpath, automatically
import it into the loader(8) env as dhcp.root-path, so that it overrides
any dhcp/bootp server-provided path.

Now if you have a dhcp server available you can easily net-boot a u-boot
system even if you don't control the dhcp server config, by setting just
two variables in the u-boot env:

loaderdev=net
rootpath=<nfsserverip>:<pathname>

Previously you had to either accept all the dhcp parameters from the
server without the ability to locally provide the rootpath, or you had
to forego dhcp and set more vars (ipaddr, netmask, serverip, rootpath).

Go back to using uintptr_t, because code that actually compiles is
infinitely less buggy than code that is theoretically correct in some
alternate universe.

The uintfptr_t type is apparently a freebsd invention, and exists only when
compiling the kernel. It's a little hard to say for sure, since it doesn't
seem to be documented anywhere except in email advice to unsuspecting and
overly-trusting souls, who then get to wear the pointy hat for blindly
following advice without investigating or testing it first.

Cast using uintfptr_t and eliminate the cast to uint64_t which is uneeded
because rounding down cannot increase the number of bits needed to express
the result.

I had no idea there was such a thing as uintfptr_t.

Requested by: bde

Cast pointer through uintptr_t on the way to uint64_t to squelch a warning.

Use 64-bit math when finding a block of ram to hold the kernel. This fixes
a problem on 32-bit systems which have ram occupying the end of the physical
address space -- for example, a block of ram at 0x80000000 with a size of
0x80000000 was overflowing 32 bit math and ending up with a calculated size
of zero.

This is a fix for one of the two problems mentioned in the PR. Something
similar will need to be done on the kernel side before the PR is closed.

PR: 201614

Fix the error checking for the ubenv command. This moves the check for an
empty ldvar (which amounts to the varname string starting with '=') into
the if block that manipulates ldvar, which avoids later referencing ldvar
when it was never initialized.

Submitted by: Thomas Skibo
Pointy hat: ian

Enhance the "ubenv import" command to allow importing a u-boot env var
directly into a loader (and thus kernel) env var, using the syntax

ubenv import ldvarname=ubvarname

Without the varname= prefix it uses the historical behavior of importing
to the name uboot.ubvarname.

Remove stray unescaped `%` in `Booting from ...` informational message

PR: 204944
MFC after: 1 week
X-MFC with: r291164
Reported by: David Binderman <dcb314@hotmail.com>
Sponsored by: EMC / Isilon Storage Division

Print more detailed info about the disk and partition chosen for booting.
No behavioral changes, just cosmetics.

A partition number of zero is not a wildcard, it's the 'a' partition in
a BSD slice, so don't print it as "<auto>". (Only slices are 1-based,
unit and partition numbers are 0-based and -1 is their wildcard marker.)

Also, after doing all the probing and choosing, print the final result as
"Booting from <disk spec>" where disk spec has all the wildcards resolved
and looks like familiar BSD slice-and-partition notation (disk0s3a, etc).

Make dtb file configurable via loader(8) variable. ubldr already checks
"fdt_file" and "fdtfile" U-Boot variables. Add one more check for
"fdt_file" loader(8) variable.

loader(8) variable takes precedence over u-boot env one

Provide bug4bug workaround for certain dumbiness of the u-boot's API_env_enum
function, which is expected to set returned env to NULL upon reaching the end
of the environment list but fails to do so in certain cases. The respective
u-boot code looks like the following (HEAD at the time of this commit):

--- api.c ---
496 static int API_env_enum(va_list ap)
...
510 *next = last;
511
512 for (i = 0; env_get_char(i) != '\0'; i = n + 1) {
513 for (n = i; env_get_char(n) != '\0'; ++n) {
514 if (n >= CONFIG_ENV_SIZE) {
515 /* XXX shouldn't we set *next = NULL?? */
516 return 0;
517 }
518 }
-------------

The net result is that any unfortunate user of the loader's ub_env_enum()
function hitting this condition would be trapped in the infinite loop, as
the main use pattern of ub_env_enum() is basically the following:

while ((env = ub_env_enum(env)) != NULL) { DO STUFF }

Which would stuck forever with the last element.

Fix bug in the ubldr introduced in the rev.283035. The new code
fails to properly consider memory regions when the loader is
located below of those regions or engulfs their lower limit. This
results in "not enough RAM to load kernel" panic, which is totally
bogus. On top of that, there are some variables that can be left
unitialized in those cases, which might cause it fail with memory
access violation instead of panic while trying to load kernel to
a wrong or non-existing address of memory.

Augment the code to properly deal with the loader being below or
at the lower bound of the memory region in question. Also, don't
leave ununitialized variables behind.

Reviewed by: ian

Print error message if autoload fails due to missing DTB blob, otherwise
ubldr just falls back to loader prompt with no apparent reason

Add a routine to obtain netboot parameters from the U-Boot env vars. Call
it from the uboot net_init() implementation. The routine uses the standard
U-Boot env vars plus a freebsd-specific variable named "rootpath" (the
corresponding u-boot variable for that would be "bootfile" except that it
refers to ubldr, so a new name was needed to communicate the path to ubldr).

This allows ubldr to load a kernel from nfs without requiring a dhcp or
bootp server to provide the server ip and rootpath parameters.

An ARM kernel can be loaded at any 2MB boundary, make ubldr aware of that.

Previously, ubldr would use the virtual addresses in the elf headers by
masking off the high bits and assuming the result was a physical address
where the kernel should be loaded. That would sometimes discard
significant bits of the physical address, but the effects of that were
undone by archsw copy code that would find a large block of memory and
apply an offset to the source/dest copy addresses. The result was that
things were loaded at a different physical address than requested by the
higher code layers, but that worked because other adjustments were applied
later (such as when jumping to the entry point). Very confusing, and
somewhat fragile.

Now the archsw copy routines are just simple copies, and instead
archsw.arch_loadaddr is implemented to choose a load address. The new
routine uses some of the code from the old offset-translation routine to
find the largest block of ram, but it excludes ubldr itself from that
range, and also excludes If ubldr splits the largest block of ram in
two, the kernel is loaded into the bottom of whichever resulting block is
larger.

As part of eliminating ubldr itself from the ram ranges, export the heap
start/end addresses in a pair of new global variables.

This change means that the virtual addresses in the arm kernel elf headers
now have no meaning at all, except for the entry point address. There is
an implicit assumption that the entry point is in the first text page, and
that the address in the the header can be turned into an offset by masking
it with PAGE_MASK. In the future we can link all arm kernels at a virtual
address of 0xC0000000 with no need to use any low-order part of the
address to influence where in ram the kernel gets loaded.

Fix fdt_platform_fixups() mem region handling. It turns out u-boot puts
several types of data into the mem-info array (DRAM, SRAM, flash). We
need to extract just the DRAM entries for translation into fdt memory
properties.

Also, increase the number of regions we can handle from 5 to 16.

Submitted by: Michal Meloun

Allow whitspace to appear between device name and unit number in loaderdev.
This allows the same value to be used in u-boot commands and the loaderdev
env var that gets passed to ubldr, for example 'fatload mmc 1 ${bootfile}".

Start to allow platforms other than U-Boot to use the FDT code in loader by
moving U-Boot specific code from libfdt.a to a new libuboot_fdt.a. This
needs to be a new library for linking to work correctly.

Differential Revision: https://reviews.freebsd.org/D1054
Reviewed by: ian, rpaulo (earlier version)
MFC after: 1 week

Move the definitions of the fdt functions from a uboot header to a new fdt
header. There is nothing in the fdt spec that ties it to U-Boot.

While here sort and fix the signature of fdt_setup_fdtp.

MFC after: 1 week

Add a 'ubenv import' command to import environment variables from the
u-boot env into the loader(8) env (which also gets them into the kernel
env). You can import selected variables or the whole environment. Each
u-boot var=value becomes uboot.var=value in the loader env. You can also
use 'ubenv show' to display uboot vars without importing them.

Give loaders more control over the Forth initialization process. In
particular, allow loaders to define the name of the RC script the
interpreter needs to use. Use this new-found control to have the
PXE loader (when compiled with TFTP support and not NFS support)
read from ${bootfile}.4th, where ${bootfile} is the name of the
file fetched by the PXE firmware.

The normal startup process involves reading the following files:
1. /boot/boot.4th
2. /boot/loader.rc or alternatively /boot/boot.conf

When these come from a FreeBSD-defined file system, this is all
good. But when we boot over the network, subdirectories and fixed
file names are often painful to administrators and there's really
no way for them to change the behaviour of the loader.

Obtained from: Juniper Networks, Inc.

Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.

Cosmetic changes to printed output, mostly related to probing devices...

- Display slice and partition as <auto> instead of 0 or -1 when they're
not set to specific values (the paritition=-1 was confusing folks).
- When loaderdev isn't set in the u-boot environment, say so rather
than displaying unknown device ''.
- Print the loader(8) ident/version info earlier, so that all device-
related info appears together afterwards.

The one change here that isn't purely cosmetic is to call setheap()
earlier. The comment says "Initialise heap as early as possible", now
that's more accurate. It shouldn't make any functional difference, but
may be safer if future changes lead to trying to allocate memory earlier.

Fix the paren nesting so that the network device is probed if the load
device type is either network or wildcard.

Fix an uninitialized variable error I perpetrated when splitting some
code into a separate function. Pass the missing value from main() to
the probe_disks() function.

Enhance the mechanism that lets you configure the ubldr boot device by
setting the u-boot environment variable loaderdev=. It used to accept only
'disk' or 'net'. Now it allows specification of unit, slice, and partition
as well. In addition to the generic 'disk' it also accepts specific
storage device types such as 'mmc' or 'sata'.

If there isn't a loaderdev env var, the historical behavior is maintained.
It will use the first storage device it finds, or a network device if
no working storage device exists.

99% of the work on this was done by Patrick Kelsey, but I made some
changes, so if anything goes wrong, blame me.

Submitted by: Patrick Kelsey <kelsey@ieee.org>

howto_names: unify declaration

Approved by: gibbs
Sponsored by: Citrix Systems R&D

boot/i386/efi/bootinfo.c:
boot/i386/libi386/bootinfo.c:
boot/ia64/common/bootinfo.c:
boot/powerpc/ofw/metadata.c:
boot/powerpc/ps3/metadata.c:
boot/sparc64/loader/metadata.c:
boot/uboot/common/metadata.c:
boot/userboot/userboot/bootinfo.c:
i386/xen/xen_machdep.c:
- Include sys/boot.h
- Remove custom definition of howto_names.

sys/boot.h:
- Define howto_names.

x86/xen/pv.c:
- Include sys/boot.h

Include needed header. I don't know why clang didn't complain about this
but gcc sure did.

If we can't find the u-boot api signature, exit with an error status that
includes "badab1"; u-boot displays the status in hex, so this may help
give some clue about why ubldr isn't working (more than -1 does anyway).

Prevent fdt data loaded from a file from overwriting the kernel environment,
by having uboot_autoload() do the fdt setup (which may load a file) rather
than waiting until we're actually in the process of launching the kernel.

As part of making this happen...
- Define LOADER_FDT_SUPPORT on the uboot/lib compile command line when
MK_FDT is set.
- Make fdt_setup_fdtb() public.
- Declare public fdt_whatever() functions in a header instead of using
scattered extern decls in .c files.

Allow the ubldr boot device to be choosen by a u-boot environment variable.

If a "loaderdev=<device>" env variable is set and the named device
exists, it is used. If the device doesn't exist, fall back to the
historic "probe" loop that prefers disk devices over network devices.

If the env var is not set, preserve the historic behavior of using the
first working disk device provided by u-boot, or a network device if no
functional disk device is found and a network device exists.

The old probe loop is reworked so that it checks all bootable devices
provided by u-boot rather than taking an early-out on the first device
found. This results in the cosmetic change of listing all potential boot
devices for the user, but the behavior of which device it chooses is the
same as it has always been.

Remove all the instances of '#undef DEBUG' from kernel.

Suggested by: rpaulo
Approved by: adrian (mentor)

Print kernel args as late as possible before jumping into kernel.

Clarify an error message. The addresses here are
really offsets within the primary mapped memory block
(which may not start at address zero).

Do metadata relocation fixup on all loaded modules, not just the kernel.

Approved by: cognet (mentor)

- Implement "fdt mres" sub-command that prints reserved memory regions
- Add "fdt addr" subcommand that lets you specify preloaded blob address
- Do not pre-initialize blob for "fdt addr"
- Do not try to load dtb every time fdt subcommand is issued,
do it only once
- Change the way DTB is passed to kernel. With introduction of "fdt addr"
actual blob address can be not virtual but physical or reside in
area higher then 64Mb. ubldr should create copy of it in kernel area
and pass pointer to this newly allocated buffer which is guaranteed to work
in kernel after switching on MMU.
- Convert memreserv FDT info to "memreserv" property of root node
FDT uses /memreserve/ data to notify OS about reserved memory areas.
Technically it's not real property, it's just data blob, sequence
of <start, size> pairs where both start and size are 64-bit integers.
It doesn't fit nicely with OF API we use in kernel, so in order to unify
thing ubldr converts this data to "memreserve" property using the same
format for addresses and sizes as /memory node.

Print kernel args when booting the kernel.

Add the flags parameter to the disk_open() function and DISK_F_NOCACHE
flag, that disables the caching of partition tables metadata.
Use this flag for floppies in the libi386/biosdisk driver.

Almost each time when loader opens a file, this leads to calling
disk_open(). Very often this is called several times for one file.
This leads to reading partition table metadata for each call. To
reduce the number of disk I/O we have a simple block cache, but it
is very dumb and more than half of I/O operations related to reading
metadata, misses this cache.

Introduce new cache layer to resolve this problem. It is independent
and doesn't need initialization like bcache, and will work by default
for all loaders which use the new DISK API. A successful disk_open()
call to each new disk or partition produces new entry in the cache.
Even more, when disk was already open, now opening of any nested
partitions does not require reading top level partition table.
So, if without this cache, partition table metadata was read around
20-50 times during boot, now it reads only once. This affects the booting
from GPT and MBR from the UFS.

Update according to the change of struct uboot_devdesc.

Build disk.c only when DISK_SUPPORT is enabled.

Update uboot's disk driver to use new DISK API.

Use disk_fmtdev() and disk_parsedev() functions from the new DISK API.

Make struct uboot_devdesc compatible with struct disk_devdesc.

Determine kernel load address dynamically from u-boot memory map.

The generic ELF loading code maps the kernel into low memory
by subtracting KERN_BASE. So the copyin/copyout/readin functions
are always called with low addresses. This code finds the largest
DRAM block from the U-Boot memory map and adds that base to
the addresses.

In particular, this fixes ubldr on AM3358, which has DRAM
mapped to 0x80000000 at power-on.

Add architecture dependent code to support NAND Framework on Marvell SoCs.

Obtained from: Semihalf
Supported by: FreeBSD Foundation, Juniper Networks

Access the device tree blob via copyin/copyout.

The code previously assumed that copyin/copyout did no
address translation and that the device tree blob could
be manipulated in-place (with only a few adjustments for
the ELF loader offset). This isn't possible on all platforms,
so the revised code uses copyout() to copy the device tree
blob into a heap-allocated buffer and then updates the
device tree with copyout(). This isn't ideal, since it
bloats the loader memory usage, but seems the only feasible
approach (short of rewriting all of the fdt manipulation
routines).

Ooops. Back out changes to main.c from r235094.

This change is specific to BeagleBone; I need to figure out how to
generalize this before committing.

Don't call strcmp with a NULL pointer.

In particular, on the AM335x, which comes up with no memory
mapped to low addresses, dereferencing the NULL causes a crash.

Teach ubldr(8) about simple MBR partitioning.

Use the common/shared CRC-32 implementation instead of duplicating it.

MFC after: 1 week

Improve device tree blob (DTB) handling in loader(8).

Enable using the statically embedded blob from the kernel, if present. The KLD
loaded DTB takes precedence, but they are both recognized and handled in the
same way.

Submitted by: Lukasz Wojcik
Obtained from: Semihalf
MFC after: 1 week

Rename the linker emulation name for powerpc and powerc64. This is needed that
we can also use the upstream binutils linker where we have to have a unique
name for the FreeBSD emulation.

MF tbemd: Minor tweaks, prefer MACHINE_CPUARCH generally to MACHINE_ARCH (which simplifies some powerpc/powerpc64 ifs)

Provide support in loader for booting 64-bit PowerPC kernels. Like amd64,
64-bit PowerPC kernels are loaded by a 32-bit loader, since nearly all
powerpc64 firmwares execute in 32-bit mode.

Get rid of bootinfo for good in loader (U-Boot-based) and ARM.

For FDT-enabled platforms the device tree is a modern replacement for bootinfo
config data.

Fix conditional FDT support in loader(8).

Initial loader(8) support for Flattened Device Tree.

o This is disabled by default for now, and can be enabled using WITH_FDT at
build time.

o Tested with ARM and PowerPC.

Reviewed by: imp
Sponsored by: The FreeBSD Foundation

Mostly revert r200691. U-Boot syscall() entry point returns 1 on success.

Use loader devices only when they initialized properly.

Minor style correction.

Fix handling of GPT disk partition index.

Obtained from: Semihalf
MFC after: 1 week

Let loader(8) for U-Boot use default storage more flexibly.

Obtained from: Semihalf
MFC after: 1 week

Fix ub_env_enum(): syscall() returns 0 when properly invoked.

Provide an effective (relocated) address when building modules metadata.

This lets modules loaded dynamically in loader(8) work for U-Boot-based
platforms.

MFC after: 1 week

- Remove trailing ";" after if statement
- Remove #if 0 section that was never needed/used

Reviewed by: raj@
MFC after: 1 month

Make GPT style partitiong endian-safe in U-Boot support library.

Submitted by: Piotr Ziecik
Obtained from: Semihalf

Fix NETIF_DEBUG compilation.

GPT style partitioning for loader(8) with U-Boot support library (tested on
ARM).

Submitted by: Piotr Ziecik kosmo ! semihalf dot com

Improve style(9) in PowerPC U-Boot support lib.

Initial storage functionality for U-Boot support library.

- Only non-sliced bsdlabel style partitioning is currently supported (but provisions
are made towards GPT support, which should follow soon)
- Enable storage support in loader on ARM

Obtained from: Semihalf

Initial support of loader(8) for ARM machines running U-Boot.

This uses the common U-Boot support lib (sys/boot/uboot, already used on
FreeBSD/powerpc), and assumes the underlying firmware has the modern API for
stand-alone apps enabled in the config (CONFIG_API).

Only netbooting is supported at the moment.

Obtained from: Marvell, Semihalf

Minor style(9) fixes for U-Boot API glue.

U-Boot API glue improvements:

- extend ub_dev_read() and ub_dev_recv() so that the actual len and
all error codes can be passed and processed properly; unify behaviour of
these routines

- introduce syscall general error code (API_ESYSC)

Backout rev 183181. It appears that I should have been using boot-device
of "cd:,\\:tbxi" with properly configured boot.tbxi, instead of booting
\boot\loader directly. Rev 183168 could probably stay, since it can be
viewed as an anti-foot-shooting measure and has no impact on normal
operation. I can revert it as well, if anybody objects.

Improve loader support for U-Boot.

- add new diag commands: devinfo, sysinfo for U-Boot-style details about the system
configuration
- better memory info summary
- style corrections

Obtained from: Semihalf

Make metadata.c shared across all platforms using U-Boot.

This will [soon] be needed for ARM.

Move U-Boot compatibility library to WARNS=2 level.

Enable GCC stack protection (aka Propolice) for userland:
- It is opt-out for now so as to give it maximum testing, but it may be
turned opt-in for stable branches depending on the consensus. You
can turn it off with WITHOUT_SSP.
- WITHOUT_SSP was previously used to disable the build of GNU libssp.
It is harmless to steal the knob as SSP symbols have been provided
by libc for a long time, GNU libssp should not have been much used.
- SSP is disabled in a few corners such as system bootstrap programs
(sys/boot), process bootstrap code (rtld, csu) and SSP symbols themselves.
- It should be safe to use -fstack-protector-all to build world, however
libc will be automatically downgraded to -fstack-protector because it
breaks rtld otherwise.
- This option is unavailable on ia64.

Enable GCC stack protection (aka Propolice) for kernel:
- It is opt-out for now so as to give it maximum testing.
- Do not compile your kernel with -fstack-protector-all, it won't work.

Submitted by: Jeremie Le Hen <jeremie@le-hen.org>

style(9) & style.Makefile(9)

Reviewed by: raj

Eliminate artificial increasing of 'netdev_opens' counter in loader's net_open().

This was introduced as a workaround long time ago for some Alpha firmware
(which is now gone), and actually prevented net_close() to ever be
called.

Certain firmwares (U-Boot) need local shutdown operations to be performed on a
network controller upon transaction end: such platform-specific hooks are
supposed to be called via netif_close() (from within net_close()).

This change effectively reverts the following CVS commit:

sys/boot/common/dev_net.c

revision 1.7
date: 2000/05/13 15:40:46; author: dfr; state: Exp; lines: +2 -1
Only probe network settings on the first open of the network device.
The alpha firmware takes a seriously long time to open the network device
the first time.

Also suppress excessive output while netbooting via loader, unless debugging.

While there, make sys/boot/uboot more style(9) compliant.

Reviewed by: imp
Approved by: cognet (mentor)

o Include glue.h
o Support multiple memory regions.

o Build libuboot with -msoft-float like everything else.
o Move the API prototypes to a separate header (glue.h)
o Allow the platform to hint libuboot about where to look
for the API signature. The uboot_address variable is
expected to be defined by the platform.

Add the appropriate license information. This file is double
licensed under GPL and BSD.

Thanks to: raj@

MFp4 (e500):

Add support for U-Boot. This uses the U-Boot API as developed by
Rafal and which is (will be) part of U-Boot 1.3.2 and later.

Credits to: raj@