MFC r330806-r330815, r330837, r330864, r330883

pc98 changes associated with the named commits are also included in this
commit, despite not having been made with the original commits due to its
removal in head.

r330806:
Minor cosmetic changes.

Make sure { on the same line as struct for all struct *devdesc. Move
some type definitions to next to the dv_type define, since that's what
sets the d_type.

r330807:
We can't use d_opendata for blkio storage.

open_disk uses d_opendata for it's own purpse. We can't store blkio
there. Fortunately, blkio is stored elsewhere and we never actually
retrieve blkio from d_opendata. Eliminate it as a source of confusion.
Eliminate all stores of d_opendata in efi since this layer doesn't own
that field.

r330808:
Make struct libi386_devdesc match the struct devdesc better

Move data to top and call it d_opendata.

r330809:
Use the actual struct devdesc at the start of all *_devdesc structs

The current system is fragile and requires very careful layout of all
*_devdesc structures. It also makes it hard to change the base
devdesc. Take a page from CAM and put the 'header' in all the derived
classes and adjust the code to match.

For OFW, move the iHandle h_handle out of a slot conflicting with
d_opendata. Due to quirks in the alignment rules, this worked.
However changing the code to use d_opendata storage now that it's a
pointer is hard, so just have a separate field for it.

All other cleanups were to make the *_devdesc structures match where
they'd taken some liberties that were none-the-less compatible enough
to work.

r330810:
Remove d_type from devdesc. It's not needed as we can fetch it from
d_dev->dv_type when we need it.

r330811:
GC unused routines.

Sponsored by: Netflix

r330812:
Use the one-line-per-file pattern here, and sort the file names.

Sponsored by: Netflix

r330813:
Move the env convenience routines out of boot1.c.

These routines are more generally useful. Even though boot1 is on its
way out, it's better to make these common during the transition than
copy them.

r330814:
Star BootCurrent entry when booting.

Sponsored by: Netflix

r330815:
Print the load and device path as well as BootCurrent and BootOrder

Sponsored by: Netflix

r330837:
biosdisk.c should not set d_opendata.

Same as 330807, d_opendata is owned by open_disk and we should not
set it.

M stand/i386/libi386/biosdisk.c

r330864:
Prefer uintXX_t to u_intXX_t

A foolish consistency is the hobgoblin of little minds, adored by
little statesmen and philosophers and divines. With consistency a
great soul has simply nothing to do. -- Ralph Waldo Emerson

r330883:
Fix typo that misteriously passes compilation.

MFC Loader Fixes Final: r327612,r327703,r327704,r327878,r327879,r327881,
r328007,r328029,r328030,r328031,r328061,r328156,r328169,r328288,r328289,
r328290,r328291,r328292,r328411,r328536,r328603,r328614,r328642,r328769,
r328779,r328780,r328781,r328782,r328783,r328806,r328808,r328826,r328835,
r328911,r328986,r328987,r328990,r328999,r329000,r329019,r329050,r329054,
r329060

r327612: Invent new #defines for the biospci_{read,write}_config function

r327703: Define __dmadat after #include'ing ufsread.c.

r327704: Fix printf missing format variables warnings.

r327878: Add GUID for UEFI boot manager variables.

r327879: Report the boot order and where we are in that boot order.

r327881: Allow this file to be included

r328007: Fix booting on some arm64 systems after r327879 by fixing the call
to utf8_to_ucs2

r328029: When returning an error and freeing allocated memory from
ucs2_to_utf8, NULL the return pointer.

r328030: Check the return value from utf8_to_ucs2 instead of whether or not
uv is NULL.

r328031: Need to free uv after we're done using it.

r328061: utf8_to_ucs2() should check for malloc failure

r328156: stand: Move sections around to fix stand/ build with ld.lld on
armv7

r328169: Remove extra copy of bootinfo.c. It's a bit rotted copy of the one
in efi/loader.

r328288: Fix some resource leaks.

r328289: Don't leak memory when displaying help.

r328290: On malloc failure, be sure to close the include file that triggered
it.

r328291: getenv does not return tainted data in the boot loader. Attempt to
clue Coverity into that fact.

r328292: There's no tainted data here, tag it as such to avoid false
positives.

r328411: loader.efi: add missing EFI GUIDs

r328536: loader: support for mixed-endianness ELF/loader and POWER8

r328603: Add missing non-POWERPC case to give the scr value something
non-zero.

r328614: Move libstand.3 to libsa.3. Update libsa.3 to include functions

r328642: Break out the interpreters (simple and forth) w/o ifdefs.

r328769: Centralize several variables.

r328779: Retire pnp.4th and the code needed only for 4th words used here.

r328780: These 4th words were an attempt to allow integration into the boot
loader scripts. However, that path won't be taken after all it
seems.

r328781: Remove pcibios forth support.

r328782: Now that we no longer conditionally compile some files outside of
ficl

r328783: Invent new LDR_INTERP for the loader interpreter to use.

r328806: We need more heap space to properly load newer powerpc kernels.

r328808: Implement strcoll as strcmp.

r328826: Make cross-endian loader changes apply only to powerpc

r328835: Fix regression introduced in r328806, preventing boot on many
platforms.

r328911: Ignore relocation tables for non-memory-resident sections.

r328986: Fix relative location of USB sources after recent move.

r328987: A more definitions to kernel emulation shim in order to build
stand/usb.

r328990: Move the stand/usb test loader into its own directory.

r328999: Fix indentation to FreeBSD standard for interp files

r329000: Move simple interpreter 'perform' into interp.c and call it

r329019: Move to tabs for indentation and to 8-space notches, per style(9).

r329050: Fix build of userboot.so

r329054: Set script.lang in the environment to either 'forth' or 'simple' to
reflect what scripting language was compiled into the loader.

r329060: loader: fix endianness conversion

PR: 225323

MFC r325834,r325997,326502: Move sys/boot to stand/

This is effectively a direct commit to stable/11, due to differences between
stable/11 and head. Changes to DTS in sys/boot/fdt/dts were often
accompanied by kernel changes. Many of these were also risc-v updates that
likely had many more dependencies to MFC.

Because of this, sys/boot/fdt/dts remains as-is while everything else in
sys/boot relocates to stand/.

r325834: Move sys/boot to stand. Fix all references to new location

r325997: Remove empty directories.

r326502: Document the sys/boot -> stand move in hier.7 and the top-level README.

MFC Loader Fixes 2017q4p6: r324649,r324650,r324651,r324652,r324653,r324654,
r324700,r324702,r324709,r324717,r324719,r324841,r324842,r324843,r324845,
r324850,r324876,r324877,r324878,r324879,r324880,r324881,r324883,r324980,
r324981,r324982,r324995,r325014,r325093,r325094,r325114,r325170,r325171,
r325172,r325173,r325174,r325175,r325176,r325248,r325286,r325310,r325332,
r325338,r325339,r325376,r325377,r325379,r325380,r325382,r325478,r325479,
r325480,r325482,r325483,r325484,r325485,r325556,r325641,r325681,r325685,
r325686,r325687,r325688,r325689,r325690,r325691,r325692,r325693,r325694,
r325743,r325744,r325748,r325775,r325779,r325780

r324649: Move common/Makefile.inc to sys/boot/loader.mk.

r324650: tweak style

r324651: create defs.mk for common definitions

r324652: Move all the ficl common code into ficl.mk

r324653: LOADER_foo_SUPPORTED

r324654: Explicitly inlcude SYSDIR in the include path

r324700: loader: initialize dv_cleanup in md.c to eliminate clang warning

r324702: We need to include disk.o in libuboot.a when we're building with
support for disk access.

r324709: Revert "Unify boot1 with loader" change r324646

r324717: libsa/ip.c: misplaced comment, ip_v is half char, not ip_p

r324719: libsa/ip: stop read loop on bad fragments

r324841: Use BOOTDIR more consistently in defs.mk rather than repeat
sys/boot.

r324842: Introduce BOOTOBJ: The top level object directory for the boot tree

r324843: Stopgap fix to the mistmatch between LOADER_GELI_SUPPORT and
LOADER_NO_GELI_SUPPORT.

r324845: Use BOOTOBJ and BOOTDIR to find geli includes and libraries.

r324850: Define LIBSA32 to LIBSA on i386 to fix build.

r324876: Move fdt and uboot defines into common uboot.mk.

r324877: End source directories with SRC rather than a hodgepodge of names

r324878: Make at91 boot loader compile again.

r324879: Prefer SRCTOP paths for bits we're grabbing from libc.

r324880: Use BOOTSRC here.

r324881: Use SYSDIR instead of ${.CURDIR}/../..<etc>/sys.

r324883: Use preferred defined paths, rather than relative paths in fdt.

r324980: Use BOOTDIR consistently.

r324981: Move BINDIR definition to defs.mk, and override where it isn't
/boot

r324982: Remove sys/boot/arm/at91 and ixp425

r324995: loader.efi: Make framebuffer commands available for arm64

r325014: Add a 'place holder' arm struct efi_fb until a real one comes

r325093: Define new EFI variables

r325094: Cleanup non-arch Makefiles

r325114: Use defs.mk defins in most MD code

r325170: Use defs.mk values for userboot

r325171: Use defs.mk name and prefer bsd.init.mk

r325172: Remove the -nostdlib stuff I added. Instead, fix LDFLAGS to be
honored correctly with the new Makefile.inc include order.

r325173: We don't need to build a special ficl for userboot.

r325174: Minor cleanup

r325175: For amd64, compile both zfs and zfs32 libraries.

r325176: Actually add zfs32/Makefile

r325248: loader ptblread() is broken with >512B sectors

r325286: efipart_strategy is using wrong offset with >512B sectors

r325310: zfs.c:vdev_read() needs to be careful about large sectors

r325332: loader: re-enable gzip support for x86

r325338: loader: fix BOOTSRC -> BOOTOBJ in a library path

r325339: This used to have bzip2 support too.

r325376: WIP: centralize machine links

r325377: mostly libsa

r325379: Revert "mostly libsa"

r325380: Revert "WIP: centralize machine links"

r325382: Cleanup stray libstand names to be libsa names.

r325478: Powerpc is a 32-bit boot loader.

r325479: Define LIBFICL32 to be libficl.a on i386 and libficl32.a on amd64.

r325480: Use DO32 for all the places that we need to flag we're building a
32-bit version of a library.

r325482: Move machine and other link creation to defs.mk

r325483: MACHINE can never be powerpc64, so cleanup code that thinks it can.

r325484: Prefer bsd.init.mk to src.opts.mk

r325485: Centralize all 32-bit builds on 64-bit platform stuff.

r325556: loader: set options before including bsd.init.mk

r325641: loader.efi: efi_devpath_is_prefix should return bool

r325681: boot1: avoid using NULL device path

r325685: libsa32 isn't needed for i386. It's already a 32-bit platform.

r325686: Simplify this if to a direct assignment.

r325687: Remove all the empty help files from the powerpc build.

r325688: FDT support doesn't make sense for ps3, remove it.

r325689: Remove LOADER_FDT_SUPPORT as a Makefile variable.

r325690: Remove LOADER_ZFS_SUPPORT as a Makefile variable

r325691: Remove useless PNP define here.

r325692: Replace LOADER_FIREWIRE_SUPPORT variable

r325693: Move LOADER_{NO,}_GELI_SUPPORT to MK_LOADER_GELI

r325694: Install the 4th files in sys/boot/forth instead of each loader

r325743: Make sure the proper loader.rc gets installed.

r325744: boot1: also check for NULL device

r325748: Use proper include file.

r325775: Add loader.conf to the list of files that are MD.

r325779: Add /boot/dts to the list of default modules.

r325780: Don't add /boot/dt*s* but /boot/dt*b*. Stupid think-o.

MFC Loader Fixes 2017q1: r311458,r312237,r312314,r312374,r312947,r313042,
r313047,r313166,r313328,r313332,r313333,r313337,r313348,r313349,r313389,
r313442,r313451,r313575,r313645,r313710,r314114,r314213,r314275,r314945,
r314948,r315008,r315408,r315427,r315645,r315646,r315648,r315653,r315850,
r316064,r316078,r316079,r316100,r316104,r316111,r316112,r316171,r316279,
r316280,r316287,r316311,r316343,r316424,r316436

r311458: Use compiler driver to link BERI boot loaders

r312237: loader.efi: find_currdev() can leak memory

r312314: loader: move device path definitions to include/efidevp.h

r312374: loader: efi devpath api usage should be more aware of NULL pointers

r312947: Remove "-Xassembler -G0" from CFLAGS.

r313042: loader.efi environment related cleanups

r313047: loader: disk/part api needs to use uint64_t offsets

r313166: loader: libefi/env.c warnings in arm build

r313328: loader: Implement disk_ioctl() to support DIOCGSECTORSIZE and
DIOCGMEDIASIZE.

r313332: loader: bcache read ahead block count should take account the large
sectors

r313333: loader: Replace EFI part devices.

r313337: loader: 313329 missed ZFS guard in loader/main.c

r313348: loader: biosdisk fix for 2+TB disks

r313349: loader: disk io should not use alloca()

r313389: efipart is also using the '%S' printf format, add -Wno-format for
it.

r313442: loader: possible NULL pointer dereference in efipart.c

r313451: loader: possible NULL pointer dereference in bcache.c

r313575: makefs: make the buffer functions look exactly like the kernel ones

r313645: loader: implement MEDIA_FILEPATH_DP support in efipart

r313710: loader: cstyle fixes and DIOCGMEDIASIZE should use uint64_t

r314114: Use LDFLAGS with CC instead of _LDFLAGS.

r314213: Remove control+r handling from geliboot's pwgets()

r314275: Remove unused macro from common/drv.c.

r314945: Some style(9) fixes. No functional changes.

r314948: Try to extract the RFC1048 data from PXE.

r315008: r314948 seems to be missing a variable or two that will break

r315408: loader: remove open_disk cache

r315427: loader: biosdisk should report IO error from INT13

r315645: loader: disk_cleanup was left in userboot_disk.c

r315646: loader: pxe.h constants have wrong values

r315648: libstand: verify value provided by nfs.read_size

r315653: loader: verify the value from dhcp.interface-mtu and use snprintf
o set mtu

r315850: The original author abused Nd (one-line description, used by
makewhatis)

r316064: Fix build with path names with 'align' or 'nop' in them.

r316078: gpt*boot: Save a bit more memory when LOADER_NO_GELI_SUPPORT is
specified

r316079: Simply retire the sedification of the boot2.s file.

r316100: Remove -fno-guess-branch-probability and -fno-unit-at-a-time.

r316104: Use `NO_WCAST_ALIGN` instead of spelling it out as -Wno-cast-align
in CFLAGS

r316111: loader: move bios getsecs into time.c

r316112: loader: ls command should display file types properly

r316171: xfsread inlined uses more space, so remove the inline tag.

r316279: loader: efipart should check disk size from partition table

r316280: loader: simplify efi_zfs_probe and avoid double probing for zfs.

r316287: Remove OLD_NFSV2 from loader and libstand

r316311: Add explicit_bzero() to libstand, and switch GELIBoot to using it

r316343: Implement boot-time encryption key passing (keybuf)

r316424: Fix sparc64 build broken by r316343 and r316076

r316436: Restore EFI boot environment functionality broken in r313333

PR: 216940 217298 217935

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

META MODE: Update dependencies with 'the-lot' and add missing directories.

This is not properly respecting WITHOUT or ARCH dependencies in target/.
Doing so requires a massive effort to rework targets/ to do so. A
better approach will be to either include the SUBDIR Makefiles directly
and map to DIRDEPS or just dynamically lookup the SUBDIR. These lose
the benefit of having a userland/lib, userland/libexec, etc, though and
results in a massive package. The current implementation of targets/ is
very unmaintainable.

Currently rescue/rescue and sys/modules are still not connected.

Sponsored by: EMC / Isilon Storage Division

Revert r267233 for now. PIE support needs to be reworked.

1. 50+% of NO_PIE use is fixed by adding -fPIC to INTERNALLIB and other
build-only utility libraries.
2. Another 40% is fixed by generating _pic.a variants of various libraries.
3. Some of the NO_PIE use is a bit absurd as it is disabling PIE (and ASLR)
where it never would work anyhow, such as csu or loader. This suggests
there may be better ways of adding support to the tree. Many of these
cases can be fixed such that -fPIE will work but there is really no
reason to have it in those cases.
4. Some of the uses are working around hacks done to some Makefiles that are
really building libraries but have been using bsd.prog.mk because the code
is cleaner. Had they been using bsd.lib.mk then NO_PIE would not have
been needed.

We likely do want to enable PIE by default (opt-out) for non-tree consumers
(such as ports). For in-tree though we probably want to only enable PIE
(opt-in) for common attack targets such as remote service daemons and setuid
utilities. This is also a great performance compromise since ASLR is expected
to reduce performance. As such it does not make sense to enable it in all
utilities such as ls(1) that have little benefit to having it enabled.

Reported by: kib

In preparation for ASLR [1] support add WITH_PIE to support building with -fPIE.

This is currently an opt-in build flag. Once ASLR support is ready and stable
it should changed to opt-out and be enabled by default along with ASLR.

Each application Makefile uses opt-out to ensure that ASLR will be enabled by
default in new directories when the system is compiled with PIE/ASLR. [2]

Mark known build failures as NO_PIE for now.

The only known runtime failure was rtld.

[1] http://www.bsdcan.org/2014/schedule/events/452.en.html
Submitted by: Shawn Webb <lattera@gmail.com>
Discussed between: des@ and Shawn Webb [2]

NO_MAN= has been deprecated in favor of MAN= for some time, go ahead
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.

btxldr: future-proof argument passing from boot1/2-ish to loader

Place the arguments at a fixed offset of 0x800 withing the argument area
(of size 0x1000). Allow variable size extended arguments first of which
should be a size of the extended arguments (including the size
parameter).

Consolidate all related definitions in a new i386/common/bootargs.h header.

Many thanks to jhb and bde for their guidance and reviews.

Reviewed by: jhb, bde
Approved by: jhb
MFC after: 1 month

Consolidate duplicate definitions of V86_CY() and V86_ZR() which check for
the carry and zero flags being set, respectively, in <btxv86.h> and use
them throughout the x86 boot code.

Start the dreaded NOFOO -> NO_FOO conversion.

OK'ed by: core

Move boot2 BSS zeroing into btx startup code out of boot1. boot1 does not
have clear idea on boot2 BSS size and leaves portion of it not zeroed out.
btxcsu.s is in much better position for this job.

Obtained from: DragonflyBSD (with minor adjustments)

- Factor out -nostdlib to an upper level Makefile.inc.

- Now that bsd.prog.mk deals with programs linked with -nostdlib
better, and has a notion of an "internal" program, use PROG
where possible. This has a good impact on the contents of
.depend files and causes programs to be linked with cc(1).

XXX: boot2 couldn't be converted as it's actually two programs.

Tested on: i386, amd64

Switch to using bsd.prog.mk; this gives us back the standard
.s.o transformation rule.

Initiate deorbit burn for the i386-only a.out related support. Moves are
under way to move the remnants of the a.out toolchain to ports. As the
comment in src/Makefile said, this stuff is deprecated and one should not
expect this to remain beyond 4.0-REL. It has already lasted WAY beyond
that.

Notable exceptions:
gcc - I have not touched the a.out generation stuff there.
ldd/ldconfig - still have some code to interface with a.out rtld.
old as/ld/etc - I have not removed these yet, pending their move to ports.
some includes - necessary for ldd/ldconfig for now.

Tested on: i386 (extensively), alpha

Fixed CLEANFILES after bsd.lib.mk sweep.

Major cleanup of bsd.lib.mk.

Get rid of the INTERNALSTATICLIB knob and just use plain INTERNALLIB.
INTERNALLIB now means to build static library only and don't install
anything. Added a NOINSTALLLIB knob for libpam/modules. To not
build any library at all, just do not set LIB.

Back out last commit. I expect our bsd.*.mk gods to remove the need for
defining so many extra things in addition to INTERNALLIB. We don't like
repetitive C code and we shouldn't for make code either.

NOPIC, NOPROFILE, NOMAN, and INTERNALSTATICLIB are redundant when using
INTERNALLIB now.

Unset MAINTAINER.

Set MAINTAINER.

$Id$ -> $FreeBSD$

Pop the return address off the stack before making a V86_CALLF call;
this allows us to implement what look like C function calls from user
space "directly" to v86 mode code. (Used for calling the PnP BIOS)

Override for ELF. Override .s.o suffix rule.

Adjust setting of argument pointer for BTX 0.87.

Fix an egregious precedence bug.

Mark exit() as __attribute__((__noreturn__))

Initial integration of the i386 bootloader and BTX.

- Discard large amounts of BIOS-related code in favour of the more compact
BTX vm86 interface.
- Build the loader module as ELF, although the resulting object is a.out,
make gensetdefs 32/64-bit sensitive and use a single copy of it.
- Throw away installboot, as it's no longer required.
- Use direct bcopy operations in the i386_copy module, as BTX
maps the first 16M of memory. Check operations against the
detected size of actual memory.

Add BTX startup/interface code.