MFC Loader Fixes Final: r327612,r327703,r327704,r327878,r327879,r327881,
r328007,r328029,r328030,r328031,r328061,r328156,r328169,r328288,r328289,
r328290,r328291,r328292,r328411,r328536,r328603,r328614,r328642,r328769,
r328779,r328780,r328781,r328782,r328783,r328806,r328808,r328826,r328835,
r328911,r328986,r328987,r328990,r328999,r329000,r329019,r329050,r329054,
r329060

r327612: Invent new #defines for the biospci_{read,write}_config function

r327703: Define __dmadat after #include'ing ufsread.c.

r327704: Fix printf missing format variables warnings.

r327878: Add GUID for UEFI boot manager variables.

r327879: Report the boot order and where we are in that boot order.

r327881: Allow this file to be included

r328007: Fix booting on some arm64 systems after r327879 by fixing the call
to utf8_to_ucs2

r328029: When returning an error and freeing allocated memory from
ucs2_to_utf8, NULL the return pointer.

r328030: Check the return value from utf8_to_ucs2 instead of whether or not
uv is NULL.

r328031: Need to free uv after we're done using it.

r328061: utf8_to_ucs2() should check for malloc failure

r328156: stand: Move sections around to fix stand/ build with ld.lld on
armv7

r328169: Remove extra copy of bootinfo.c. It's a bit rotted copy of the one
in efi/loader.

r328288: Fix some resource leaks.

r328289: Don't leak memory when displaying help.

r328290: On malloc failure, be sure to close the include file that triggered
it.

r328291: getenv does not return tainted data in the boot loader. Attempt to
clue Coverity into that fact.

r328292: There's no tainted data here, tag it as such to avoid false
positives.

r328411: loader.efi: add missing EFI GUIDs

r328536: loader: support for mixed-endianness ELF/loader and POWER8

r328603: Add missing non-POWERPC case to give the scr value something
non-zero.

r328614: Move libstand.3 to libsa.3. Update libsa.3 to include functions

r328642: Break out the interpreters (simple and forth) w/o ifdefs.

r328769: Centralize several variables.

r328779: Retire pnp.4th and the code needed only for 4th words used here.

r328780: These 4th words were an attempt to allow integration into the boot
loader scripts. However, that path won't be taken after all it
seems.

r328781: Remove pcibios forth support.

r328782: Now that we no longer conditionally compile some files outside of
ficl

r328783: Invent new LDR_INTERP for the loader interpreter to use.

r328806: We need more heap space to properly load newer powerpc kernels.

r328808: Implement strcoll as strcmp.

r328826: Make cross-endian loader changes apply only to powerpc

r328835: Fix regression introduced in r328806, preventing boot on many
platforms.

r328911: Ignore relocation tables for non-memory-resident sections.

r328986: Fix relative location of USB sources after recent move.

r328987: A more definitions to kernel emulation shim in order to build
stand/usb.

r328990: Move the stand/usb test loader into its own directory.

r328999: Fix indentation to FreeBSD standard for interp files

r329000: Move simple interpreter 'perform' into interp.c and call it

r329019: Move to tabs for indentation and to 8-space notches, per style(9).

r329050: Fix build of userboot.so

r329054: Set script.lang in the environment to either 'forth' or 'simple' to
reflect what scripting language was compiled into the loader.

r329060: loader: fix endianness conversion

PR: 225323

MFC r325834,r325997,326502: Move sys/boot to stand/

This is effectively a direct commit to stable/11, due to differences between
stable/11 and head. Changes to DTS in sys/boot/fdt/dts were often
accompanied by kernel changes. Many of these were also risc-v updates that
likely had many more dependencies to MFC.

Because of this, sys/boot/fdt/dts remains as-is while everything else in
sys/boot relocates to stand/.

r325834: Move sys/boot to stand. Fix all references to new location

r325997: Remove empty directories.

r326502: Document the sys/boot -> stand move in hier.7 and the top-level README.

MFC Loader Fixes 2017q4p6: r324649,r324650,r324651,r324652,r324653,r324654,
r324700,r324702,r324709,r324717,r324719,r324841,r324842,r324843,r324845,
r324850,r324876,r324877,r324878,r324879,r324880,r324881,r324883,r324980,
r324981,r324982,r324995,r325014,r325093,r325094,r325114,r325170,r325171,
r325172,r325173,r325174,r325175,r325176,r325248,r325286,r325310,r325332,
r325338,r325339,r325376,r325377,r325379,r325380,r325382,r325478,r325479,
r325480,r325482,r325483,r325484,r325485,r325556,r325641,r325681,r325685,
r325686,r325687,r325688,r325689,r325690,r325691,r325692,r325693,r325694,
r325743,r325744,r325748,r325775,r325779,r325780

r324649: Move common/Makefile.inc to sys/boot/loader.mk.

r324650: tweak style

r324651: create defs.mk for common definitions

r324652: Move all the ficl common code into ficl.mk

r324653: LOADER_foo_SUPPORTED

r324654: Explicitly inlcude SYSDIR in the include path

r324700: loader: initialize dv_cleanup in md.c to eliminate clang warning

r324702: We need to include disk.o in libuboot.a when we're building with
support for disk access.

r324709: Revert "Unify boot1 with loader" change r324646

r324717: libsa/ip.c: misplaced comment, ip_v is half char, not ip_p

r324719: libsa/ip: stop read loop on bad fragments

r324841: Use BOOTDIR more consistently in defs.mk rather than repeat
sys/boot.

r324842: Introduce BOOTOBJ: The top level object directory for the boot tree

r324843: Stopgap fix to the mistmatch between LOADER_GELI_SUPPORT and
LOADER_NO_GELI_SUPPORT.

r324845: Use BOOTOBJ and BOOTDIR to find geli includes and libraries.

r324850: Define LIBSA32 to LIBSA on i386 to fix build.

r324876: Move fdt and uboot defines into common uboot.mk.

r324877: End source directories with SRC rather than a hodgepodge of names

r324878: Make at91 boot loader compile again.

r324879: Prefer SRCTOP paths for bits we're grabbing from libc.

r324880: Use BOOTSRC here.

r324881: Use SYSDIR instead of ${.CURDIR}/../..<etc>/sys.

r324883: Use preferred defined paths, rather than relative paths in fdt.

r324980: Use BOOTDIR consistently.

r324981: Move BINDIR definition to defs.mk, and override where it isn't
/boot

r324982: Remove sys/boot/arm/at91 and ixp425

r324995: loader.efi: Make framebuffer commands available for arm64

r325014: Add a 'place holder' arm struct efi_fb until a real one comes

r325093: Define new EFI variables

r325094: Cleanup non-arch Makefiles

r325114: Use defs.mk defins in most MD code

r325170: Use defs.mk values for userboot

r325171: Use defs.mk name and prefer bsd.init.mk

r325172: Remove the -nostdlib stuff I added. Instead, fix LDFLAGS to be
honored correctly with the new Makefile.inc include order.

r325173: We don't need to build a special ficl for userboot.

r325174: Minor cleanup

r325175: For amd64, compile both zfs and zfs32 libraries.

r325176: Actually add zfs32/Makefile

r325248: loader ptblread() is broken with >512B sectors

r325286: efipart_strategy is using wrong offset with >512B sectors

r325310: zfs.c:vdev_read() needs to be careful about large sectors

r325332: loader: re-enable gzip support for x86

r325338: loader: fix BOOTSRC -> BOOTOBJ in a library path

r325339: This used to have bzip2 support too.

r325376: WIP: centralize machine links

r325377: mostly libsa

r325379: Revert "mostly libsa"

r325380: Revert "WIP: centralize machine links"

r325382: Cleanup stray libstand names to be libsa names.

r325478: Powerpc is a 32-bit boot loader.

r325479: Define LIBFICL32 to be libficl.a on i386 and libficl32.a on amd64.

r325480: Use DO32 for all the places that we need to flag we're building a
32-bit version of a library.

r325482: Move machine and other link creation to defs.mk

r325483: MACHINE can never be powerpc64, so cleanup code that thinks it can.

r325484: Prefer bsd.init.mk to src.opts.mk

r325485: Centralize all 32-bit builds on 64-bit platform stuff.

r325556: loader: set options before including bsd.init.mk

r325641: loader.efi: efi_devpath_is_prefix should return bool

r325681: boot1: avoid using NULL device path

r325685: libsa32 isn't needed for i386. It's already a 32-bit platform.

r325686: Simplify this if to a direct assignment.

r325687: Remove all the empty help files from the powerpc build.

r325688: FDT support doesn't make sense for ps3, remove it.

r325689: Remove LOADER_FDT_SUPPORT as a Makefile variable.

r325690: Remove LOADER_ZFS_SUPPORT as a Makefile variable

r325691: Remove useless PNP define here.

r325692: Replace LOADER_FIREWIRE_SUPPORT variable

r325693: Move LOADER_{NO,}_GELI_SUPPORT to MK_LOADER_GELI

r325694: Install the 4th files in sys/boot/forth instead of each loader

r325743: Make sure the proper loader.rc gets installed.

r325744: boot1: also check for NULL device

r325748: Use proper include file.

r325775: Add loader.conf to the list of files that are MD.

r325779: Add /boot/dts to the list of default modules.

r325780: Don't add /boot/dt*s* but /boot/dt*b*. Stupid think-o.

MFC r303555,r303556,r303936,r303962,r304317,r304532,r305026,r305107,r305132,
r305178,r305353,r305814,r306159,r306380,r306504: Loader fixes, 2016q3

r303555: bcache should support reads shorter than sector size

r303556: Improve boot loader quote parsing

r303936: Add kernel environment variables under smbios.system

r303962: Add the missing space between .asciz directive and opening quote
for some lines with #ifdef BTXLDR_VERBOSE/#endif

r304317: boot1.efi Free() should check for NULL to provide consistent
behavior

r304532: Replace sprintf -> snprintf for command_errbuf provisioned from
dynamic content.

r305026: Emulate efi_cons_poll when WaitForKey is not available

r305107: Create a hook 'post-initialize' for people that want to define
something to read in .conf files after all other .conf files for the purpose
of overriding.

r305132: Remove accidentally committed stray comment.

r305178: bd_int13probe() should check extended info if sector info is bad

r305353: Don't use -N to set the OMAGIC with data and text writeable and
data not page aligned.

r305814: ufsread: Do not cast struct direct from void *

r306159: Consistently declare getsecs(void) with proper return type and void
when no arguments are present.

r306380: loader command interpreter should reset command_errmsg

r306504: Fix a cluster of bugs in list EFI environment variables

PR: 204602, 211958, 211958
Relnotes: yes ("Improve boot loader quote parsing")

MFC r310702: btxldr: process all PT_LOAD segments, not just the first two

With default settings GNU ld generates two PT_LOADs for loader.sym while
LLD generates three, because it creates a rodata segment. Previously
btxldr terminated phdr processing after two PT_LOADs. Remove the early
termination to process all PT_LOADs.

Sponsored by: The FreeBSD Foundation

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Implement (ACFLAGS|CFLAGS|CXXFLAGS).SRC globally.

Sponsored by: EMC / Isilon Storage Division

META MODE: Update dependencies with 'the-lot' and add missing directories.

This is not properly respecting WITHOUT or ARCH dependencies in target/.
Doing so requires a massive effort to rework targets/ to do so. A
better approach will be to either include the SUBDIR Makefiles directly
and map to DIRDEPS or just dynamically lookup the SUBDIR. These lose
the benefit of having a userland/lib, userland/libexec, etc, though and
results in a massive package. The current implementation of targets/ is
very unmaintainable.

Currently rescue/rescue and sys/modules are still not connected.

Sponsored by: EMC / Isilon Storage Division

Revert r267233 for now. PIE support needs to be reworked.

1. 50+% of NO_PIE use is fixed by adding -fPIC to INTERNALLIB and other
build-only utility libraries.
2. Another 40% is fixed by generating _pic.a variants of various libraries.
3. Some of the NO_PIE use is a bit absurd as it is disabling PIE (and ASLR)
where it never would work anyhow, such as csu or loader. This suggests
there may be better ways of adding support to the tree. Many of these
cases can be fixed such that -fPIE will work but there is really no
reason to have it in those cases.
4. Some of the uses are working around hacks done to some Makefiles that are
really building libraries but have been using bsd.prog.mk because the code
is cleaner. Had they been using bsd.lib.mk then NO_PIE would not have
been needed.

We likely do want to enable PIE by default (opt-out) for non-tree consumers
(such as ports). For in-tree though we probably want to only enable PIE
(opt-in) for common attack targets such as remote service daemons and setuid
utilities. This is also a great performance compromise since ASLR is expected
to reduce performance. As such it does not make sense to enable it in all
utilities such as ls(1) that have little benefit to having it enabled.

Reported by: kib

In preparation for ASLR [1] support add WITH_PIE to support building with -fPIE.

This is currently an opt-in build flag. Once ASLR support is ready and stable
it should changed to opt-out and be enabled by default along with ASLR.

Each application Makefile uses opt-out to ensure that ASLR will be enabled by
default in new directories when the system is compiled with PIE/ASLR. [2]

Mark known build failures as NO_PIE for now.

The only known runtime failure was rtld.

[1] http://www.bsdcan.org/2014/schedule/events/452.en.html
Submitted by: Shawn Webb <lattera@gmail.com>
Discussed between: des@ and Shawn Webb [2]

NO_MAN= has been deprecated in favor of MAN= for some time, go ahead
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.

btxldr: future-proof argument passing from boot1/2-ish to loader

Place the arguments at a fixed offset of 0x800 withing the argument area
(of size 0x1000). Allow variable size extended arguments first of which
should be a size of the extended arguments (including the size
parameter).

Consolidate all related definitions in a new i386/common/bootargs.h header.

Many thanks to jhb and bde for their guidance and reviews.

Reviewed by: jhb, bde
Approved by: jhb
MFC after: 1 month

Define several extra macros in bsd.sys.mk and sys/conf/kern.pre.mk, to
get rid of testing explicitly for clang (using ${CC:T:Mclang}) in
individual Makefiles.

Instead, use the following extra macros, for use with clang:
- NO_WERROR.clang (disables -Werror)
- NO_WCAST_ALIGN.clang (disables -Wcast-align)
- NO_WFORMAT.clang (disables -Wformat and friends)
- CLANG_NO_IAS (disables integrated assembler)
- CLANG_OPT_SMALL (adds flags for extra small size optimizations)

As a side effect, this enables setting CC/CXX/CPP in src.conf instead of
make.conf! For clang, use the following:

CC=clang
CXX=clang++
CPP=clang-cpp

MFC after: 2 weeks

Upgrade our copy of llvm/clang to r126079, from upstream's trunk.

This contains many improvements, primarily better C++ support, an
integrated assembler for x86 and support for -pg.

Use -Wl,-N instead of the undocumented -N option for GCC.

GCC forwards the -N flag directly to ld. This flag is not documented and
not supported by (for example) Clang. Just use -Wl,-N.

Submitted by: Pawel Worach

Remove now unused label.

Submitted by: Christoph Mallon
MFC after: 4 weeks
(along with r185779 and r185780)

Optimiza assembly in the previous r185779, to save whooping 16 bytes.

Submitted by: Christoph Mallon
MFC after: 4 weeks
(including r185779)

Respect RBX_MUTE flag from boot[012].

MFC after: 4 weeks

Start the dreaded NOFOO -> NO_FOO conversion.

OK'ed by: core

Back out last revision that unnecessarily changed valid assembler
line comments and damaged the CVS history.

Prompted by: bde, jhb

After talking to Bruce Evans and reading more standards specs,
switch to using C99-style comments everywhere in preprocessed
assembler. The reason is that lines starting with the regexp
'^[[:space:]]#' are treated as preprocessing directives, and
while it seems to work now with GCC, it's not necessarily has
to work. Use C99 comments `//' for the trailing comments to
save whitespace.

Use C (and CPP) style comments for assembler-with-cpp sources,
for lines that start with a comment.

Removed now redundant CLEANFILES assignments.

Not read enough of my patch by: obrien ;)

Use a more compact syntax for passing the "binary" options to 'ld'.

Simplify the building of our i386 'binary' boot components by directly
producing them using 'ld' options rather than post-processing with 'objcopy'.

Idea by: Ryan Sommers <ryans@gamersimpact.com>

Get rid of unnecessary use of m4(1) by using cpp(1) instead.
(John tells me there were problems when trying this before,
but it appears to be safe these day.)

OK'ed by: jhb
Repocopied by: joe

- Factor out -nostdlib to an upper level Makefile.inc.

- Now that bsd.prog.mk deals with programs linked with -nostdlib
better, and has a notion of an "internal" program, use PROG
where possible. This has a good impact on the contents of
.depend files and causes programs to be linked with cc(1).

XXX: boot2 couldn't be converted as it's actually two programs.

Tested on: i386, amd64

Fix an incorrect quote character in an M4 test conditon. Basically, one
of the verbose print statements that BTXLDR_VERBOSE enables wasn't properly
enabled.

Build on amd64. Yes, I know this isn't particularly nice.

Revert MEM_USR back to 0xa000 for BTX clients. Instead, adjust boot2
to run at 0xc000 by changing its virtual start address from 0x1000 to
0x2000.

Tested by: phk

Correctly adjust for moved start address.

It seems that the existence of a "depend" target in src/sys/boot is not
to be taken as an indication that it actually does what one would expect,
at least it clearly threw my testing off.

Apologies to: jhb

Initiate deorbit burn for the i386-only a.out related support. Moves are
under way to move the remnants of the a.out toolchain to ports. As the
comment in src/Makefile said, this stuff is deprecated and one should not
expect this to remain beyond 4.0-REL. It has already lasted WAY beyond
that.

Notable exceptions:
gcc - I have not touched the a.out generation stuff there.
ldd/ldconfig - still have some code to interface with a.out rtld.
old as/ld/etc - I have not removed these yet, pending their move to ports.
some includes - necessary for ldd/ldconfig for now.

Tested on: i386 (extensively), alpha

Unset MAINTAINER.

Set MAINTAINER.

Convert the short stub of real-mode code into 16-bit assembly.

Mega i386 loader commit.
- Don't hard code 0x10000 as the entry point for the loader. Instead add
src/sys/boot/i386/Makefile.inc which defines a make variable with the
entry point for the loader. Move the loader's entry point up to
0x20000, which makes PXE happy.
- Don't try to use cpp to parse btxldr for the optional BTXLDR_VERBOSE,
instead use m4 to achieve this. Also, add a BTXLDR_VERBOSE knob in the
btxldr Makefile to turn this option on.
- Redo parts of cdldr's Makefile so that it now builds and installs cdboot
instead of having i386/loader/Makefile do that. Also, add in some more
variables to make the pxeldr Makefile almost identical and thus to ease
maintainability.
- Teach cdldr about the a.out format. Cdldr now parsers the a.out header
of the loader binary and relocates it based on that. The entry point of
the loader no longer has to be hardcoded into cdldr. Also, the boot
info table from mkisofs is no longer required to get a useful cdboot.
- Update the lsdev function for BIOS disks to parse other file systems
(such as DOS FAT) that we currently support. This is still buggy as
it assumes that a floppy with a DOS boot sector actually has a MBR and
parses it as such. I'll be fixing this in the future.
- The biggie: Add in support for booting off of PXE-enabled network
adapters. Currently, we use the TFTP API provided by the PXE BIOS.
Eventually we will switch to using the low-level NIC driver thus
allowing both TFTP and NFS to be used, but for now it's just TFTP.

Submitted by: ps, alfred
Testing by: Benno Rice <benno@netizen.com.au>

Add the new cdldr CD bootstrap loader. This patch includes the following:

- Fix btxldr to preserve a NULL bootinfo pointer when it copies the kernel
arguments.
- Add the cdldr bootstrap program. This program is tacked onto the
beginning of the standard 3rd stage boot loader (/boot/loader) to form
the CD boot loader (/boot/cdboot). When a CD is booted, the cdboot file
is copied into memory instead and executed. The cdldr stub emulates the
environment normally provided by boot2 and then starts the loader. This
booting method does not emulate a floppy drive, but boots directly off of
the CD. This should fix the problems some BIOS's have with emulating a
2.88 MB floppy image.
- Add support to the loader to recognize that it has been booted by cdldr
instead of boot2 and use a simpler method of extracting the BIOS boot
device.

Blank out the twiddly thing when outputting the ``BTX loader'' message.

$Id$ -> $FreeBSD$

Push version numbers up to 1.00. This is just intended to reflect
that the BTX code can be regarded as stable: there are no associated
code changes.

Suggested by: obrien

Make the BTX loader much more quiet about what it's doing. This removes
most of the "what the (*^&%(*^ was that?" stuff that normally flies by.

Don't use an absolute path to objcopy.
Noticed by: Scott Mace <smace@intt.org>

Fix for "get base memory" bug found and mostly fixed by Mike.

The BIOS memory size is only a word. Some BIOSsen have garbage in the other
16 bits we were reading here, causing loss of arguments.

Make these compile when there is an obj dir.

Initial integration of the i386 bootloader and BTX.

- Discard large amounts of BIOS-related code in favour of the more compact
BTX vm86 interface.
- Build the loader module as ELF, although the resulting object is a.out,
make gensetdefs 32/64-bit sensitive and use a single copy of it.
- Throw away installboot, as it's no longer required.
- Use direct bcopy operations in the i386_copy module, as BTX
maps the first 16M of memory. Check operations against the
detected size of actual memory.

Resynch with working sources before BTX integration.

- Use format-independant module allocator.
- Conditionalise ISA PnP support.
- Simplify PnP enumerator interface.
- Improve module/object searching.
- Add missing depend/install targets in BTX makefiles.
- Pass the kernel environment and module data in extended bootinfo fields.
- Add a pointer to the end of the kernel + modules in bootinfo.
- Fix parsing of old-style kernel arguments.

Add btxldr, a BTX loader for ELF clients.