MFC r351865, r351837:
ficl: add xemit word
ficl: add uIsGreater word (needed for xemit)

We need xemit to support userboot.so loading images from CURRENT.

MFC r330806-r330815, r330837, r330864, r330883

pc98 changes associated with the named commits are also included in this
commit, despite not having been made with the original commits due to its
removal in head.

r330806:
Minor cosmetic changes.

Make sure { on the same line as struct for all struct *devdesc. Move
some type definitions to next to the dv_type define, since that's what
sets the d_type.

r330807:
We can't use d_opendata for blkio storage.

open_disk uses d_opendata for it's own purpse. We can't store blkio
there. Fortunately, blkio is stored elsewhere and we never actually
retrieve blkio from d_opendata. Eliminate it as a source of confusion.
Eliminate all stores of d_opendata in efi since this layer doesn't own
that field.

r330808:
Make struct libi386_devdesc match the struct devdesc better

Move data to top and call it d_opendata.

r330809:
Use the actual struct devdesc at the start of all *_devdesc structs

The current system is fragile and requires very careful layout of all
*_devdesc structures. It also makes it hard to change the base
devdesc. Take a page from CAM and put the 'header' in all the derived
classes and adjust the code to match.

For OFW, move the iHandle h_handle out of a slot conflicting with
d_opendata. Due to quirks in the alignment rules, this worked.
However changing the code to use d_opendata storage now that it's a
pointer is hard, so just have a separate field for it.

All other cleanups were to make the *_devdesc structures match where
they'd taken some liberties that were none-the-less compatible enough
to work.

r330810:
Remove d_type from devdesc. It's not needed as we can fetch it from
d_dev->dv_type when we need it.

r330811:
GC unused routines.

Sponsored by: Netflix

r330812:
Use the one-line-per-file pattern here, and sort the file names.

Sponsored by: Netflix

r330813:
Move the env convenience routines out of boot1.c.

These routines are more generally useful. Even though boot1 is on its
way out, it's better to make these common during the transition than
copy them.

r330814:
Star BootCurrent entry when booting.

Sponsored by: Netflix

r330815:
Print the load and device path as well as BootCurrent and BootOrder

Sponsored by: Netflix

r330837:
biosdisk.c should not set d_opendata.

Same as 330807, d_opendata is owned by open_disk and we should not
set it.

M stand/i386/libi386/biosdisk.c

r330864:
Prefer uintXX_t to u_intXX_t

A foolish consistency is the hobgoblin of little minds, adored by
little statesmen and philosophers and divines. With consistency a
great soul has simply nothing to do. -- Ralph Waldo Emerson

r330883:
Fix typo that misteriously passes compilation.

MFC r329345: Eliminate bsd.stand.mk and -fPIC 32-bit intel builds

OK. We don't really need a bsd.stand.mk, and it was causing a -fPIC
for the toolchain to be added (bogusly) when building on amd64. Pull
all relevant defs back into defs.mk and delete bsd.stand.mk.

This saves about 15-20k on i386 loader and zfsloader which when
combined with Lua give us a lot more stack space in those constrained
environments.

MFC r330612: stand/ficl: Fix testmain

testmain is a userland application intended to be built with standard
headers and whatnot, which we broke.

Fix it by having the testmain build clobber cflags, reducing it to just the
set of defines/includes it needs to build.

MFC Loader Fixes Final: r327612,r327703,r327704,r327878,r327879,r327881,
r328007,r328029,r328030,r328031,r328061,r328156,r328169,r328288,r328289,
r328290,r328291,r328292,r328411,r328536,r328603,r328614,r328642,r328769,
r328779,r328780,r328781,r328782,r328783,r328806,r328808,r328826,r328835,
r328911,r328986,r328987,r328990,r328999,r329000,r329019,r329050,r329054,
r329060

r327612: Invent new #defines for the biospci_{read,write}_config function

r327703: Define __dmadat after #include'ing ufsread.c.

r327704: Fix printf missing format variables warnings.

r327878: Add GUID for UEFI boot manager variables.

r327879: Report the boot order and where we are in that boot order.

r327881: Allow this file to be included

r328007: Fix booting on some arm64 systems after r327879 by fixing the call
to utf8_to_ucs2

r328029: When returning an error and freeing allocated memory from
ucs2_to_utf8, NULL the return pointer.

r328030: Check the return value from utf8_to_ucs2 instead of whether or not
uv is NULL.

r328031: Need to free uv after we're done using it.

r328061: utf8_to_ucs2() should check for malloc failure

r328156: stand: Move sections around to fix stand/ build with ld.lld on
armv7

r328169: Remove extra copy of bootinfo.c. It's a bit rotted copy of the one
in efi/loader.

r328288: Fix some resource leaks.

r328289: Don't leak memory when displaying help.

r328290: On malloc failure, be sure to close the include file that triggered
it.

r328291: getenv does not return tainted data in the boot loader. Attempt to
clue Coverity into that fact.

r328292: There's no tainted data here, tag it as such to avoid false
positives.

r328411: loader.efi: add missing EFI GUIDs

r328536: loader: support for mixed-endianness ELF/loader and POWER8

r328603: Add missing non-POWERPC case to give the scr value something
non-zero.

r328614: Move libstand.3 to libsa.3. Update libsa.3 to include functions

r328642: Break out the interpreters (simple and forth) w/o ifdefs.

r328769: Centralize several variables.

r328779: Retire pnp.4th and the code needed only for 4th words used here.

r328780: These 4th words were an attempt to allow integration into the boot
loader scripts. However, that path won't be taken after all it
seems.

r328781: Remove pcibios forth support.

r328782: Now that we no longer conditionally compile some files outside of
ficl

r328783: Invent new LDR_INTERP for the loader interpreter to use.

r328806: We need more heap space to properly load newer powerpc kernels.

r328808: Implement strcoll as strcmp.

r328826: Make cross-endian loader changes apply only to powerpc

r328835: Fix regression introduced in r328806, preventing boot on many
platforms.

r328911: Ignore relocation tables for non-memory-resident sections.

r328986: Fix relative location of USB sources after recent move.

r328987: A more definitions to kernel emulation shim in order to build
stand/usb.

r328990: Move the stand/usb test loader into its own directory.

r328999: Fix indentation to FreeBSD standard for interp files

r329000: Move simple interpreter 'perform' into interp.c and call it

r329019: Move to tabs for indentation and to 8-space notches, per style(9).

r329050: Fix build of userboot.so

r329054: Set script.lang in the environment to either 'forth' or 'simple' to
reflect what scripting language was compiled into the loader.

r329060: loader: fix endianness conversion

PR: 225323

MFC Loader Fixes 2017q4p7: r324844,r326089,r326926,r326440,r326484,r326494,
r326588,r326708,r326784,r326914,r327390,r328446,r326090,r326143,r326144,
r326182,r326384,r326421,r326440,r326441,r326442,r326443,r326444,r326445,
r326446,r326447,r326448,r326484,r326485,r326486,r326487,r326488,r326490,
r326491,r326492,r326493,r326494,r326495,r326504,r326507,r326509,r326584,
r326585,r326586,r326587,r326588,r326589,r326590,r326591,r326592,r326593,
r326594,r326600,r326616,r326671,r326707,r326708,r326709,r326710,r326711,
r326712,r326714,r326720,r326768,r326772,r326784,r326792,r326812,r326854,
r326855,r326856,r326858,r326886,r326887,r326914,r326926,r326927,r326960,
r326961,r326962,r326963,r327351,r327453,r327390,r327523,r327524,r326489,
r327880,r328437,r328438,r328439,r328441,r328446,r328448,r328449,r328612,
r328613,r328615

While here, undo our libfdt hack of not including <stdlib.h> if we're
compiling _STANDALONE.

r324844: When building standalone, don't define errno. Let the definition from
stand.h override. This is similar to what we do in the kernel.

r326089: loader.efi: efipart does not recognize partitionless disks

r326090: net_parse_rootpath() has no parameters

r326143: Fix theoretical integer overflow issues. If the product here is

r326144: Mark the func pointer as __dead2. It looks up loader_main, which

r326182: Modify all FreeBSD bootloaders on PowerPC AIM (Book-S) systems

r326384: Use const pointers to avoid casting away constness.

r326421: loader.efi: efipart should exclude iPXE stub block protocol

r326440: Remove stale dependency on ufsread.c

r326441: Minor flags cleanup

r326442: Cleanup CFALGS usage here

r326443: We don't need both _STAND and _STANDALONE, use the latter.

r326444: Move geli to common DO32 stuff

r326445: Fix random() and srandom() prototypes to match the standard.

r326446: Undefine _STANDALONE since this is test code.

r326447: Tweaks to the beri boot loader so that it builds w/o warnings.

r326448: Fix all warnings related to geli and ZFS support on x86.

r326484: Const poison the propname.

r326485: Delcare md_load in libofw.h. Make all prototypes match for ofw

r326486: Include machine/md_var to pick up __syncicache prototype.

r326487: Cast mdp (a vm_offset_t) to void * to match prototype.

r326488: e_entry can be smaller than a pointer. Cast it to an intptr_t

r326490: Declare our strange brand of main().

r326491: Disconnet ps3 from the build. There's too many warnings to fix.

r326492: Cast void * pointer to char * so the arg matches the %s format.

r326493: Provide a md_load64 prototype.

r326494: Mark two things as unused (since they are only sometimes used)

r326495: Now it's safe to bump WARNS to 1.

r326504: Switch to proper MK_LOADER_GELI tests.

r326507: increase maximum size of zfsboot

r326509: loader.efi: add note about iPXE into the efipart.c

r326584: When building standalone, include stand.h rather than the kernel

r326585: Include ficl.h before anything else

r326586: No need to include the userland md5.h, the kernel one is just fine.

r326587: Use the kernel relative paths, rather than the userland relative
paths

r326588: Need to include skein in the include path

r326589: Make sure we include the right path for skein.h

r326590: Prefer stdint.h to inttypes.h

r326591: This isn't NetBSD specific code. Include these for any kernel /

r326592: Don't inherit CFLAGS. This a specialized test program.

r326593: Stop building with the standard system headers.

r326594: Now that we offer a semi-sane standards-ish set of #include files,
stop hacking includes with sed.

r326600: Since this is contrib code, create an upstreamable version of my

r326616: dhcp_try_rfc1048() is not used any more

r326671: Avoid setting -Wno-tentative-definition-incomplete-type with gcc.

r326707: Add partial support signal.h functioanlity. Pull in
machine/signal.h

r326708: Remove _KERNEL hack now that errno.h does the right thing when
built standalone.

r326709: Provide implementations for iscntrl, ispunct and isgraph.

r326710: Put the files we're copying over into a few variables and clean
hings up.

r326711: Const poison a couple of interfaces.

r326712: Create interp class.

r326714: boot1.c needs EFI_ZFS_BOOT too, so add it globally.

r326720: This path belongs in ficl/Makefile, not the common defines for
users

r326768: Fix a comment to be more accurate

r326772: Fix regression with lua import

r326784: Revert part of 362772. It was causing problems for includes

r326792: Attempt to unbreak buildworld

r326812: Revert r326792, r326784, r326772, r326712

r326854: libefi: make efichar.h more usable in stand code

r326855: Cargo cut a fix for the regressions r326585 caused.

r326856: Fix comments after bump in size.

r326858: Revert r326855: Cargo cut a fix for the regressions r326585 caused.

r326886: Panic in sbrk if setheap hasn't been called yet. This is preferable
o a mysterious crash

r326887: Remove the 'mini libstand in libstand' that util.[ch] provided.

r326914: Move loader-only defines to loader.mk from defs.mk

r326926: Move loader help file definitions to being 100% inside of
loader.mk.

r326927: libficl is only ever used in a loader (never a boot) program. Move
it.

r326960: Simplify things a little. The RETURN macro isn't required.

r326961: Interact is always called with NULL. Simplify code a little

r326962: Hoist btx include stuff to i386/Makefile.inc

r326963: No need to use relative paths like this here.

r327351: Fix ubldr. uboot/lib uses defines for the loader.

r327453: Add a validbcd() routine that uses the bcd2bin_data[] array

r327390: Garbage-collect loader.ps3. It is currently disconnected from the
build and kboot replaces.

r327523: Don't clobber system LDFLAGS for beri boot loaders.

r327524: Use 'extern uint8_t' instead of 'extern void' for external symbols.

r326489: Allow this file to be used in libsa without warning...

r327880: Move getsecs() prototype to stand.h from net.h so it can be used

r328437: Split panic routine

r328438: Implement abort() as a call to panic.

r328439: Provide abs form stdlib.h.

r328441: abort() should be marked __dead2 since it won't return.

r328446: Now that exit is __dead2, we need to tag ub_exit() as __dead2.

r328448: Make exit() never return until host_exit can be written.

r328449: Tag unreachable places as such. I left the while (1); in place

r328612: Move strtold wrapper from strtol.c to its own strtold.c.

r328613: Kill copies of strtol and strtoul.

r328615: Update stand.h for changes for strto*l

PR: 223969

MFC r325834,r325997,326502: Move sys/boot to stand/

This is effectively a direct commit to stable/11, due to differences between
stable/11 and head. Changes to DTS in sys/boot/fdt/dts were often
accompanied by kernel changes. Many of these were also risc-v updates that
likely had many more dependencies to MFC.

Because of this, sys/boot/fdt/dts remains as-is while everything else in
sys/boot relocates to stand/.

r325834: Move sys/boot to stand. Fix all references to new location

r325997: Remove empty directories.

r326502: Document the sys/boot -> stand move in hier.7 and the top-level README.

MFC Loader Fixes 2017q4p6: r324649,r324650,r324651,r324652,r324653,r324654,
r324700,r324702,r324709,r324717,r324719,r324841,r324842,r324843,r324845,
r324850,r324876,r324877,r324878,r324879,r324880,r324881,r324883,r324980,
r324981,r324982,r324995,r325014,r325093,r325094,r325114,r325170,r325171,
r325172,r325173,r325174,r325175,r325176,r325248,r325286,r325310,r325332,
r325338,r325339,r325376,r325377,r325379,r325380,r325382,r325478,r325479,
r325480,r325482,r325483,r325484,r325485,r325556,r325641,r325681,r325685,
r325686,r325687,r325688,r325689,r325690,r325691,r325692,r325693,r325694,
r325743,r325744,r325748,r325775,r325779,r325780

r324649: Move common/Makefile.inc to sys/boot/loader.mk.

r324650: tweak style

r324651: create defs.mk for common definitions

r324652: Move all the ficl common code into ficl.mk

r324653: LOADER_foo_SUPPORTED

r324654: Explicitly inlcude SYSDIR in the include path

r324700: loader: initialize dv_cleanup in md.c to eliminate clang warning

r324702: We need to include disk.o in libuboot.a when we're building with
support for disk access.

r324709: Revert "Unify boot1 with loader" change r324646

r324717: libsa/ip.c: misplaced comment, ip_v is half char, not ip_p

r324719: libsa/ip: stop read loop on bad fragments

r324841: Use BOOTDIR more consistently in defs.mk rather than repeat
sys/boot.

r324842: Introduce BOOTOBJ: The top level object directory for the boot tree

r324843: Stopgap fix to the mistmatch between LOADER_GELI_SUPPORT and
LOADER_NO_GELI_SUPPORT.

r324845: Use BOOTOBJ and BOOTDIR to find geli includes and libraries.

r324850: Define LIBSA32 to LIBSA on i386 to fix build.

r324876: Move fdt and uboot defines into common uboot.mk.

r324877: End source directories with SRC rather than a hodgepodge of names

r324878: Make at91 boot loader compile again.

r324879: Prefer SRCTOP paths for bits we're grabbing from libc.

r324880: Use BOOTSRC here.

r324881: Use SYSDIR instead of ${.CURDIR}/../..<etc>/sys.

r324883: Use preferred defined paths, rather than relative paths in fdt.

r324980: Use BOOTDIR consistently.

r324981: Move BINDIR definition to defs.mk, and override where it isn't
/boot

r324982: Remove sys/boot/arm/at91 and ixp425

r324995: loader.efi: Make framebuffer commands available for arm64

r325014: Add a 'place holder' arm struct efi_fb until a real one comes

r325093: Define new EFI variables

r325094: Cleanup non-arch Makefiles

r325114: Use defs.mk defins in most MD code

r325170: Use defs.mk values for userboot

r325171: Use defs.mk name and prefer bsd.init.mk

r325172: Remove the -nostdlib stuff I added. Instead, fix LDFLAGS to be
honored correctly with the new Makefile.inc include order.

r325173: We don't need to build a special ficl for userboot.

r325174: Minor cleanup

r325175: For amd64, compile both zfs and zfs32 libraries.

r325176: Actually add zfs32/Makefile

r325248: loader ptblread() is broken with >512B sectors

r325286: efipart_strategy is using wrong offset with >512B sectors

r325310: zfs.c:vdev_read() needs to be careful about large sectors

r325332: loader: re-enable gzip support for x86

r325338: loader: fix BOOTSRC -> BOOTOBJ in a library path

r325339: This used to have bzip2 support too.

r325376: WIP: centralize machine links

r325377: mostly libsa

r325379: Revert "mostly libsa"

r325380: Revert "WIP: centralize machine links"

r325382: Cleanup stray libstand names to be libsa names.

r325478: Powerpc is a 32-bit boot loader.

r325479: Define LIBFICL32 to be libficl.a on i386 and libficl32.a on amd64.

r325480: Use DO32 for all the places that we need to flag we're building a
32-bit version of a library.

r325482: Move machine and other link creation to defs.mk

r325483: MACHINE can never be powerpc64, so cleanup code that thinks it can.

r325484: Prefer bsd.init.mk to src.opts.mk

r325485: Centralize all 32-bit builds on 64-bit platform stuff.

r325556: loader: set options before including bsd.init.mk

r325641: loader.efi: efi_devpath_is_prefix should return bool

r325681: boot1: avoid using NULL device path

r325685: libsa32 isn't needed for i386. It's already a 32-bit platform.

r325686: Simplify this if to a direct assignment.

r325687: Remove all the empty help files from the powerpc build.

r325688: FDT support doesn't make sense for ps3, remove it.

r325689: Remove LOADER_FDT_SUPPORT as a Makefile variable.

r325690: Remove LOADER_ZFS_SUPPORT as a Makefile variable

r325691: Remove useless PNP define here.

r325692: Replace LOADER_FIREWIRE_SUPPORT variable

r325693: Move LOADER_{NO,}_GELI_SUPPORT to MK_LOADER_GELI

r325694: Install the 4th files in sys/boot/forth instead of each loader

r325743: Make sure the proper loader.rc gets installed.

r325744: boot1: also check for NULL device

r325748: Use proper include file.

r325775: Add loader.conf to the list of files that are MD.

r325779: Add /boot/dts to the list of default modules.

r325780: Don't add /boot/dt*s* but /boot/dt*b*. Stupid think-o.

MFC r324648: Rename top level Makefile.ficl to ficl.mk.

MFC r307322,r307323,r307324,r307326,r307327,r307338,r307879,r307908,r307911,
r307942,r307950,r307951,r307954,r307955,r308125,r308195,r308476,r308534,
r308535,r308776,r308843,r310236,r310726: Loader fixes, 2016q4

r307322: Remove /boot/boot.conf, deprecated for 16 years

r307323: Remove fetching of pInterp.

r307324: Create a new linker set, Xficl_compile_set which contains a list of
functions to call to register new forth words.

r307326: In UEFI mode expose the SMBIOS anchor base address via kenv

r307327: Update i386 build of loader.efi (but leave it disabled) so that we
at least build it now.

r307338: Create a pcibios-version environment FORTH word.

r307879: Preliminary support for EFI in boot loader.

r307908: Fix the build on both arm64 and when WITHOUT_FORTH is defined.

r307911: Add better comment...

r307942: Really make WITHOUT_FORTH (MK_FORTH==no) work.

r307950: Add it (Makefile.ficl) to the right place

r307951: Fix two backwards tests.

r307954: Back out the move to the loader script from -N.

r307955: LIBSTAND goes last, so put it last here too.

r308125: In loader.efi, instead of exiting directly, try to fallback.

r308195: efinet_dev_print should honor verbose option.

r308476: boot/forth spelling issue in forth word

r308534: The file_loadraw function grew an argument, update install function

r308535: Add support for LOADER_RC setting in the pkgfs manifes

r308776: loader: zfs toplevel vdev must have spa set.

r308843: loader: smbios version check is not correct

r310236: Renumber license clauses to avoid skipping #3

r310726: cdboot: add explict suffix to ambiguous or instruction

PR: 214375

MFC r313982, r314068:
sys: Replace zero with NULL for pointers.

Found with: devel/coccinelle

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Implement UEFI set environment variable, as well as exporting the EFI
version. This is also scriptable, though additional scripting will be
needed.

Differential Review: https://reviews.freebsd.org/D4494
MFC After: 3 days

Implement uuid-to-string and uuid-from-string. uuid-from-string takes
a string, interprets it as a standard UUID, and returns a binary from
of the UUID. uuid-to-string does the reverse. The binary UUID is in
allocated memory, so you'll need to free it with 'free' after you are
done using it. It won't be automatically garbage collected. Likewise
with the string...

MFC After: 3 days

Minor space tweak to make things consistent.

MFC After: 3 days

Add more .NOMETA missed in r291320

Sponsored by: EMC / Isilon Storage Division

Currently BERI's loader is including the 32-bit version of the FICL MIPS sysdep.h (sys/boot/ficl/mips/sysdep.h) instead of the 64-bit version (sys/boot/ficl/mips64/sysdep.h).

Although this may not be an issue in practice, it would be more correct if the 64-bit version was used. Also, using the 64-bit version would make it easier to add support for 64-bit ubldr on MIPS.

Approved by: adrian (mentor)
Differential Revision: https://reviews.freebsd.org/D5310

DIRDEPS_BUILD: Regenerate without local dependencies.

These are no longer needed after the recent 'beforebuild: depend' changes
and hooking DIRDEPS_BUILD into a subset of FAST_DEPEND which supports
skipping 'make depend'.

Sponsored by: EMC / Isilon Storage Division

ficl: Replace rand(3) with random(3).

Be a little more consistent with random(3) and push an
unsigned value.

Again, this has no effect as this code doesn't get compiled
for the boot code.

ficl: Replace rand(3) with random(3).

While the later is a better random generator than the former, the main
reason of the change is that random() has a better chance to work with
libstand(3).

At this time we don't include random number generators in bootforth
so this has no effect.

Welcome the RISC-V 64-bit kernel.

This is the final step required allowing to compile and to run RISC-V
kernel and userland from HEAD.

RISC-V is a completely open ISA that is freely available to academia
and industry.

Thanks to all the people involved! Special thanks to Andrew Turner,
David Chisnall, Ed Maste, Konstantin Belousov, John Baldwin and
Arun Thomas for their help.
Thanks to Robert Watson for organizing this project.

This project sponsored by UK Higher Education Innovation Fund (HEIF5) and
DARPA CTSRD project at the University of Cambridge Computer Laboratory.

FreeBSD/RISC-V project home: https://wiki.freebsd.org/riscv

Reviewed by: andrew, emaste, kib
Relnotes: Yes
Sponsored by: DARPA, AFRL
Sponsored by: HEIF5
Differential Revision: https://reviews.freebsd.org/D4982

ficl on i386 should cast to unsigned char output to support efi i386

make it possible for efi_console to recognize and translate box characters
on i386 build (unsigned versus signed char passed as int issue).

Submitted by: Toomas Soome <tsoome at me.com>
Reviewed by: emaste, smh, dteske
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D4993

loader.efi: add terminal emulation support

This is based on the vidconsole implementation.

Submitted by: Toomas Soome <tsoome@me.com>
Reviewed by: adrian
MFC after: 2 weeks
Relnotes: Yes
Differential Revision: https://reviews.freebsd.org/D4797

META_MODE: Remove DEP_MACHINE from Makefile.depend files.

This has not been needed since r246865 in projects/bmake.

Sponsored by: EMC / Isilon Storage Division

META_MODE: Remove DEP_RELDIR from Makefile.depend files.

This has not been needed since r284171 in projects/bmake.

Sponsored by: EMC / Isilon Storage Division

Use CFLAGS_NO_SIMD in preference to varying lists of -mno-xxxx flags.
Go ahead and defined -D_STANDALONE for all targets (only strictly
needed for some architecture, but harmless on those it isn't required
for). Also add -msoft-float to all architectures uniformly rather
that higgley piggley like it is today.

Differential Revision: https://reviews.freebsd.org/D3496

Add META_MODE support.

Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision: D2796
Reviewed by: brooks imp

Revert my previoius commit as it explicit pollute the ficl on other
architectures. It will be reworked by GSoC students.

Requested by: ngie, bms and pfg.

Include header libi386.h to silence the clang warning of implicit declaration
of functions biospci_*.

Differential Revision: D2668
Reviewed by: rodrigc

Add the arm64 machine dependent parts of ficl. This is not machine
dependent, as most copies of these files are identical, but unfortunately
this is still needed.

Sponsored by: The FreeBSD Foundation

The System V ABI for amd64 allows functions to use space in a 128 byte
redzone below the stack pointer for scratch space and requires
interrupt and signal frames to avoid overwriting it. However, EFI uses
the Windows ABI which does not support this. As a result, interrupt
handlers in EFI push their interrupt frames directly on top of the
stack pointer. If the compiler used the red zone in a function in the
EFI loader, then a device interrupt that occurred while that function
was running could trash its local variables. In practice this happens
fairly reliable when using gzipfs as an interrupt during decompression
can trash the local variables in the inflate_table() function
resulting in corrupted output or hangs.

Fix this by disabling the redzone for amd64 EFI binaries. This
requires building not only the loader but any libraries used by the
loader without redzone support.

Thanks to Jilles for pointing me at the redzone once I found the stack
corruption.

Differential Revision: https://reviews.freebsd.org/D2054
Reviewed by: imp
MFC after: 2 weeks
Sponsored by: Cisco Systems, Inc.

Cleanup trailing newlines noticed while looking for other issues.

Add support for calling pcibios routines from the
bootloader. Implement the following routines:
pcibios-device-count count the number of instances of a devid
pcibios-read-config read pci config space
pcibios-write-config write pci config space
pcibios-find-devclass find the nth device with a given devclass
pcibios-find-device find the nth device with a given devid
pcibios-locator convert bus device function ti pcibios locator
These commands are thin wrappers over their PCI BIOS 2.1 counterparts. More
informaiton, such as it is, can be found in the standard.

Export a nunmber of pcibios.X variables into the environment to report
what the PCI IDENTIFY command returned.

Also implmenet a new command line primitive (pci-device-count), but don't
include it by default just yet, since it depends on the recently added
words and any errors here can render a system unbootable.

This is intended to allow the boot loader to do special things based
on the hardware it finds. This could be have special settings that are
optimized for the specific cards, or even loading special drivers. It
goes without saying that writing to pci config space should not be
done without a just cause and a sound mind.

Sponsored by: Netflix

Remove ia64.

This includes:
o All directories named *ia64*
o All files named *ia64*
o All ia64-specific code guarded by __ia64__
o All ia64-specific makefile logic
o Mention of ia64 in comments and documentation

This excludes:
o Everything under contrib/
o Everything under crypto/
o sys/xen/interface
o sys/sys/elf_common.h

Discussed at: BSDcan

Fix printf args for 64-bit archs

Sponsored by: The FreeBSD Foundation

Correct a variable's type for 64-bit Ficl

FICL_INT is long.

Fix the mips64el build: mips64el should use the mips64 directory, not the mips64el directory.

Build boot/ficl as 64-bit library on amd64

The 32-bit bootloaders on amd64 now use the 32-bit version in ficl32,
as is done with libstand32. The native 64-bit ficl will be used by the
upcoming UEFI loader.

Sponsored by: The FreeBSD Foundation

Build FICL support into little-endian 64-bit MIPS boot-loader fragments;
while this won't actually be used for anything (yet), it doesn't hurt to
ensure it is exposed to the tinderbox.

Requested by: imp, jmallett
MFC after: 3 weeks

Give mips64 its own ficl configuration that sizes pointers/etc suitably for
a 64-bit architecture, rather than trying to share the 32-bit MIPS ficl
configuration.

When building ficl on MIPS, don't conflate 32-bit and 64-bit ISAs -- unlike
x86, we don't want a 32-bit loader on a 64-bit CPU. Use quite conservative
code generation -- e.g., softfloat, no GOT, etc -- suitable for early boot.

MFC after: 3 weeks
Sponsored by: DARPA, AFRL

For libstand and sys/boot, split off gcc-only flags into CFLAGS.gcc.

MFC after: 3 days
X-MFC-With: r259730

Fix the freaddir implementation for the stand-alone interpreter.

Bug pointed out by: Jan Beich <jbeich@tormail.org>

Fix the build of the testmain target. This target compiles a Forth
interpreter that can be run on the system and as such cannot be
compiled against libbstand. On the one hand this means we need to
include the usual headers for system interfaces that we use and
on the the other hand we can only use standard system interfaces.

While here, define local variables only when needed to make this
WARNS=2 clean on amd64.

PR: 172542
Obtained from: peterj@
Pointed out by: Jan Beich <jbeich@tormail.org>

Add 2 builtin words for working with directories:
isdir? ( fd -- bool )
freaddir ( fd -- ptr len TRUE | FALSE )

The 'isdir?' word returns `true' if the file descriptor is for a
directory and `false' otherwise.

The 'freaddir' word reads the next directory entry and if successful,
returns its name and 'true'. Otherwise 'false' is returned.

These words give the loader the ability to scan directories and read
files contained in them for 'rc.d'-like flexibility in handling which
modules to load and/or which tunables to set.

Obtained from: Juniper Networks, Inc.

The CELL fields can be various integer types depending on the platform
(ARM uses 'int' and 'unsigned'; i386 uses 'long' and 'unsigned long'),
so we need explicit casts to long and unsigned long here to ensure
that the result matches the printf %ld and %lx specifiers.

Fix two broken macros.

Build a 64 bit version of the FICL library. This is need for a 64 bit
EFI bootloader (amd64 only for now).

This is not yet connected to the build.

boot: use -march=i386 for both i386 and amd64 builds

.. so that consistent compilation algorithms are used for both
architectures as in practice the binaries are expected to be
interchangeable (for time being).
Previously i386 used default setting which were equivalent to
-march=i486 -mtune=generic.
The only difference is using smaller but slower "leave" instructions.

Discussed with: jhb, dim
MFC after: 29 days

Try to avoid all files dependence on the modification time of the large and
often modified directory created symbolic links points to - it cause
unnecessary full rebuilds each time make runs when directory is changed.
So do it only if symbolic link does not exists, which usually means that
objdir is clean anyway.

MFC after: 1 week

Fix clang warnings.

Add a version of the FreeBSD bootloader which can run in userland, packaged
as a shared library. This is intended to be used by BHyVe to load FreeBSD
kernels into new virtual machines.

Fix typos - remove duplicate "the".

PR: bin/154928
Submitted by: Eitan Adler <lists at eitanadler.com>
MFC after: 3 days

On i386 and amd64, consistently use the following options whenever we
want to avoid using any "advanced" CPU features:

-mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -msoft-float

Remove an empty .if/.endif pair from sys/boot/ficl/Makefile that I
forgot to cleanup in the last commit.

In lib/libstand, sys/boot/ficl and sys/boot/zfs, -mno-sse3 should also
be used for amd64, not just for i386.

MF tbemd: Minor tweaks, prefer MACHINE_CPUARCH generally to MACHINE_ARCH (which simplifies some powerpc/powerpc64 ifs)

Provide support in loader for booting 64-bit PowerPC kernels. Like amd64,
64-bit PowerPC kernels are loaded by a 32-bit loader, since nearly all
powerpc64 firmwares execute in 32-bit mode.

Add a missing prototype

PR: 145232
Submitted by: gcooper
MFC after: 1 week

Add basic mips support. This has been compile tested only.

Initial support of loader(8) for ARM machines running U-Boot.

This uses the common U-Boot support lib (sys/boot/uboot, already used on
FreeBSD/powerpc), and assumes the underlying firmware has the modern API for
stand-alone apps enabled in the config (CONFIG_API).

Only netbooting is supported at the moment.

Obtained from: Marvell, Semihalf

Optimize for size on pc98. It enables to boot a kernel again.

I don't know what's wrong (loader, boot2 or others), but this change is
effective.

Tested by: NAKAJI Hiroyuki
MFC after: 3 days

Update to FICL 3.03 (the last release before FICL4 rewrite).

The relevant changes for FreeBSD (excerpt from the release note):

* Newly implemented CORE EXT words: CASE, OF, ENDOF, and ENDCASE. Also
added FALLTHROUGH, which works like ENDOF but jumps to the instruction
just after the next OF.
* Bugfix: John-Hopkins locals syntax now accepts | and -- in the comment
(between the first -- and the }.)
* Bugfix: Changed vmGetWord0() to make Purify happier. The resulting
code is no slower, no larger, and slightly more robust.

Add -march=i386 to fix amd64 build by generating the same code
as i386 would do.

Copy powerpc ficl files sysdep files (maybe we need a common one)

Remove Alpha bits for ficl.

Add -mno-sse3 for prescott/nocona

Ensure GCC does not use FP registers in integer code.
I really don't like cluttering up the compiler invocation,
but this bigger hammer will fix reported problems for now.

Instead of just not defining a bunch of words when TESTMAIN is set, provide
stubs that at least handle the stack correctly. This makes it much easier to
experiment with loader scripts from userland.

For variables that are only checked with defined(), don't provide
any fake value.

Part 2 of fixing the boot code: gcc 3.4 fixes.

The whole problem seems to be size. Which is odd, because it is said
that size doesn't matter. Anyway... Add -Os to strategic places in the
makefile to have the final loader be as mall as possible. This seems
to be enough to make it work. For now... I think something is more
fundamentally wrong; or something more fundamental is wrong. Potato,
potaato.

Untangle building of AMD64 boot code.

Tested on: amd64 (sledge)

Disable floating point on PowerPC for the ficl library.

Define 'arch-powerpc'

FICL system-dependent files for powerpc. Taken from the i386 versions,
which were the most relevant.

bsd.lib.mk,v 1.143 no longer uses ld(1) directly to strip
symbols from intermediate object files, so these hacks to
get AMD64 compile are no longer needed.

Tested on: sledge.FreeBSD.org

Build on amd64. Yes, I know this isn't particularly nice.

Set arch-pc98 env to true for pc98.

Fix typos, mostly s/ an / a / where appropriate and a few s/an/and/
Add FreeBSD Id tag where missing.

Make ficl work on sparc64. The assumption that int == long == void * is
very pervasive in this code. This fixes a few of those assumptions and
band-aids over some others.

Tested on: ia32 alpha sparc64

Reviewed by: peter jake (in concept)

Fix for stand-alone compiling

Reviewed by: mini

MD ficl files for sparc64.

More s/file system/filesystem/g

Removed now unused INTERNALSTATICLIB.
INTERNALLIB now implies NOPIC and NOPROFILE.
Removed gratuitous NOMAN.

Back out last commit. I expect our bsd.*.mk gods to remove the need for
defining so many extra things in addition to INTERNALLIB. We don't like
repetitive C code and we shouldn't for make code either.

NOPIC, NOPROFILE, NOMAN, and INTERNALSTATICLIB are redundant when using
INTERNALLIB now.

-ffreestanding is the word.
(also resort some CFLAGS such that the more "important" value are first so
they are easier to see)

Partical style cleanup.

Upgrade to FICL version 3.02. Anything wrong is my fault, everything right is
due Jon Mini.

PR: 36308
Submitted by: Jon Mini <mini@haikugeek.com>
MFC after: 4 weeks

- Add 'fwrite' and 'fseek' words for writing to and seeking on files.
- Change the 'fopen' keyword to accept a mode parameter. Note that this
will break existing 4th scripts that use fopen. Thus, the loader
version has been bumped and loader.4th has been changed to check for a
sufficient version on i386 and alpha. Be sure that you either do a full
world build or install or full build and install of sys/boot after this
since loader.old won't work with the new 4th files and vice versa.

PR: kern/32389
Submitted by: Jonathan Mini <mini@haikugeek.com>
Sponsored by: ClickArray, Inc.

Remove unused filed.

PR: kern/32395
Submitted by: Jonathan Mini <mini@haikugeek.com>

Revert rev 1.7 which moved us away from POSIX character classes.
The community feels our base AWK must handle them.

Substitute "[:space:]" with the character constants it expands to.
This was a silent 'failure' when using Bell-Labs awk.

Submitted by: David Wolfskill <david@catwhisker.org>

Avoid an alignment fault on 64bit architectures.

Reviewed by: dcs

Add definition for arch-ia64.

Add ia64 system dependancies.

Turn dictthreshold and dictincrease into cell-sized variables, so
they can be used with cell operators like !.

As I did this, I noticed the whole CELL thing might have problems with
big endian architectures with sizeof(int)!=sizeof(void*).

As in with ":", check dictionary space and increase it if necessary before
"create".

FICL 2.05 has a flawed definition of BASE. Fix it.

Spotted by: Peter Jeremy <peter.jeremy@alcatel.com.au>

Bring in ficl 2.05.

This version has a step debugger, which now completely replaces the
old trace feature. Also, we moved all of the FreeBSD-specific MI
code to loader.c, reducing the diff between this and the official
FICL distribution.

setlocale(3) has been fixed to match POSIX standard:
LC_ALL takes precedence over other LC_* envariables.

Cosmetique: don't use localized `date`

Don't use the Gawkism strftime(). Pass in the date stamp on the awk
command line instead.

Approved by: dcs

Fix this so that we actually define HAVE_PNP when its available.

Only build PnP parts if the platform supports PnP.

Add the infrastructure necessary to handle PnP from a Forth script.

Also, export the file_findfile() function. Again, this is taken from
work in progress but frozen for the time being. Since it works, I'd
rather commit and remove any uglyness later than hide it on my tree.

Add -mpreferred-stack-boundary=2 to CFLAGS for i386. This and libstand
reduces /boot/loader from 163840 bytes to 155648 and pxeboot from 165888
bytes to 157696 bytes.

Fix REFILL. It must throw RESTART instead of OUTOFTEXT so that
execution can take place at the point where it stopped after the
input buffer has been refilled.

Add ANS Forth CORE EXT and FILE word SOURCE-ID.

Make abort" functional in interpret mode. This behavior is undefined
by ANS Forth standard, but it's useful.

Also, define the constant true in a more strict way. C might garantee
2-complement math, but Forth doesn't.

Make comment reflect reality.

Add setenv, getenv, setenv?, unsetenv, copyin and copyout to FICL.

Remove a bug that caused local variables declarations in the format
{ | internal -- } to be rejected.

Apply a number of fixes for the Alpha platform.

Add something that was missing from the original 2.04 distribution.

Add "int" sized manipulation words.

Strip spaces and comments more agressively.

Bring in FICL 2.04. No bump of loader version is required by this
commit.

Stop libficl from using floating point registers - this was cauing
dictDelete() to fault early on the Alpha and was the original cause of
the Alpha ficl failures.

Lay the groundwork for on-demand dictionary expansion.

Correct a bug whereby allocations to the dictionary would not be allowed
unless four times the space requested was available.

Add heap?, so we can tell how much free space we have left.

Implement a machine-independent (word size-independent) FICL_TRUE.

ANS Forth for logical not is 0=, not invert. Replace wrong usage.

Add testmain.o to CLEANFILES.

Try my hand again at removing ugly testmain target.

Zeroes structure before using it.

Revert ill-considered simplification in 1.13.

Pointed by: peter

Improve Makefile by using .PATH.

Taught by: peter

Improve on ficl port to alpha.

Files sysdep.[ch] are now in ${MACHINE_ARCH} subdirectory. Internal
#if's used to identify the platform where removed.

Make rule for target testmain was greatly simplified, because it was
easier simplifying it than changing it to support the new location of
sysdep.[ch].

(a repo-copy was done on sysdep.[ch], of course)

(Hopefully) make all necessary changes for ficl to support alpha.

Fix world-breaking bug, add $FreeBSD$ tag.

This happened to be my first "for real" broken world. I had broken
it once before, but nobody noticed, so it didn't count.

So, how do I get the "I broke world and all I got was the lousy t-shirt"
t-shirt?

Bring in ficl version 2.03. No version bump for loader.

$Id$ -> $FreeBSD$

Reactivate trace! after making it egcs-friendly.

Disable trace! until it gets fixed.

Remove redundant code.

Make $ print different lines in, well, different lines.

Remove dictHashSummary internal function; it used doubles.

Submitted by: Daniel C. Sobral <dcs@newsguy.com>

Make FICL_TRACE default.

Add freebsd.fr, for FreeBSD specific forth source. Add $ and %, to
replace the lost @ and - functionality of include. $ has the opposite
behavior of @ though, since the default behavior was inverted.

Make ficlExec recursive. This removes vmReset and vmQuit from the
error cases, but the replacement should be doing everything they
did, except what did shouldn't be doing, and might do a little more
they ought to be doing.

Fix ficlExecFD to take kern/9412 changes into account. Also modify it
not to vmThrow errors. This is not what the comments say it does, and
it doesn't work when there is no ficlExec environment (like it's only
use in sys/boot/common/interp_forth.c).

PR: bin/9772

The trace facility was full of bugs. Correct that by using the
right variable.

PR: bin/9756

The previous fix for "type" was absolutely lousy. Not only the buffer
allocated was not big enough, but it ended up to being used where it
was supposed to be used. The person who did that ought to be shot, but
since I'm a good person, I'll forgive myself...

PR: bin/9743

Add conditional compilation facilities to ficl. This had been previously
committed to RELENG_3 instead of HEAD, so let's HEAD catch up.

PR: bin/9662

From the PR:

I added a FICL_TRACE-conditioned trace facility based on "see".
It is ugly because words' functions are almost all static, and ficlExec,
where the trace has to be located, can't get their pointers. So, #ifdef
this staticization, and add most of see's body into ficlExec. Duplication
of code, uglyness, etc. But it is cleanly #ifdef'ed, and works like a
charm.

It does not provide "step" facility, though, just trace. It is
tunable at run-time through "trace!". If anyone (most likely me :) ever
wants a step facility, I'll add it. Should be easy.

PR: bin/9652
Submitted by: "Daniel C. Sobral" <dcs@newsguy.com>

From the PR:

FICL's TYPE copies the counted string to HERE, as abial has
remarked. Answering to abial's question, this is NOT garanteed to have
enough space.
...
We have dynamic memory. Even before memory-alloc got in, we
already had dynamic memory. Use it, then! (ficlMalloc is sysdep, so I
suppose that's why it was not used for TYPE; ficl is probably designed
to work without a working ficlFree).

Submitted by: "Daniel C. Sobral" <dcs@newsguy.com>

Add EXCEPTION word set.

Make TIB handling use buffer size to conform with ANS Forth.

Add ANS MEMORY-ALLOC word set.

See the PRs for extensive details.

PR: kern/9412 kern/9442 kern/9514
Submitted by: PRs from Daniel Sobral <dcs@newsguy.com>

Transition from using Perl to using awk for our text-manipulation
needs. This removes the dependancy on Perl for the generation of the
loader, allowing the world to be built on a perl-free system.

Submitted by: Joe Abley <jabley@clear.co.nz>

Change counter-intuitive pc@ and pc! to inb and outb.

Submitted by: jkh

Add words "pc!" and "pc@" which allow to manipulate I/O ports. This can
be helpful when directly fiddling with PC hardware.

Also, define "arch-i386" appropriately, as suggested by Daniel C. Sobral.

Add support for some FACILITY words:

key? ( -- flag) \ check to see if there's a key to be read from input
ms ( u -- ) \ wait that many milliseconds
seconds ( -- u ) \ get number of seconds from midnight.

'words' now outputs the list page by page - this probably should go
through libstand's pager, but will have to wait for closer integration of
built-ins with Forth...

Submitted partially by: W Gerald Hicks <wghicks@bellsouth.net>

Add new word ".#" which acts like "." but doesn't output a space separator.
I found it impossible to output parametrized strings like "\033[%d;%dH"
without building them first in pad area.

o Add proper stack checking to all file words
o add fkey and fread
o eliminate fexists now that this can be expressed in HLL forth
( : fexists fopen dup -1 <> if fclose 1 else drop 0 then ; ) :-)

Once we get the ability to write files, it should be possible to do
stand-alone rescue work from the 3rd stage boot. :)

Eliminate much code cruft by extending simple file I/O API to include
fopen and fclose.

Take init out of here - it's a layering violation.

OK, fload/fexec work properly when both compiling and interpreting
now. Gack.

switch words to default mode.

Submit better fix to the prompt bug.

Make fexists/fload work with existing string literals instead. Doing
my own string literal handling is just too wonky.

Argh, I had this right[er] before. More commits to follow.

Use proper literal names.

Don't not print the prompt string; not doing this has funny side-effects.
Make the prompt an empty string instead.

Remember a bit more of my forth and do:

o Add fexists word to check for the presence of a file
o make fexists and fload immediate words which DTRT both interpreted
and compiled (doh!)
o add an init word which gets run at bootstrapping time to do extra
post-coldload initialization (in the default implementation, we
look for /boot/boot.4th and load it if found).

1. rebuild all elements of testmain properly for safety.
2. add fload and key prims for doing simple file and terminal I/O, respectively

Explicitly invoke perl to run the softwords.pl script in case the
script is not executable.
Submitted by: Don Lewis <Don.Lewis@tsc.tdk.com>

Improve the assertion testing for successful softword compilation.
The excessive use of assert() vs. robust error checking is annoying.

Fix includes; we were still pulling in <stdlib.h> in some places.
Support building a standalone interpreter with 'make testmain', remove
win32 and egregious gets() in testmain.c

Make this work in the libstand environment; don't use stdio/stdlib headers.

Add the Ficl (Forth Inspired Command Language) interpreter. If all goes well,
this will allow us to manage bloat in the loader by using a bytecoded HLL
rather than lots of C code. It also offers an opportunity for vendors
or others with special applications to significantly customise the boot
process without having to commit to a divergent code branch.

This early commit is to allow others to experiment with the most effective
mechanisms for integrating FICL with the loader as it currently stands.

Ficl is distributed with the following license conditions:

"Ficl is freeware. Use it in any way that you like, with the understanding
that the code is not supported."

All source files contain authorship attributions.

Obtained from: John Sadler (john_sadler@alum.mit.edu)