Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC UEFI loader

This MFC consists of the following SVN revisions:
258741 261568 261603 261668 263115 263117 263968 264078 264087 264088
264092 264095 264115 264132 264208 264261 264262 264263 264319 265028
265057 268974

Detailed commit messages:

r258741: Note that libstand is 32-bit on amd64 and powerpc64

r261568: Build libstand as a 64-bit library on amd64

The 32-bit bootloaders now link against libstand.a in
sys/boot/libstand32, so there is no need to force /usr/lib/libstand.a
to be 32-bit.

r261603: Don't force efi to a 32-bit build on amd64

r261668: Build libstand as a 64-bit library on ppc64

The 32-bit bootloaders now link against libstand.a in
sys/boot/libstand32, so there is no need to force /usr/lib/libstand.a
to be 32-bit.

This is equivalent to r261568 for amd64.

r263115: Add amd64 EFI headers

r263117: Connect 64-bit boot ficl to the build

It is not yet used, but this will ensure it doesn't get broken.

r263968: Use EFI types for EFI values (silences warnings).

EFI UINTN is actually a 64-bit type on 64-bit processors.

r264078: Put each source file on a separate line

This will simplify rebasing the amd64 UEFI patch set.

r264087: Build boot/ficl as 64-bit library on amd64

The 32-bit bootloaders on amd64 now use the 32-bit version in ficl32,
as is done with libstand32. The native 64-bit ficl will be used by the
upcoming UEFI loader.

r264088: Merge efilib changes from projects/uefi

r247216: Add the ability for a device to have an "alias" handle.

r247379: Fix network device registration.

r247380: Adjust our load device when we boot from CD under UEFI.

The process for booting from a CD under UEFI involves adding a FAT
filesystem containing your loader code as an El Torito boot image.
When UEFI detects this, it provides a block IO instance that points
at the FAT filesystem as a child of the device that represents the CD
itself. The problem being that the CD device is flagged as a "raw
device" while the boot image is flagged as a "logical partition".
The existing EFI partition code only looks for logical partitions and
so the CD filesystem was rendered invisible.

To fix this, check the type of each block IO device. If it's found to
be a CD, and thus an El Torito boot image, look up its parent device
and add that instead so that the loader will then load the kernel from
the CD filesystem. This is done by using the handle for the boot
filesystem as an alias.

Something similar to this will be required for booting from other media
as well as the loader will live in the EFI system partition, not on the
partition containing the kernel.

r247381: Remove a scatalogical debug printf that crept in.

r264092: Add -fPIC for amd64

r264095: Support UEFI booting on amd64 via loader.efi

This is largely the work from the projects/uefi branch, with some
additional refinements. This is derived from (and replaces) the
original i386 efi implementation; i386 support will be restored later.

Specific revisions of note from projects/uefi:

r247380:

Adjust our load device when we boot from CD under UEFI.

The process for booting from a CD under UEFI involves adding a FAT
filesystem containing your loader code as an El Torito boot image.
When UEFI detects this, it provides a block IO instance that points at
the FAT filesystem as a child of the device that represents the CD
itself. The problem being that the CD device is flagged as a "raw
device" while the boot image is flagged as a "logical partition". The
existing EFI partition code only looks for logical partitions and so
the CD filesystem was rendered invisible.

To fix this, check the type of each block IO device. If it's found to
be a CD, and thus an El Torito boot image, look up its parent device
and add that instead so that the loader will then load the kernel from
the CD filesystem. This is done by using the handle for the boot
filesystem as an alias.

Something similar to this will be required for booting from other
media as well as the loader will live in the EFI system partition, not
on the partition containing the kernel.

r246231:

Add necessary code to hand off from loader to an amd64 kernel.

r246335:

Grab the EFI memory map and store it as module metadata on the kernel.

This is the same approach used to provide the BIOS SMAP to the kernel.

r246336:

Pass the ACPI table metadata via hints so the kernel ACPI code can
find them.

r246608:

Rework copy routines to ensure we always use memory allocated via EFI.

The previous code assumed it could copy wherever it liked. This is not
the case. The approach taken by this code is pretty ham-fisted in that
it simply allocates a large (32MB) buffer area and stages into that,
then copies the whole area into place when it's time to execute. A more
elegant solution could be used but this works for now.

r247214:

Fix a number of problems preventing proper handover to the kernel.

There were two issues at play here. Firstly, there was nothing
preventing UEFI from placing the loader code above 1GB in RAM. This
meant that when we switched in the page tables the kernel expects to
be running on, we are suddenly unmapped and things no longer work. We
solve this by making our trampoline code not dependent on being at any
given position and simply copying it to a "safe" location before
calling it.

Secondly, UEFI could allocate our stack wherever it wants. As it
happened on my PC, that was right where I was copying the kernel to.
This did not cause happiness. The solution to this was to also switch
to a temporary stack in a safe location before performing the final
copy of the loaded kernel.

r246231:

Add necessary code to hand off from loader to an amd64 kernel.

r246335:

Grab the EFI memory map and store it as module metadata on the kernel.

This is the same approach used to provide the BIOS SMAP to the kernel.

r246336:

Pass the ACPI table metadata via hints so the kernel ACPI code can
find them.

r246608:

Rework copy routines to ensure we always use memory allocated via EFI.

The previous code assumed it could copy wherever it liked. This is not
the case. The approach taken by this code is pretty ham-fisted in that
it simply allocates a large (32MB) buffer area and stages into that,
then copies the whole area into place when it's time to execute. A more
elegant solution could be used but this works for now.

r247214:

Fix a number of problems preventing proper handover to the kernel.

There were two issues at play here. Firstly, there was nothing
preventing UEFI from placing the loader code above 1GB in RAM. This
meant that when we switched in the page tables the kernel expects to
be running on, we are suddenly unmapped and things no longer work. We
solve this by making our trampoline code not dependent on being at any
given position and simply copying it to a "safe" location before
calling it.

Secondly, UEFI could allocate our stack wherever it wants. As it
happened on my PC, that was right where I was copying the kernel to.
This did not cause happiness. The solution to this was to also switch
to a temporary stack in a safe location before performing the final
copy of the loaded kernel.

r247216:

Use the UEFI Graphics Output Protocol to get the parameters of the
framebuffer.

r264115: Fix printf format mismatches

r264132: Connect sys/boot/amd64 to the build

r264208: Do not build the amd64 UEFI loader with GCC

The UEFI loader causes buildworld to fail when building with (in-tree)
GCC, due to a typedef redefinition. As it happens the in-tree GCC
cannot successfully build the UEFI loader anyhow, as it does not support
__attribute__((ms_abi)). Thus, just avoid trying to build it with GCC,
rather than disconnecting it from the build until the underlying issue
is fixed.

r264261: Correct a variable's type for 64-bit Ficl

FICL_INT is long.

r264262: Fix printf args for 64-bit archs

r264263: Add explicit casts to quiet warnings in libefi

r264319: Fix EFI loader object tree creation on 9.x build hosts

Previously ${COMPILER_TYPE} was checked in sys/boot/amd64, and the efi
subdirectory was skipped altogether for gcc (since GCC does not support
a required attribute). However, during the early buildworld stages
${COMPILER_TYPE} is the existing system compiler (i.e., gcc on 9.x build
hosts), not the compiler that will eventually be used. This caused
"make obj" to skip the efi subdirectory. In later build stages
${COMPILER_TYPE} is "clang", and then the efi loader would attempt to
build in the source directory.

r265028 (dteske): Disable the beastie menu for EFI console ...

which doesn't support ANSI codes (so things like `at-xy', `clear', and
other commands don't work making it impossible to generate a living
menu).

r265057 (nwhitehorn): Turn off various fancy instruction sets...

as well as deduplicate some options. This makes the EFI loader build
work with CPUTYPE=native in make.conf on my Core i5.

r268974 (sbruno): Supress clang warning for FreeBSD printf %b and %D formats

Relnotes: Yes
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Fix typos - remove duplicate "the".

PR: bin/154928
Submitted by: Eitan Adler <lists at eitanadler.com>
MFC after: 3 days

Update to FICL 3.03 (the last release before FICL4 rewrite).

The relevant changes for FreeBSD (excerpt from the release note):

* Newly implemented CORE EXT words: CASE, OF, ENDOF, and ENDCASE. Also
added FALLTHROUGH, which works like ENDOF but jumps to the instruction
just after the next OF.
* Bugfix: John-Hopkins locals syntax now accepts | and -- in the comment
(between the first -- and the }.)
* Bugfix: Changed vmGetWord0() to make Purify happier. The resulting
code is no slower, no larger, and slightly more robust.

Fix typos, mostly s/ an / a / where appropriate and a few s/an/and/
Add FreeBSD Id tag where missing.

Make ficl work on sparc64. The assumption that int == long == void * is
very pervasive in this code. This fixes a few of those assumptions and
band-aids over some others.

Tested on: ia32 alpha sparc64

Reviewed by: peter jake (in concept)

Upgrade to FICL version 3.02. Anything wrong is my fault, everything right is
due Jon Mini.

PR: 36308
Submitted by: Jon Mini <mini@haikugeek.com>
MFC after: 4 weeks

Avoid an alignment fault on 64bit architectures.

Reviewed by: dcs

As in with ":", check dictionary space and increase it if necessary before
"create".

FICL 2.05 has a flawed definition of BASE. Fix it.

Spotted by: Peter Jeremy <peter.jeremy@alcatel.com.au>

Bring in ficl 2.05.

This version has a step debugger, which now completely replaces the
old trace feature. Also, we moved all of the FreeBSD-specific MI
code to loader.c, reducing the diff between this and the official
FICL distribution.

Only build PnP parts if the platform supports PnP.

Add the infrastructure necessary to handle PnP from a Forth script.

Also, export the file_findfile() function. Again, this is taken from
work in progress but frozen for the time being. Since it works, I'd
rather commit and remove any uglyness later than hide it on my tree.

Fix REFILL. It must throw RESTART instead of OUTOFTEXT so that
execution can take place at the point where it stopped after the
input buffer has been refilled.

Add ANS Forth CORE EXT and FILE word SOURCE-ID.

Add setenv, getenv, setenv?, unsetenv, copyin and copyout to FICL.

Apply a number of fixes for the Alpha platform.

Add "int" sized manipulation words.

Bring in FICL 2.04. No bump of loader version is required by this
commit.

Lay the groundwork for on-demand dictionary expansion.

Add heap?, so we can tell how much free space we have left.

(Hopefully) make all necessary changes for ficl to support alpha.

Bring in ficl version 2.03. No version bump for loader.

Remove dictHashSummary internal function; it used doubles.

Submitted by: Daniel C. Sobral <dcs@newsguy.com>

The previous fix for "type" was absolutely lousy. Not only the buffer
allocated was not big enough, but it ended up to being used where it
was supposed to be used. The person who did that ought to be shot, but
since I'm a good person, I'll forgive myself...

PR: bin/9743

From the PR:

I added a FICL_TRACE-conditioned trace facility based on "see".
It is ugly because words' functions are almost all static, and ficlExec,
where the trace has to be located, can't get their pointers. So, #ifdef
this staticization, and add most of see's body into ficlExec. Duplication
of code, uglyness, etc. But it is cleanly #ifdef'ed, and works like a
charm.

It does not provide "step" facility, though, just trace. It is
tunable at run-time through "trace!". If anyone (most likely me :) ever
wants a step facility, I'll add it. Should be easy.

PR: bin/9652
Submitted by: "Daniel C. Sobral" <dcs@newsguy.com>

From the PR:

FICL's TYPE copies the counted string to HERE, as abial has
remarked. Answering to abial's question, this is NOT garanteed to have
enough space.
...
We have dynamic memory. Even before memory-alloc got in, we
already had dynamic memory. Use it, then! (ficlMalloc is sysdep, so I
suppose that's why it was not used for TYPE; ficl is probably designed
to work without a working ficlFree).

Submitted by: "Daniel C. Sobral" <dcs@newsguy.com>

Add EXCEPTION word set.

Make TIB handling use buffer size to conform with ANS Forth.

Add ANS MEMORY-ALLOC word set.

See the PRs for extensive details.

PR: kern/9412 kern/9442 kern/9514
Submitted by: PRs from Daniel Sobral <dcs@newsguy.com>

Change counter-intuitive pc@ and pc! to inb and outb.

Submitted by: jkh

Add words "pc!" and "pc@" which allow to manipulate I/O ports. This can
be helpful when directly fiddling with PC hardware.

Also, define "arch-i386" appropriately, as suggested by Daniel C. Sobral.

Add support for some FACILITY words:

key? ( -- flag) \ check to see if there's a key to be read from input
ms ( u -- ) \ wait that many milliseconds
seconds ( -- u ) \ get number of seconds from midnight.

'words' now outputs the list page by page - this probably should go
through libstand's pager, but will have to wait for closer integration of
built-ins with Forth...

Submitted partially by: W Gerald Hicks <wghicks@bellsouth.net>

Add new word ".#" which acts like "." but doesn't output a space separator.
I found it impossible to output parametrized strings like "\033[%d;%dH"
without building them first in pad area.

o Add proper stack checking to all file words
o add fkey and fread
o eliminate fexists now that this can be expressed in HLL forth
( : fexists fopen dup -1 <> if fclose 1 else drop 0 then ; ) :-)

Once we get the ability to write files, it should be possible to do
stand-alone rescue work from the 3rd stage boot. :)

Eliminate much code cruft by extending simple file I/O API to include
fopen and fclose.

OK, fload/fexec work properly when both compiling and interpreting
now. Gack.

switch words to default mode.

Make fexists/fload work with existing string literals instead. Doing
my own string literal handling is just too wonky.

Argh, I had this right[er] before. More commits to follow.

Use proper literal names.

Remember a bit more of my forth and do:

o Add fexists word to check for the presence of a file
o make fexists and fload immediate words which DTRT both interpreted
and compiled (doh!)
o add an init word which gets run at bootstrapping time to do extra
post-coldload initialization (in the default implementation, we
look for /boot/boot.4th and load it if found).

1. rebuild all elements of testmain properly for safety.
2. add fload and key prims for doing simple file and terminal I/O, respectively

Fix includes; we were still pulling in <stdlib.h> in some places.
Support building a standalone interpreter with 'make testmain', remove
win32 and egregious gets() in testmain.c

Make this work in the libstand environment; don't use stdio/stdlib headers.

Add the Ficl (Forth Inspired Command Language) interpreter. If all goes well,
this will allow us to manage bloat in the loader by using a bytecoded HLL
rather than lots of C code. It also offers an opportunity for vendors
or others with special applications to significantly customise the boot
process without having to commit to a divergent code branch.

This early commit is to allow others to experiment with the most effective
mechanisms for integrating FICL with the loader as it currently stands.

Ficl is distributed with the following license conditions:

"Ficl is freeware. Use it in any way that you like, with the understanding
that the code is not supported."

All source files contain authorship attributions.

Obtained from: John Sadler (john_sadler@alum.mit.edu)