#
344413 |
|
21-Feb-2019 |
kevans |
MFC Loader Interpreter Coexistance: r337806, r337868, r337914, r337927, r337989, r338052, r338064, r338407, r338418, r338474
r337806: Create a loader for each interpreter for x86 BIOS and all EFI
Create loader_{4th,lua,simp}{,.efi}. All of these are installed by default. Create LOADER_DEFAULT_INTERP to specify the default interpreter when no other is specified. LOADER_INTERP is the current interpreter language building. Turn building of lua on by default to match 4th. simploader is a simplified loader build w/o any interpreter language (but with a simple loader). This is the historic behavir you got with WITHOUT_FORTH. Make a hard link to the default loader. This has to be a hard link rather than the more desirable soft link because older zfsboot blocks don't support symlinks.
r337868: stand: Use -Oz/-Os for all loader/stand builds.
While we're not super size constrained, the x86 BIOS /boot/loader has to be less than about 520k-530k to be reliable. The LUA loader is at this size today. -Oz saves 15-20% on the size, keeping us safely small enough (comparable to where we were with the 4th loader). This will also help with sjg's work on bringing in bearssl, though we may again be looking for space in the LUA loader.
Size table for clang 6.0.0: default -O1 -Os -Oz 4th 442368 417792 389120 376832 lua 524288 479232 446464 430080
r337914: Install links for loader.efi.
r337927: Add ashldi3 and ashrdi3 to mips.
Now that we're using -Os, mips needs these routines.
r337989: Copy the boot loader from the new location for the co-existing loaders.
r338052: libsa: Add lshrdi3.c for powerpc* and mips
r338064: Turn back the clock just a little: make userboot.so always be 4th
Turns out there was a hidden dependency we hasn't counted upon. The host load /boot/userboot.so to boot the VMs it runs. This means that the change to lua meant suddently that nobody could run their older VMs because LUA wasn't in 10.0, last month's HardenedBSD, 11.2 or whatever. Even more than for the /boot/loader* binaries, we need a good coexistance strategy for this. While that's being designed and implemented, drop back to always 4th for userboot.so. This will fail safe in all but the most extreme environments (but lua-only hacks to .lua files won't be processes in VMs until we fix it).
r338407: lualoader: Print error messages from command failures at the prompt
Previously lualoader would remain silent, rather than printing command_errmsg or noting that a command had failed or was not found.
r338418: userboot: handle guest interpreter mismatches more intelligently
The switch to lualoader creates a problem with userboot: the host is inclined to build userboot with Lua, but the host userboot's interpreter must match what's available on the guest. For almost all FreeBSD guests in the wild, Lua is not yet available and a Lua-based userboot will fail.
This revision updates userboot protocol to version 5, which adds a swap_interpreter callback to request a different interpreter, and tries to determine the proper interpreter to be used based on how the guest /boot/loader is compiled. This is still a bit of a guess, but it's likely the best possible guess we can make in order to get it right. The interpreter is now embedded in the resulting executable, so we can open /boot/loader on the guest and hunt that down to derive the interpreter it was built with.
Using -l with bhyveload will not allow an intepreter swap, even if the loader specified happens to be a userboot with the wrong interpreter. We'll simply complain about the mismatch and bail out.
For legacy guests without the interpreter marker, we assume they're 4th. For new guests with the interpreter marker, we'll read it and swap over to the proper interpreter if it doesn't match what the userboot we're using was compiled with.
Both flavors of userboot are installed by default, userboot_4th.so and userboot_lua.so. This fixes the build WITHOUT_FORTH as a coincidence, which was broken by userboot being forced to 4th.
r338474: Be a little conservative about when to force size optimizations.
Reports have come in that there's issue with powerpc and sparc64 since we've switched to using -Oz / -Os. We don't strictly need them for !x86, so be conservative about when we enable them.
|
#
344399 |
|
21-Feb-2019 |
kevans |
MFC GELI Loader Improvements: r336252, r336254, r336256, r336354, r336532-r336534, r336537, r336626, r337326, r337349, r341071, r341160, r341420, r341473, r341651, r342793
Note that this MFC contains some seemingly unrelated zfsloader bits -- this was needed in order to pull in some later fixes for GELI hand-off w/ ZFS bits included.
r336252: Extend loader(8) geli support to all architectures and all disk-like devices.
This moves the bulk of the geli support from lib386/biosdisk.c into a new geli/gelidev.c which implements a devsw-type device whose dv_strategy() function handles geli decryption. Support for all arches comes from moving the taste-and-attach code to the devopen() function in libsa.
After opening any DEVT_DISK device, devopen() calls the new function geli_probe_and_attach(), which will "attach" the geli code to the open_file struct by creating a geli_devdesc instance to replace the disk_devdesc instance in the open_file. That routes all IO for the device through the geli code.
A new public geli_add_key() function is added, to allow arch/vendor-specific code to add keys obtained from custom hardware or other sources.
With these changes, geli support will be compiled into all variations of loader(8) on all arches because the default is WITH_LOADER_GELI.
r336254: Use if rather than case for a simple boolean. gcc thinks blks is undefined sometimes with the case, but enc is always 0 or 1, so and if / else is better anyway.
r336256: Fix glitched indentation (and rewrap as needed due to deeper indent). No functional changes.
r336354: zfsboot: fix build with WITHOUT_LOADER_GELI
r336532: Collapse zfsloader functionality back down into loader.
We no longer really need a separate zfsloader. It was useful when we were first supporting ZFS and had limited ability to properly boot off of ZFS without the special boot loader. Now that the boot loader has matured, go the way loader.efi pioneered and just build one binary. Change the name of the loader to load in the secondary boot blocks to be just /boot/loader. Provide a symbolic link from zfsloader to loader so people who have not upgraded their boot blocks are not affected. This has the happy benefit of making coexistence easier as well (fewer binaries in the matrix).
r336533: Eliminate zfsloader man page.
Remove all cross references to zfsloader.8 and /boot/zfsloader. Move ZFS specific info into loader.8.
r336534: NM and OBJCOPY are already defined for all builds. There's no need to conditionally define them here.
r336537: Mention zfsloader being folded into loader in UPDATING.
r336626: Older zfs boot blocks don't support symlinks. install the link to zfsloader as a hard link. While newer ones do, the whole point of the link was to transition to the new world order smoothly. A hard link is less flexible, but it works and will result in fewer bumps. Adjust UPDATING entry to match.
r337326: loader: biosdisk.c has leftover geli header.
A small cleanup, remove unneeded #include.
r337349: zfsboot: Fix startup crash
On a FreeNAS mini XL, with geli encrypted drives the loader crashed in geli_read().
When we iterate over the list of disks and allocate the zfsdsk structures we don’t zero out the gdev pointer. In one case that resulted in geli_read() (called on the bogus pointer) dividing by zero.
Use calloc() to ensure the zfsdsk structure is always zeroed, so the pointer is initialised to NULL. As a side benefit it gets rid of one #ifdef LOADER_GELI_SUPPORT.
r341071: Restore the ability to override the disk unit/partition at the boot: prompt in gptboot.
When arch-independent geli support was added, a new static 'gdsk' struct was added, but there was still a static 'dsk' struct, and when you typed in an alternate disk/partition, the string was parsed into that struct, which was then never used for anything. Now the string gets parsed into gdsk.dsk, the struct that's actually used.
r341160: Add comments describing the bootargs handoff between loader(8) and gptboot or zfsboot, when loader(8) is the BTX loader. No functional changes.
r341420: Eliminate duplicated code and struct member definitions in the handoff of args data between gptboot/zfsboot and loader(8).
Despite what seems like a lot of changes here, there are no actual changes in behavior, or in the data layout in the structures involved. This is just eliminating identical code pasted into multiple locations.
In detail, the changes are...
- Move struct zfs_boot_args definition from libsa/zfs/libzfs.h to i386/common/bootargs.h because it is specific to x86 booting and the handoff between zfsboot and loader, and has no relation to the zfs library code in general.
- The geli_boot_args and zfs_boot_args structs both contain an identical set of member variables containing geli information. Extract this out to a new geli_boot_data struct, and embed it in the arg-passing structs.
- Provide new routines geli_import_boot_data() and geli_export_boot_data() that can be shared between gptboot, zfsboot, and loader instead of pasting identical code into several different .c files.
- Remove some checks for a NULL pointer that can never be true because the pointer being tested was set using pointer math (kargs + 1) and that can never result in NULL in this code.
r341473: Fix args cross-threading between gptboot(8) and loader(8) with zfs support.
When loader(8) is built with zfs support enabled, it assumes that any extarg data present is a zfs_boot_args struct, but if the first-stage loader was gptboot(8) the extarg data is actually a geli_boot_args struct. Luckily, zfsboot(8) and gptzfsboot(8) have always passed KARGS_FLAGS_ZFS along with KARGS_FLAGS_EXTARG, so we can use KARGS_FLAGS_ZFS to decide whether the extarg data is a zfs_boot_args struct.
To avoid similar problems in the future, gptboot(8) now passes a new KARGS_FLAGS_GELI to indicate that extarg data is geli_boot_args. In loader(8), if the neither KARGS_FLAGS_ZFS nor KARGS_FLAGS_GELI is set but extarg data is present (which will be the case for gptboot compiled before this change), we now check for the known size of the geli_boot_args struct passed by the older versions of gptboot as a way of confirming what type of extarg data is present.
In a semi-related tidying up, since loader's main() has already decided what type of extarg data is present and set the global 'zargs' var accordingly, don't repeat the check in extract_currdev, just check whether zargs is NULL or not.
r341651: Don't reference zfs-specific variables if LOADER_ZFS_SUPPORT is undefined because the variables will be undefined too.
r342793: MK_ZFS -> {MK_ZFS|MK_LOADER_ZFS}, this is so we can diable userland / kernel ZFS but keep the boot-loaders when using ZoL port.
Relnotes: yes (GELI support extended) Relnotes: yes (zfsloader has been collapsed into loader and may be removed after boot blocks have been updated)
|
#
329183 |
|
12-Feb-2018 |
kevans |
MFC Loader Fixes Final: r327612,r327703,r327704,r327878,r327879,r327881, r328007,r328029,r328030,r328031,r328061,r328156,r328169,r328288,r328289, r328290,r328291,r328292,r328411,r328536,r328603,r328614,r328642,r328769, r328779,r328780,r328781,r328782,r328783,r328806,r328808,r328826,r328835, r328911,r328986,r328987,r328990,r328999,r329000,r329019,r329050,r329054, r329060
r327612: Invent new #defines for the biospci_{read,write}_config function
r327703: Define __dmadat after #include'ing ufsread.c.
r327704: Fix printf missing format variables warnings.
r327878: Add GUID for UEFI boot manager variables.
r327879: Report the boot order and where we are in that boot order.
r327881: Allow this file to be included
r328007: Fix booting on some arm64 systems after r327879 by fixing the call to utf8_to_ucs2
r328029: When returning an error and freeing allocated memory from ucs2_to_utf8, NULL the return pointer.
r328030: Check the return value from utf8_to_ucs2 instead of whether or not uv is NULL.
r328031: Need to free uv after we're done using it.
r328061: utf8_to_ucs2() should check for malloc failure
r328156: stand: Move sections around to fix stand/ build with ld.lld on armv7
r328169: Remove extra copy of bootinfo.c. It's a bit rotted copy of the one in efi/loader.
r328288: Fix some resource leaks.
r328289: Don't leak memory when displaying help.
r328290: On malloc failure, be sure to close the include file that triggered it.
r328291: getenv does not return tainted data in the boot loader. Attempt to clue Coverity into that fact.
r328292: There's no tainted data here, tag it as such to avoid false positives.
r328411: loader.efi: add missing EFI GUIDs
r328536: loader: support for mixed-endianness ELF/loader and POWER8
r328603: Add missing non-POWERPC case to give the scr value something non-zero.
r328614: Move libstand.3 to libsa.3. Update libsa.3 to include functions
r328642: Break out the interpreters (simple and forth) w/o ifdefs.
r328769: Centralize several variables.
r328779: Retire pnp.4th and the code needed only for 4th words used here.
r328780: These 4th words were an attempt to allow integration into the boot loader scripts. However, that path won't be taken after all it seems.
r328781: Remove pcibios forth support.
r328782: Now that we no longer conditionally compile some files outside of ficl
r328783: Invent new LDR_INTERP for the loader interpreter to use.
r328806: We need more heap space to properly load newer powerpc kernels.
r328808: Implement strcoll as strcmp.
r328826: Make cross-endian loader changes apply only to powerpc
r328835: Fix regression introduced in r328806, preventing boot on many platforms.
r328911: Ignore relocation tables for non-memory-resident sections.
r328986: Fix relative location of USB sources after recent move.
r328987: A more definitions to kernel emulation shim in order to build stand/usb.
r328990: Move the stand/usb test loader into its own directory.
r328999: Fix indentation to FreeBSD standard for interp files
r329000: Move simple interpreter 'perform' into interp.c and call it
r329019: Move to tabs for indentation and to 8-space notches, per style(9).
r329050: Fix build of userboot.so
r329054: Set script.lang in the environment to either 'forth' or 'simple' to reflect what scripting language was compiled into the loader.
r329060: loader: fix endianness conversion
PR: 225323
|
#
329175 |
|
12-Feb-2018 |
kevans |
MFC Loader Fixes 2017q4p7: r324844,r326089,r326926,r326440,r326484,r326494, r326588,r326708,r326784,r326914,r327390,r328446,r326090,r326143,r326144, r326182,r326384,r326421,r326440,r326441,r326442,r326443,r326444,r326445, r326446,r326447,r326448,r326484,r326485,r326486,r326487,r326488,r326490, r326491,r326492,r326493,r326494,r326495,r326504,r326507,r326509,r326584, r326585,r326586,r326587,r326588,r326589,r326590,r326591,r326592,r326593, r326594,r326600,r326616,r326671,r326707,r326708,r326709,r326710,r326711, r326712,r326714,r326720,r326768,r326772,r326784,r326792,r326812,r326854, r326855,r326856,r326858,r326886,r326887,r326914,r326926,r326927,r326960, r326961,r326962,r326963,r327351,r327453,r327390,r327523,r327524,r326489, r327880,r328437,r328438,r328439,r328441,r328446,r328448,r328449,r328612, r328613,r328615
While here, undo our libfdt hack of not including <stdlib.h> if we're compiling _STANDALONE.
r324844: When building standalone, don't define errno. Let the definition from stand.h override. This is similar to what we do in the kernel.
r326089: loader.efi: efipart does not recognize partitionless disks
r326090: net_parse_rootpath() has no parameters
r326143: Fix theoretical integer overflow issues. If the product here is
r326144: Mark the func pointer as __dead2. It looks up loader_main, which
r326182: Modify all FreeBSD bootloaders on PowerPC AIM (Book-S) systems
r326384: Use const pointers to avoid casting away constness.
r326421: loader.efi: efipart should exclude iPXE stub block protocol
r326440: Remove stale dependency on ufsread.c
r326441: Minor flags cleanup
r326442: Cleanup CFALGS usage here
r326443: We don't need both _STAND and _STANDALONE, use the latter.
r326444: Move geli to common DO32 stuff
r326445: Fix random() and srandom() prototypes to match the standard.
r326446: Undefine _STANDALONE since this is test code.
r326447: Tweaks to the beri boot loader so that it builds w/o warnings.
r326448: Fix all warnings related to geli and ZFS support on x86.
r326484: Const poison the propname.
r326485: Delcare md_load in libofw.h. Make all prototypes match for ofw
r326486: Include machine/md_var to pick up __syncicache prototype.
r326487: Cast mdp (a vm_offset_t) to void * to match prototype.
r326488: e_entry can be smaller than a pointer. Cast it to an intptr_t
r326490: Declare our strange brand of main().
r326491: Disconnet ps3 from the build. There's too many warnings to fix.
r326492: Cast void * pointer to char * so the arg matches the %s format.
r326493: Provide a md_load64 prototype.
r326494: Mark two things as unused (since they are only sometimes used)
r326495: Now it's safe to bump WARNS to 1.
r326504: Switch to proper MK_LOADER_GELI tests.
r326507: increase maximum size of zfsboot
r326509: loader.efi: add note about iPXE into the efipart.c
r326584: When building standalone, include stand.h rather than the kernel
r326585: Include ficl.h before anything else
r326586: No need to include the userland md5.h, the kernel one is just fine.
r326587: Use the kernel relative paths, rather than the userland relative paths
r326588: Need to include skein in the include path
r326589: Make sure we include the right path for skein.h
r326590: Prefer stdint.h to inttypes.h
r326591: This isn't NetBSD specific code. Include these for any kernel /
r326592: Don't inherit CFLAGS. This a specialized test program.
r326593: Stop building with the standard system headers.
r326594: Now that we offer a semi-sane standards-ish set of #include files, stop hacking includes with sed.
r326600: Since this is contrib code, create an upstreamable version of my
r326616: dhcp_try_rfc1048() is not used any more
r326671: Avoid setting -Wno-tentative-definition-incomplete-type with gcc.
r326707: Add partial support signal.h functioanlity. Pull in machine/signal.h
r326708: Remove _KERNEL hack now that errno.h does the right thing when built standalone.
r326709: Provide implementations for iscntrl, ispunct and isgraph.
r326710: Put the files we're copying over into a few variables and clean hings up.
r326711: Const poison a couple of interfaces.
r326712: Create interp class.
r326714: boot1.c needs EFI_ZFS_BOOT too, so add it globally.
r326720: This path belongs in ficl/Makefile, not the common defines for users
r326768: Fix a comment to be more accurate
r326772: Fix regression with lua import
r326784: Revert part of 362772. It was causing problems for includes
r326792: Attempt to unbreak buildworld
r326812: Revert r326792, r326784, r326772, r326712
r326854: libefi: make efichar.h more usable in stand code
r326855: Cargo cut a fix for the regressions r326585 caused.
r326856: Fix comments after bump in size.
r326858: Revert r326855: Cargo cut a fix for the regressions r326585 caused.
r326886: Panic in sbrk if setheap hasn't been called yet. This is preferable o a mysterious crash
r326887: Remove the 'mini libstand in libstand' that util.[ch] provided.
r326914: Move loader-only defines to loader.mk from defs.mk
r326926: Move loader help file definitions to being 100% inside of loader.mk.
r326927: libficl is only ever used in a loader (never a boot) program. Move it.
r326960: Simplify things a little. The RETURN macro isn't required.
r326961: Interact is always called with NULL. Simplify code a little
r326962: Hoist btx include stuff to i386/Makefile.inc
r326963: No need to use relative paths like this here.
r327351: Fix ubldr. uboot/lib uses defines for the loader.
r327453: Add a validbcd() routine that uses the bcd2bin_data[] array
r327390: Garbage-collect loader.ps3. It is currently disconnected from the build and kboot replaces.
r327523: Don't clobber system LDFLAGS for beri boot loaders.
r327524: Use 'extern uint8_t' instead of 'extern void' for external symbols.
r326489: Allow this file to be used in libsa without warning...
r327880: Move getsecs() prototype to stand.h from net.h so it can be used
r328437: Split panic routine
r328438: Implement abort() as a call to panic.
r328439: Provide abs form stdlib.h.
r328441: abort() should be marked __dead2 since it won't return.
r328446: Now that exit is __dead2, we need to tag ub_exit() as __dead2.
r328448: Make exit() never return until host_exit can be written.
r328449: Tag unreachable places as such. I left the while (1); in place
r328612: Move strtold wrapper from strtol.c to its own strtold.c.
r328613: Kill copies of strtol and strtoul.
r328615: Update stand.h for changes for strto*l
PR: 223969
|