bsdinstall: Only show menu if there are more items to be installed

Obtained from: Rubicon Communications, LLC ("Netgate")
Sponsored by: Rubicon Communications, LLC ("Netgate")

(cherry picked from commit 95f0da5be1e3456c930f5f9538cbc099c65f2014)

Git Hash: 27988b8333007d319eb9e9929bda787c5012e59e
Git Author: renato@netgate.com

MFC r366186: bsdconfig, bsdinstall: Prune dead mirrors

Prune dead mirrors from the list of mirrors in bsdconfig and bsdinstall.
All these return NXDOMAIN when trying to resolve them.

Approved by: emaste

MFC r361300: Fix indentation in bsdinstall-created wpa_supplicant.conf

PR: base/221982
Reported by: emaste
Reviewed by: emaste, allanjude
Differential Revision: https://reviews.freebsd.org/D23641

MFC r358419:

bsdinstall: remove the Italian mirror

ftp.it.FreeBSD.org has been down for several years.

PR: 244481
Reported by: xgeoplasma6@gmail.com

MFC r353004, r353012:
r353004:
Explicitly add opensolaris_load="YES" to loader.conf through the
installer when installing the system on a ZFS root filesystem.

For arm64, zfs_load="YES" does not add opensolaris.ko as a kld
dependency, so add it explicitly to prevent boot-time failures
out-of-box.

r353012:
Add a comment explaining why the opensolaris_load line in loader.conf
is explicitly added.

PR: 240478
Approved by: re (kib, early MFC)
Sponsored by: Rubicon Communications, LLC (Netgate)

MFC r320138:

Fixed bsdinstall location of vfs.zfs.min_auto_ashift

Sponsored by: Multiplay

MFC r328839:

Add the ThinkPad X1 (sandybridge) to the bsdinstall blacklist

Reviewed by: AllanJude (for mfc)

MFC r327474:

bsdinstall: inform users that typing will not be visible

Some users, especially those that are new, might be confused when passwd
does not echo anything. Inform users that the password will not be
visible.

PR: 196113

MFC r324206:

bsdinstall(8) hardening menu: Utilize new kern.randompid=1 behaviour

Enabling the PID randomization option in bsdinstall(8)'s hardening menu
now randomizes the effective value of kern.randompid on each boot.

Previous behaviour:
When kern.randompid was enabled via the the bsdinstall(8) hardening menu,
a random value was generated and placed in the systems /etc/sysctl.conf as
kern.randompid=value
This makes the value of kern.randompid static across reboots.

New behaviour:
When kern.randompid is enabled via the bsdinstall(8) hardening menu, the
line kern.randompid=1 is placed in the systems /etc/sysctl.conf.
This takes advantage of a new kernel feature and makes the value of
kern.randompid be randomized by the kernel on each reboot.

MFC r326094: Fix indentation in bsdinstall-created wpa_supplicant.conf

r311690 (MFC of r309934) cleaned up some cases in bsdinstall to use
heredocs but broke the indentation of the generated output, because <<-
heredocs strip leading tabs.

PR: 221982

MFC r321226: bsdinstall: improve checksum mismatch error for snapshots

The usual case of a mismatched checksum for installer snapshots
(e.g., -CURRENT, -ALPHA*) is that a newer snapshot has been built and
the old install sets have been replaced. Provide a specific error
message for checksum mismatches there that suggests looking for a newer
snapshot.

Submitted by: Guangyuan Yang <yzgyyang@outlook.com>
Sponsored by: The FreeBSD Foundation

MFC r323448: bsdinstall: Ignore error return from newaliases(1)

This was originally added as "exit $SUCCESS" but with nothing to set the
SUCCESS variable. Thus it became an exit with no argument, which just
exits with the status of the preceding command.

Sponsored by: The FreeBSD Foundation

MFC r322374: bsdinstall: record DHCP config after obtaining lease

Previously we added an ifconfig_$INTERFACE line to rc.conf for each
unsuccessful DCHP attempt.

PR: 219515
Sponsored by: The FreeBSD Foundation

MFC r321734: bsdinstall: default to UEFI-only boot on arm64

Sponsored by: The FreeBSD Foundation

bsdinstall: use consistent EFI configuration across platforms

MFC r320007:
- increase arm64 EFI partition to 200M, as x86
- use EFI_BOOTPART_SIZE and EFI_BOOTPART_PATH macros on x86
- increase ZFS EFI partition to 200M

MFC r320008: bsdinstall: correct comment after r320007

PR: 201898
Approved by: re (kib)
Relnotes: Yes
Sponsored by: The FreeBSD Foundation

MFC: r319863
bsdinstall: Make ZFS min_auto_ashift adjustment persistent

MFC: r319864
bsdinstall: support Auto ZFS mode for ARM64

Approved by: re (gjb)

MFC r318632, r318633:

bsdinstall: mount is not needed for the ZFS install case
bsdinstall: do not use distextract in scripted mode

MFC r313448:

bsdinstall: Make sure chroot filesystems are umounted after use

* DISTDIR_IS_UNIONFS is set every time BSDINSTALL_DISTDIR is mounted inside
BSDINSTALL_CHROOT. Use this flag to decide if it needs to be umounted
* BSDINSTALL_CHROOT/dev is mounted when 'bsdinstall mount' is called, there is
no need to mount it again when user goes to shell after installation

Reviewed by: allanjude
Obtained from: pfSense
MFC after: 1 week
Sponsored by: Rubicon Communications (Netgate)
Differential Revision: https://reviews.freebsd.org/D8573

MFC improvements to bsdinstall's wlanconfig module

MFC [85] revisions 309719-309720, 309901-309902, 309904-309911,
309913-309920, 309922-309924, 309926, 309928, 309930, 309932, 309934,
309937-309942, 309944-309952, 309958-309998, and 310038 (described below)

r309719: Remove unnecessary trailing backslashes
r309720: Functions in their own section
r309901: Comments
r309902: Use $( ... ) instead of `...`
r309904: Change "[ ! -z ... ]" => "[ ... ]" and "[ -z ... ]" => "[ ! ... ]"
r309905: Remove unnecessary local initializers
r309906: Consolidate locals
r309907: Replace funny block with something easy to digest
r309908: Collapse tiny if statements
r309909: Change "[ ! ... ] && ..." to "[ ... ] || ..."
r309910: Remove unnecessary quotes around number in test
r309911: Group fallbacks together
r309913: Allow $BSDINSTALL_TMPETC to contain whitespace or special chars
r309914: Add missing `-e' parameter to sed invocations
r309915: "echo | sed | sed | awk" is silly (changed to "echo | awk")
r309916: Be internally consistent (": > ..." is used elsewhere in this file)
r309917: awk(1) match() takes a regex, use /.../ to remind ourselves of this
r309918: Remove unnecessary `-n' parameter to head/tail
r309919: Whitespace
r309920: Use provided API instead of hard-coded status integers
r309922: Centralize backtitle string
r309923: There is zero harm in always passing --default-item to dialog
r309924: Always pass --default-item parameter to dialog
r309926: Change the name of a variable from $def_item_... to $default_...
r309928: Use ~ instead of match()
r309930: Use ternary operator
r309932: Remove an unnecessary "return $?" at end of function
r309934: Consolidate redirects into here documents
r309937: Whitespace (dialog options separated to minimize diffs)
r309938: Use provided API (change "dialog" to "$DIALOG")
r309939: Fix incorrect use of provided API
r309940: Reorder dialog parameters based on commonality for readability
r309941: Use provided API to centralize dialog title strings
r309942: Allow the script path to contain whitespace and special characters
r309944: Fix invalid parameter expansion (change $@ to "$@")
r309945: 1 is the default descriptor for redirects without an fd prefix
r309946: Use more succinct awk syntax
r309947: Remove unnecessary semi-colons
r309948: Remove incomplete and unnecessary creation of fd3
r309949: Utilize provided i18n strings
r309950: Whitespace
r309951: Remove an unnecessary call to f_dialog_title_restore()
r309952: Move the secondary condition into the action clause
r309958: Quote WLAN_IFACE (pedantic)
r309959: Use oft-neglected syntax "startcondition, stopcondition { ... }"
r309960: Add missing backslash
r309961: Stop repeating strings (centralize prompt string)
r309962: More efficiently make use of the exit status
r309963: Avoid non-standard options
r309964: Sort the domains
r309965: Whitespace alignment
r309966: Sanitize dialog output for portability/compatibility requirements
r309967: Use more generic f_yesno() from provided API
r309968: Properly quote variable
r309969: Send stderr to the same place as stdout
r309970: Remove completely unnecesary parentheses
r309971: Start deconstructing a conveluted hunk of code
r309972: If the first ping succeeded, why on Earth should we ping it again?
r309973: Why use $? when you can use the command itself
r309974: These two error messages have always been backwards since inception
r309975: Continued resolution of conveluted statement
r309976: You don't need parentheses for awk's printf
r309977: Whitespace and alignment
r309978: Neither printf nor print need parens in awk
r309979: This statement has too many backslashes
r309980: Just use print
r309981: Add missing quotes
r309982: Remove unnecessary quotes
r309983: Use the provided API for calculating the appropriate size of menus
r309984: Whitespace alignment
r309985: Comment
r309986: There's an API function for displaying errors
r309987: There's an API function for displaying yes/no dialogs
r309988: There's an API function for displaying pauses
r309989: There's an API function for catching errors
r309990: Calculate proper size of menu list dialog
r309991: Simplify bringup of interface after changes and catch errors
r309992: Restore previous comment
r309993: Why test $? when you can test the command
r309994: Wordsmithing
r309995: Simplify loop by moving predicate to clause
r309996: Simplify single-line if statements
r309997: The flags of a WLAN need to be quoted
r309998: It's completely pointless to replace newlines with space
r310038: Revert r309918 -- modern POSIX has deprecated -<#>/+<#> syntax

PR: bin/214933

MFC r309716: Add support for "hidden" Wi-Fi networks

PR: bin/214933
Submitted by: Maxim Filimonov <che@bein.link>
Reviewed by: dteske, allanjude, adrian

MFC r303877:

Write kern.randompid to /etc/sysctl.conf

Approved by: allanjude

MFC r303631:

Fix non-functional bsdinstall services dialog.

The most recent version of bsdinstall does not seem to respect any of
the checkboxes in the "Choose the services you would like to be started
at boot" dialog. None of the chosen services end up in the rc.conf file
that is installed onto the target system.

This is caused by the bsdinstall/scripts/hardening script, which
implements the new hardening options dialog. The script starts by
overwriting the previously written rc.conf.services file:

echo -n > $BSDINSTALL_TMPETC/rc.conf.services

which is obviously incorrect. It should clear out rc.conf.hardening
instead.

Approved by: re (kib)
Reviewed by: allanjude
PR: 211506
Differential Revision: https://reviews.freebsd.org/D7387

MFC r302897:

Add new System Hardening menu and options to bsdinstall.

This patch add new 'hardening' file responsible for new bsdinstall
'System Hardening' menu allowing users to set some sane and carefully
picked system security options (like random process id's, hiding
other users/groups processes and others).

All options are OFF by default in this patch due to POLA principle
with intention to turn change some of them to ON by default in future.

Reviewed by: adrian, allanjude, bdrewery, nwhitehorn
Approved by: re@, adrian, allanjude

MFC: r302790, r302795

Add new menu to bsdinstall to allow user to set date and time after
selecting timezone. The 'skip' button is the default selection.

Submitted by: des
Approved by: re (gjb)

MFC: r302940

Fix error in bsdinstall triggered when doing a GELI encrypted install to
an MBR formatted disk

Approved by: re (gjb)
Sponsored by: ScaleEngine Inc.

MFC: r302941

At the start of the installation process, all ZFS pools are exported and
all GELI instances are detached, to allow a restarted install to proceed.

PR: 210814
Approved by: re (gjb)

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Add more machines to the bsdinstall workaround blacklist

Apply the LenovoFix (fake partition in pMBR is 2nd rather than 1st)
and GPT Active fix (mark the fake pMBR partition active) to respective
hardware that is confirmed to require such to boot correct.

Submitted by: Sam Fourman Jr. <sfourman@gmail.com> (ThinkPad W520)
Submitted by: Matthias Apitz <guru@unixarea.de> (Latitude E6330)
Approved by: re (gjb)
Sponsored by: ScaleEngine Inc.

Fix an error in bsdinstall when ZFS installation uses a bootpool

A separate bootpool is used when the selected partition type is MBR
or when the disk is GELI encrypted and a UEFI loader is selected.

BIOS avoids the bootpool by using bootcode and loader that support GELI.

bsdinstall did not remount the bootpool after creating the main pool,
which caused an error when mkdir tried to create /boot/zfs

PR: 210717
Approved by: re (gjb)
Sponsored by: ScaleEngine Inc.

Fix bsdinstall for root-on-zfs with MBR partitioning

Fix an error where vfs.root.mountfrom was not always set as required
when creating a bootpool. After the recent geliboot changes, it was only
set if the main pool was encrypted.

Also resolve an error where the bootpool was unmounted twice causing
bsdinstall to stop with an error message about the failed command.

Approved by: re (gjb)
Sponsored by: BSDCan Hacker Lounge

bsdinstall: add country/regulatory domain configuration dialog.

Allow to setup country and/or regulatory domain before scanning
(since channel list is affected by regulatory restrictions this may
impact scan results).

PR: 182600 (originally)
Differential Revision: https://reviews.freebsd.org/D6325

bsdinstall: unbreak static address assignment for wired networks.

This fixes regression introduced in r298946 (fixes static address
assignment for wireless networks).

Reported & submitted by: allanjude

Only set kern.geom.part.mbr.enforce_chs=0 once, instead of once per disk

Sponsored by: ScaleEngine Inc.

Add support for RAID 1+0 (striped mirrors) to bsdinstall/zfsboot

Sponsored by: ScaleEngine Inc.

Only export and re-import the root pool if installing on MBR

This step is required in order to dd the boot2 bits into the ZFS partition

Sponsored by: ScaleEngine Inc.

Fix missing pool name in zfs set command

Fix error in bsdinstall where additional filesystems cannot be mounted

Do not set canmount=noauto on the boot environment at create time, because
this causes / to not be mounted, and since the chroot is read only, new
mountpoints cannot be created.

The property is set later, when other properties are adjusted

Reported by: HardenedBSD
Sponsored by: ScaleEngine Inc.

bsdinstall/zfsboot GPT+BIOS+GELI installs now make use of GELIBOOT

In this configuration, a separate bootpool is not required.
This allows ZFS Boot Environments to be used with GELI encrypted ZFS pools.

Support for GPT+EFI+GELI is planned for the future.

Tested by: Joseph Mingrone, HardenedBSD
Relnotes: yes
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D5869

Fix default ZFS layout to work better by default with sysutils/beadm

The root file system is mounted early via vfs.root.mountfrom.
The canmount=noauto property only affects the zfs rc.d script.
This ensures that the 'default' BE is not mounted overtop of another BE when
one is selected from the beastie menu

Sponsored by: ScaleEngine Inc.

Fix typo in comments for the zfsboot script

Submitted by: Colby Swandale <colby@taplaboratories.com.au> (via github)

bsdinstall/zfsboot: Do not mirror swap when swapsize is 0

PR: 209415
Submitted by: Ganael LAPLANCHE <ganael.laplanche@corp.ovh.com>
MFC after: 2 weeks

bsdinstall: fix static address assignment for protected wireless networks.

Filter out IFCONFIG_PREFIX variable (which may contain WPA keyword)
from ifconfig parameters.

PR: 169199

usr.sbin/bsdconfig: minor spelling fixes on comments.

No functional change.

Add a new installation type to bsdinstall/zfsboot: BIOS+UEFI

Installs both pmbr+gptzfsboot as well as boot1.efifat in separate partitions
The resulting system can be booted with either UEFI or BIOS/CSM.
Preference is controlled by the user's firmware boot settings.

This is now the default for zfsboot installs

PR: 208629
Submitted by: Galael LAPLANCHE <ganael.laplanche@corp.ovh.com> (original version)

bsdinstall/zfsboot should create /tmp and /var/tmp if they do not exist

PR: 208629
PR: 208568
Submitted by: Galael LAPLANCHE <ganael.laplanche@corp.ovh.com>
MFC after: 2 weeks

Do not add swap to fstab when swapsize is 0

If a user requested encrypted swap, but 0 sized, it would still be added to fstab

PR: 208630
Submitted by: Ganael LAPLANCHE <ganael.laplanche@corp.ovh.com>
MFC after: 2 weeks
Sponsored by: ScaleEngine Inc.

Remove 50% ZFS conditional from bsdinstall/zfsboot

Remove the requirement that the resulting partition table be atleast 50% ZFS

PR: 208094
Requested by: brooks
Sponsored by: ScaleEngine Inc.

The zfsboot automated part of bsdinstall now supports UEFI

MFC after: 3 days
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D4960

Never 4k align the MBR bootpool because zfsldr can not deal with a gap

If the bootpool does not start at the first sector of the BSD partition
then zfsldr seeks to the wrong offset inside the ZFS vdev label, and is
unable to find zfsboot, so the system does not boot

If 4k alignment is requested, align the BSD partition in the MBR table,
and align the swap and data pool, but the bootpool must start at sector 1

While here, if 4k alignment is requested, disable MBR CHS alignment, as
this results in not-4k aligned partitions.

Reported by: Alex Wilkinson
MFC after: 5 days
Sponsored by: ScaleEngine Inc.

bsdinstall: Suggest the GPT+Active workaround on Dell T5810

The Dell Precision Tower 5810 fails to boot from GPT in Legacy/BIOS mode
without the Active flag in the Protective MBR. Suggest the workaround
during installation.

Since an increasing number of Dell systems exhibit this behavior,
I imagine all Dells past a certain date will do so. I would like
to suggest the workaround for all Dells with a BIOS date of, say,
2014 or later, but I would need to test a variety of systems before
committing such a change.

Reviewed by: allanjude, dteske
MFC after: 5 days
Relnotes: We should probably suggest using GPT+Active on "recent" Dells.
Sponsored by: Dell Inc.
Differential Revision: https://reviews.freebsd.org/D4075

Merge ^/projects/release-install-debug:

- Rework MANIFEST generation and parsing via bsdinstall(8).
- Allow selecting debugging distribution sets during install.
- Rework bsdinstall(8) to fetch remote debug distribution sets
when they are not available on the local install medium.
- Allow selecting additional non-GENERIC kernels during install.
At present, GENERIC is still required, and installed by default.

Tested with: head@r293203
Sponsored by: The FreeBSD Foundation

Explicitly set permissions on entropy files

Differential Revision: https://reviews.freebsd.org/D3933
Submitted by: jmg
Reviewed by: delphij, markm
Approved by: secteam (delphij)
MFC after: 3 days
X-MFC-to: stable/10 stable/9

Adapt to new wireless scheme where base wlan interfaces do not show up in
ifconfig anymore.

Use LIBEXECDIR for /usr/libexec.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Add a number of models to the bsdinstall GPT hack blacklist

PR: 194359
Approved by: bapt (mentor)
MFC after: 2 weeks
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D3525

Replace `return' outside of function with exit

MFC after: 3 days
X-MFC-to: stable/10
Pointy hat to: me

Use a variable for readability

MFC after: 3 days
X-MFC-to: stable/10

Update copyright

MFC after: 3 days
X-MFC-to: stable/10

Better to reset trap and explicitly exit success

MFC after: 3 days
X-MFC-to: stable/10

Remove trailing newline at EOF

MFC after: 3 days
X-MFC-to: stable/10

Produce meaningful exit code

MFC after: 3 days
X-MFC-to: stable/10

Explicitly exit with success

MFC after: 3 days
X-MFC-to: stable/10

Explicitly exit with success

MFC after: 3 days
X-MFC-to: stable/10

Update copyrights

MFC after: 3 days
X-MFC-to: stable/10

Produce meaningful exit code

MFC after: 3 days
X-MFC-to: stable/10

Remove use of return outside of function

MFC after: 3 days
X-MFC-to: stable/10

Ignore error results from newaliases(1)

MFC after: 3 days
X-MFC-to: stable/10

Ensure the local MANIFEST is always used when verifying remote
distribution sets.

MFC after: immediately
Sponsored by: The FreeBSD Foundation

Add the Dell E7240 laptop and Intel DP965LT motherboard to the list for the GPT active workaround

PR: 194359
Requested by: sbruno, hiren
Approved by: marcel
MFC after: 3 days
Relnotes: yes
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D3139

Add support for two workarounds for known issues booting GPT in legacy mode on some hardware

For Lenovo laptops with buggy bios (x220, t420, t520):
Write the 0xee entry into the second slot in the pmbr instead of the first

For some Dell and HP models:
The BIOS gives a warning message when booting in legacy mode from a GPT partitioned disk where the 0xee partition in the pmbr is not flagged active
For models known to have this problem, mark the pmbr active during installation

Use smbios data to identify machines known to be affected by any of the above, and offer the user the option to apply the workaround

In bsdinstall's ufs auto mode (autopart partition wizard):
Allow users to select which type of partition table to use
Keep current defaults: MBR for BIOS, GPT for UEFI
This allows users to choose GPT for legacy boot if they wish

PR: 184910
PR: 194359
Reviewed by: Michael Dexter
Approved by: marcel
MFC after: 3 days
X-MFC-With: r285594
Relnotes: yes
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D3091

Make bsdinstall's zfsboot script align partitions to 4k/1m when the user requests it

PR: 195174
Reviewed by: darius
Approved by: brueffer
MFC after: 3 days
Relnotes: yes
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D3062

Set a mountpoint on the root of the pool so user-created datasets have a mountpoint to inherit

Reviewed by: darius
Approved by: brueffer
MFC after: 3 days
Relnotes: yes
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D3063

Have bsdinstall's zfsboot script make /var/audit its own dataset, so it is not part of the OS boot environment

PR: 199864
Submitted by: Jason Unovitch
Approved by: brueffer
MFC after: 3 days
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D2861

Fix a typo in bsdinstall

A variable was misspelled resulting in chmod executing on the installer instead of on the target chroot

PR: 191402
Submitted by: Martin Simmons <martin@lispworks.com>
Approved by: brueffer
MFC after: 3 days
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D3061

Add META_MODE support.

Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision: D2796
Reviewed by: brooks imp

Fix a syntax error in bsdinstall/zfsboot

Differential Revision: https://reviews.freebsd.org/D2571
Submitted by: Michael Dexter
Approved by: dteske
MFC after: 1 week
X-MFC-With: r283023, r282443
Sponsored by: ScaleEngine Inc.

bsdinstall/zfsboot: Skip adding swap lines to /etc/fstab if swap is 0 sized

Differential Revision: https://reviews.freebsd.org/D2571
Reviewed by: dim
Approved by: eadler (mentor)
Sponsored by: ScaleEngine Inc.

Add a sanity check to the swap size in zfsboot of bsdinstall
Loop until the user enters a valid size (>100mb or 0)

Differential Revision: https://reviews.freebsd.org/D2299
Reported By: Shawn Webb
Reviewed by: roberto
Approved by: brd
MFC after: 2 weeks
Sponsored by: ScaleEngine Inc.

Add "GELI Passphrase:" prompt to boot loader.

A new loader.conf(5) option of geom_eli_passphrase_prompt="YES" will now
allow you to enter your geli(8) root-mount credentials prior to invoking
the kernel.

See check-password.4th(8) for details.

Differential Revision: https://reviews.freebsd.org/D2105
Reviewed by: imp, kmoore
Discussed on: -current
MFC after: 3 days
X-MFC-to: stable/10
Relnotes: yes

Unbreak ZFS+GELI install option.

Reported by: delphij
Submitted by: delphij
Pointy hat to: dteske (me)
MFC after: 3 days
X-MFC-to: stable/10 stable/9
X-MFC-with: r281160

Fix permissions on ZFS root encryption key (644 -> 600).

MFC after: 3 days
X-MFC-to: stable/10 stable/9
Security: CVE-2015-1415
Reported by: Pierre Kim

Whoops! "arc commit --revision" != "arc diff --update"

Add "GELI Passphrase:" prompt to boot loader.

Summary:
Add "GELI Passphrase:" prompt to boot loader.

A new loader.conf(5) option of geom_eli_passphrase_prompt="YES" will now
allow you to enter your geli(8) root-mount credentials prior to invoking
the kernel.

See check-password.4th(8) for details.

Differential Revision: https://reviews.freebsd.org/D2105
Reviewed by: (your name[s] here)
MFC after: 3 days
X-MFC-to: stable/10
Relnotes: yes

Test Plan:
Drop a head copy of check-password.4th into /boot and then apply the patch
(only the patch to /boot/check-password.4th is required; no other changes are
required but you do have to have a HEAD copy of check-password.4th to
apply the patch).

NB: The rest of your /boot files can be up to 2 years old but no older.
NB: The test won't work unless your kernel has the following change

https://svnweb.freebsd.org/base?view=revision&revision=273489

Now, put into /boot/loader.conf:

geom_eli_passphrase_prompt="YES"

and reboot.

You should be prompted for a GELI passphrase before the menu (if enabled),
just after loading loader.conf(5).

NB: It doesn't matter if you're using GELI or not. However if you are using
GELI and a sufficiently new enough release (has SVN r273489) and you entered
the proper passphrase to mount your GELI encrypted root device(s), you should
notice that the boot process did not stop (you went from loader all the way to login).

Reviewers: cperciva, allanjude, scottl, kmoore

Subscribers: jkh, imp

Differential Revision: https://reviews.freebsd.org/D2105

Update copyright(s)

MFC after: 3 days
X-MFC-to: stable/10

Whitespace cleanup(s).

MFC after: 3 days
X-MFC-to: stable/10

Remove a non-required unsafe condition added in the previous commit

Reviewed by: bapt
Approved by: dteske
Sponsored by: ScaleEngine Inc.

Fix the handbook install option in bsdinstall

bsdconfig's f_package_add doesn't seem to support using the pkg repo from /etc/pkg/FreeBSD.conf, it also tries to run the commands on the installer image, not in the destination chroot

Instead, manually bootstrap pkg in the chroot, and then install the requested packages (in the chroot)

Doesn't use pkg -c, because pkg is not installed on the installer image

PR: 196250
Differential Revision: https://reviews.freebsd.org/D2026
Approved by: bapt
Sponsored by: ScaleEngine Inc.

Fix bsdinstall when working with geli boot disks

PR: 196790
Differential Revision: https://reviews.freebsd.org/D566
Submitted by: Michael Gmelin <grembo@FreeBSD.org>
MFC after: 2 weeks
Sponsored by: Multiplay

Switch zfsboot installer force 4K option to use vfs.zfs.min_auto_ashift=12

Previously we used gnop(8) to fake 4K sector size disks but ZFS now has a
sysctl to control this when creating new top level vdev's so use that
instead.

Differential Revision: D566
MFC after: 1 month
Sponsored by: Multiplay

Change the /var dataset in the default ZFS layout to have the
ZFS property canmount=off so that /var/db/pkg and other such directories
are part of the / dataset, and only /var/mail, /var/log, and /var/crash
are excluded from the ZFS boot environment (beadm).

PR: 193971
Approved by: jmg
MFC after: ASAP
Relnotes: yes
Sponsored by: ScaleEngine Inc.

Add a line to the post-installation configuration dialog to enable the
local_unbound service.

MFC after: 3 days

ZFS support isn't actually experimental anymore, so no need to scare
people.

Make the root-on-zfs part of the installer warn a user who booted the installer via UEFI that we do not support booting ZFS via UEFI yet

PR: 193595
Approved by: nwhitehorn
MFC after: 5 days
Sponsored by: ScaleEngine Inc.
CR: https://reviews.freebsd.org/D782

Rename the choices in the partitioning methods dialog to reflect current
reality. In particular, draw a connection between the auto ZFS script and
the auto UFS one, since they fulfill similar functions. I'm not sure the
auto ZFS code is actually experimental anymore, so it might be worth
changing that label still.

Make the default choice for the chroot shell at the end be "No". This allows
just pressing enter repeatedly to successfully install a reasonable system.

Add ZFS support to the bsdinstall partition editor and sade.

Submitted by: Kurt Lidl (original version)
MFC after: 6 weeks

Give a brief message as to what error was encountered to help the user along.

Reviewed by: nwhitehorn
MFC after: 2 weeks

Strip the patch level from the release name before trying to fetch the
distribution.

PR: 170264
Approved by: nathanw
MFC after: 1 week

Fix syntax error thrown at the point of creating the root pool, caused by
an embedded newline appearing within the options string surrounded by
double-quotes. Rework the logic that goes into setting dataset options on
the root pool dataset while we're here -- added two new variables (which
can be altered via scripting) ZFSBOOT_POOL_CREATE_OPTIONS and also
ZFSBOOT_BOOT_POOL_CREATE_OPTIONS for setting pool/dataset attributes at
the time of pool creation. The former is for setting options on the root
pool (zroot) and the latter is for setting options on the optional separate
boot pool (bootpool) implicitly enabled when using either GELI or MBR. The
default value for the root pool variable (ZFSBOOT_POOL_CREATE_OPTIONS) is
"-O compress=lz4 -O atime=off" and the default value for separate boot pool
variable (ZFSBOOT_BOOT_POOL_CREATE_OPTIONS) is NULL (no additional options
for the separate boot pool dataset).

Reviewed by: allanjude
MFC after: 7 days
X-MFC-with: r266107-266109

Fix the "disks" variable reuse.

It starts off being used to track the grammar for the number of disks
(singular vs plural) and then it is reused as the list of available disks.

Replace the variable with disks_grammar and move 'disk' and 'disks' to
msg_ vars so they can be translated in the future.

Submitted by: Allan Jude <freebsd@allanjude.com>
Reviewed by: roberto
MFC after: 2 weeks
Sponsored by: ScaleEngine Inc.

Updates to the datasets created by zfsboot.

Set compress=lz4 for the entire pool, removing it from the individual
datasets

Remove exec=no from /usr/src, breaks the test suite.

Submitted by: Allan Jude <freebsd@allanjude.com>
Reviewed by: roberto
MFC after: 2 weeks
Sponsored by: ScaleEngine Inc.

Here is a patch for the bsdinstall root-on-zfs stuff that adds optional
encryption for swap, and optional gmirror for swap (which can be combined)

Submitted by: Allan Jude <freebsd@allanjude.com>
Requested By: roberto
Sponsored By: ScaleEngine Inc.
MFC after: 2 weeks

Update zfsboot to coincide with r264840 to bsdconfig(8) adding
GEOM support (thereby adding GEOM support to the disk selection
menu of bsdinstall(8)'s `zfsboot' module updated herein).

MFC after: 1 week
X-MFC-with: 264840

When merging docsinstall and zfsboot updates to stable/9 it was discovered
that the slightly older dialog(1) requires --separate-output when using the
--checklist widget to force response to produce unquoted values (whereas in
stable/10 --checklist widget without --separate-output will only quote the
checklist labels in the response if the label is multi-word (contains any
whitespace).

Since these enhancements (see revisions 263956 and 264437) were developed
originally on 10, the --separate-output option was omitted. When merged to
stable/9, we (Allan Jude) and I found during testing that the "always-
quoting" of the response was causing things like struct interpolation to
fail (`f_struct device_$dev' would produce `f_struct device_\"da0\"' for
example -- literal quotes inherited from dialog(1) --checklist response).

To see the behavior, execute the following on stable/9 versus stable/10:

dialog --checklist disks: 0 0 0 da0 "" off da1 "" off

Check both items and hit enter. On stable/10, the response is:

da0 da1

On stable/9 the response is:

"da0" "da1"

If you use the --separate-output option, the response is the same for both:

da0
da1

So applying --separate-output on every platform until either one of two
things occurs 1) dialog(1,3) gets synchronized between stable/9, higher or
2) we drop support for stable/9.

MFC after: 3 days
Reviewed by: Allan Jude

Fix typo in debug/log statement.

Submitted by: Rick Miller <vmiller@hostileadmin.com>

NO_MAN= has been deprecated in favor of MAN= for some time, go ahead
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.

Revert r263638 (create /root dataset) for now as it breaks install and needs
more work in cases such as single-user mode.

Create a root dataset so that /root is outside of the boot environment.

Discussed with: dteske on sysinstall@
MFC after: 1 month

Add zfsboot to the scripted interface of bsdinstall(8); oops!
NB: If the zfsboot variables ($ZFSBOOT_*) are set, a script is
assumed to want zfsboot module instead of scriptedpart module.

Submitted by: Loïc Brarda <loic.brarda@cern.ch>
Reviewed by: nwhitehorn@
MFC after: 3 days

Dummy commit (whitespace changes and style nits) to show previous commit
(SVN r260866) was [in-part] Submitted-by: Christoph Mallon ...
<christoph.mallon@gmx.de>

MFC After: 3 days
X-MFC-With: r260866

Fix a bad comparison operator (s/==/=/), and address a use-case issue where-
in the one-line comment associated with the dumpdev setting was not present
for the case where the user deselects the dumpdev service (restoring pre-
r256348 behaviour.

MFC After: 3 days

Allow bsdinstall to use WPA-Enterprise networks when installing. This
only allows basic username/password config, and does not provide the
ability to set any of the other WPA options. Regardless, this is
generally sufficient to associate.

Perhaps in the future this could allow full configuring (e.g. being able
to set "anonymous identity", and perhaps some of the more obscure WPA
options), though perhaps that will better belong in bsdconfig when that
grows wlan config ability.

MFC after: 1 week

Lower the priority of the "connect to any open wireless network" network
description, so guaranteeing that any other defined network will be tried
first.

MFC after: 1 week

Remove what appears to be a stray debugging printf.

MFC after: 1 week

Move the installer's keymap solution introduced by SVN r259468. The new
location of /etc/rc.local on the install media is more appropriate as it
knows serial vs. non-serial and can also do the change earlier (so that
even the initial Install dialog can benefit from the change).

MFC after: 3 days

When using a bootpool, set the cachefile property so that it gets
imported automatically upon reboot to the newly installed system.

MFC after: 1 day

Fix a regression that occurred with SVN revisions 259474 and 259476 where
installation to 3-4+ (depending on vdev type) vdevs would result in odd
error messages where the zpool `create' command appeared to repeat itself
(an artifact of printf when you supply too many arguments -- caused by
neglecting to properly quote the multi-word expansion of $*vdevs when
creating the pool(s)). Example error below (taken from bsdinstall_log):

DEBUG: zfs_create_boot: Creating root pool...
DEBUG: zfs_create_boot: zpool create -o altroot=/mnt -m none -f "zroot" \
ada0p3.nop ada1p3.nopzpool create ada2p3.nop "ada3p3.nop"
DEBUG: zfs_create_boot: retval=1 <output below>
cannot open 'ada1p3.nopzpool': no such GEOM provider

In bsdinstall_log, it's rather confusing to see the following sequence:
DEBUG: Running installation step: hostname
rm: /tmp/bsdinstall_etc/fstab: No such file or directory

The two lines are unrelated, and the rm is spurious. Let's add `-f' to
that rm(1) so it doesn't confuse us when debugging an install.

MFC after: 3 days

Fix "[: -eq: argument expected" error introduced by SVN r256343. The code
should not have used DIALOG_CANCEL because dialog.subr wasn't included to
define it. The effect of the error was that you could not cancel the
partition dialog. Discovered by checking bsdinstall_log for something else.

MFC after: 3 days

Auto-enable 4k sector alignment when disk encryption is requested (it is
required in such a case). But don't prevent the user from pointing the
gun at his/her foot -- you can disable 4k alignment after enabling geli).

MFC after: 3 days

Fix a long-standing edge-case that would result in a ghosted `zroot' pool
and subsequent headaches caused by multiple pools with the same name.
Specifically, blast away any labels on the designated swap partition.
Problem was when you install to a given layout *with* swap and then turn
around and re-install the same layout *without* swap (we weren't doing a
labelclear for the swap device, so would end up with an "UNAVAIL" status
zroot pool that may only exist in the pool cache).

MFC after: 3 days

Add kern.geom.label.disk_ident.enable="0" to loader.conf(5).

Discussed on: -current, -stable
MFC after: 3 days

De-uglify the geli(8)-setup infobox by adding a newline.

MFC after: 3 days

fletcher4 is currently the default.

Discussed on: -current
MFC after: 3 days

Improve default ZFS disk layout (tested):
+ For GPT, always provision zfs# partition after swap [for resizability]
+ For MBR, always use a boot pool to relialy place root vdevs at EOD
NB: Fixes edge-cases where MBR combination failed boot (e.g. swap-less)
+ Generalize boot pool logic so it can be used for any scheme (namely MBR)
+ Update existing comments and some whitespace fixes
+ Change some variable names to make reading/debugging the code easier
in zfs_create_boot() (namely prepend zroot_ or bootpool_ to property)
+ Because zroot vdevs are at EOD, no longer need to calculate partsize
(vdev consumes remaining space after allocating swap)
+ Optimize processing of disks -- no reason to loop over the disks 3-4
separate times when we can logically use a single loop to do everything

Discussed on: -stable
MFC after: 3 days

Bug-fixes and debugging improvments:
+ De-obfuscate debugging to show actual values
+ Change graid(8) syntax; s/destroy/delete/ [destroy is not invalid syntax]
+ Log commands that were previously quiet
+ Added some new comemnts and updated some existing ones
+ Add missing local for `disk' used in zfs_create_boot()
+ Use $disks instead of multiply-expanding $* in zfs_create_boot()
+ Pedantically unset variable holding geli(8) passphrase after use
+ Pedantically add double-quotes around zpool names and zfs datasets
+ Fix quotation expansion for zpool_cache entries of loader.conf(5)
+ Some limited whitespace changes

MFC after: 3 days

Accept NULL input as also meaning zero swap.

MFC after: 3 days

As per discussions on -current, re-add /var/mail (removed in r257842) but
this time with atime=on in support of various software that requires it.

Discussed on: -current
MFC after: 3 days

Mask error from newaliases(1) when the hostname is not fully qualified.

MFC after: 3 days

Add a fix for Long-standing problem with VMware. Described in below links:
https://communities.vmware.com/thread/107230
https://communities.vmware.com/docs/DOC-11677

Basically, ignore the ``function 62'' and ``function 63'' interpretations
of the left/right command key when we're in the lengthiest portion of the
installation (initiated by the `auto' module).

The net effect is that you can now (once you've started the installer from
the media) escape the VM without prematurely terminating the current action
due to spurious escape sequence.

MFC after: 3 days

I caught the following snippet at the end of my /var/log/bsdinstall_log:
===
DEBUG: Running installation step: services
local: Not in a function
/usr/libexec/bsdinstall/services: cannot create : Read-only file system
/usr/libexec/bsdinstall/services: /tmp/bsdinstall/etc/rc.conf.services: \
Permission denied
===
The `local: Not in a function' is obvious, and was introduced by myself in
SVN revision 256348.

The latter two are caused by the attempt to use "\" to continue the line
after using the ">>" redirect. This appears to attempt to write a file with
the name " " in the current directory and subsequently attempts to execute
the file that was originally intended for writing (which is not executable;
hence the `Permission denied'). That was introduced in SVN r228192 about
2 years ago, apparently unnoticed until I started going over the debug
outputs very carefully.

MFC after: 3 days

Fix a regression introduced by SVN r257842; resulting in mountroot prompt
after attempting to install to encrypted ZFS root (caused by a typo in a
variable name -- ZFSBOOT_BOOT_FSNAME -> ZFSBOOT_BOOTFS_NAME).

MFC after: 3 days

Fix a regression introduced by SVN r257842. Result was that after
successfully installing to encrypted ZFS root, the passphrase is
not accepted and a message about "incorrect key" is displayed.

MFC after: 3 days

sed(1): Use /i instead of /I

the i modifier was added in r259132 since POSIX recently agreed to add
it. Switch uses of /I to /i.

Fix a regression introduced by SVN r257842 that prevents Encryption from
being enabled.

MFC after: 3 days

Switch default MBR bootcode from /boot/boot0 to /boot/mbr.

Reviewed by: Allan Jude, nwhitehorn
Discussed on: -current

Check the partition scheme before blowing away disks, instead of after.
The effects of this patch would only be noticeable if you were purposefully
setting a bad value and trying to see what happens; and leaving the disks
intact if a bad value has been set seems fair.

I often forget that debugFile can begin with a + for deciding that debug
should be sent to both the file (minus the leading +) and stdout.

Copy the install log to the deployed /var/log directory so that we have
post-mortem data on how the machine was configured/installed.

Semi-revert this one file out of r257842. The ttys(5) stuff, as we
discussed on -current, was not supposed to go in but get solved in
a different way -- e.g., init(8) logic).

Discussed on: -current

A series of changes tested together as a whole:
+ Add a `-D FILE" command-line option for overriding the path to the
bsdinstall log file (BSDINSTALL_LOG env var).
+ Document new `-D FILE' in the man page for bsdinstall.
+ If FILE in `-D FILE' begins with a +, debug output goes to stdout
(interleaved between dialog(1) invocations/output) as well as to FILE
(minus the leading + of course).
+ If BSDINSTALL_LOG cannot be written, then debugging is disabled (except in
the case of a leading + in the pathname, wherein debug will still be
printed to stdout).
+ Update source code formatting style.
+ Fix a dangling participle ("Begun ..." -> "Began ...")
+ Rewrite the docsinstall script (was necessary to abate direct dependency
on BSDINSTALL_LOG (instead, use fault-tolerant bsdconfig framework which
displays appropriate errors for package management).
NB: docsinstall is still using pkg(8) after this change.
+ Add additional debug output for dhclient/rtsol/wpa_cliscan
+ Display script errors in a textbox rather than just on stdout
+ Update many coments.
+ Add new f_show_err() API call (like f_show_msg but changes the dialog
title to "Error")(see bsdconfig's `common.subr').
+ Add new f_eval_catch() API call for executing a command via eval but not
before logging the command to debug. Several example cases documented in
API header for function in bsdconfig's `common.subr'.
+ Fix dialog auto-sizing when launched as an rvalue to a pipe for indirected
scripts (previously would default to 24x80 sizing in this case, now it can
autosize to full size even when in a pipe chain).
+ Fix bug in f_snprintf if $format argument began with "-"; printf would
misinterpret as a flag. (this is in bsdcofig's `strings.subr').
+ Add accompanying f_sprintf() and f_vsprintf() to go along with already
existing f_snprintf() and f_vsnprintf() (see bsdconfig's `strings.subr').
+ Remove some unnecessary default ZFS datasets from the automatic "zfsboot"
script. Such as: /usr/ports/distfiles /usr/ports/packages /usr/obj /var/db
/var/empty /var/mail and /var/run (these can all be created as-needed once
the system is installed).
+ Remove setuid=off for /usr/home (as discussed from last round of CFT).
+ Fix some i18n string violations in "zfsboot".
+ Bolster debugging output in "zfsboot".
+ Fix some string quoting issues in "zfsboot".
+ Fix some variable scope issues in "zfsboot".
+ Change "Create" to "Install" in "zfsboot" main menu.
+ Increase error checking in "zfsboot" (type-check arguments and such).
+ Add call to "graid destroy" killing automatic metadata (part of the series
of pedantic destructions we do when bootstrapping a new/naked disk).
+ Make judicious use of new f_eval_catch() in "zfsboot".
+ Fixup some variable names for consistency (zfsboot).
+ Fix an underride syntax parameter expansion folly (zfsboot).
+ Confirm layout if not explicitly chosen when blindly proceeding (no
longer have to touch anything on the ZFS menu if it scares you, just
choose the omnibus "Install" option at the top and you'll be prompted to
select vdev type and disks in the layout confirmation dialog).
+ Change numbered menu items to alphabetic for more efficient navigation.
+ Consolidate vdev selection and disk selection into a single stateful
menu which performs validation and allows backing out to each previous
menu as you go deeper.
+ Redesign the ``Last Chance'' dialog (still using the same colors, but
make it conform to a tolerable width and make disks appear in a block-
quote style indented region).
+ Fix a bug wherein we used the a lowercase variable name by accident
(actual variable name declared as all-uppercase) at the time of
initializing fstab(5) (not believed to cause any issues though).
+ Update the geli setup infobox for each provider being initialized
(not just at the onset -- since each ``geli init'' causes kernel messages
to push our infobox off-screen).

Reviewed by: Allan Jude <freebsd@allanjude.com>
Discussed on: -current
MFC after: 3 days

Switch to using pkg(8) for the doc install in bsdinstall(8).
This also bootstraps the system with pkg(8) for future use.

PR: 183488
Submitted by: Joe Ennis
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Make ZFSBOOT_BEROOT_NAME be ROOT by default. This is what sysutils/beadm
and OpenSolaris/Illumos beadm use.

Remove /usr/local and /var/db/pkg datasets. Andriy Gapon writes:
I want to note that a good implementation of BEs should support a dependent
datasets feature. Unfortunately, it seems that we do not have any good BE
implementation for FreeBSD right now. If we had, personally I'd prefer to
have /usr/local in separate filesystem.

NOTE: Until then, remove these datasets.

Discussed on: src-committers
Submitted by: Bryan Drewery <bryan@shatow.net>
Reviewed by: Allan Jude <freebsd@allanjude.com>
MFC after: 3 days

Fix SVN r256540: s/TMPETC/TMPBOOT/ for loader.conf(5) GELI entries.

gjb writes: when doing encrypted zfs install, loader.conf should be updated
so bootpool is imported after boot:
zpool_cache_load="YES"
zpool_cache_type="/boot/zfs/zpool.cache"
zpool_cache_name="/boot/zfs/zpool.cache"
otherwise /boot is a broken symlink after the system is up.
http://lists.freebsd.org/pipermail/freebsd-fs/2013-July/017891.html
NOTE: Fix a comment while here.

Reviewed by: gjb

Add executable bit to docsinstall [old] and entropy [new] scripts.

Approved by: re (gjb)

Remove the dumpdev configuration dialog, merge it into the regular
services configuration and enable it by default. Committed with
slight change to menu text for length and content.

Submitted by: Allan Jude <freebsd@allanjude.com>
Approved by: re (glebius)

Incorporate PR bin/161547 to detect when an interface is wireless.
Patch from PR modified slightly for whitespace and style.

PR: bin/161547
Submitted by: Warren Block <wblock@wonkity.com>
Reviewed by: Allan Jude <freebsd@allanjude.com>
Approved by: re (glebius)

Rewrite the keymap module to display an actual menu of selectable
keymaps *and* provide a mechanism for testing the selection. With
this commit, bsdinstall is no longer dependent on kbdmap(1). The
keymap test menu was originally submitted by Warren Block but was
modified).

Submitted by: Warren Block <wblock@wonkity.com>
Reviewed by: Allan Jude <freebsd@allanjude.com>
Approved by: re (glebius)

Add zfsboot module as an option for automatic configuration. Default is
to run interactively but it can be scripted too (optinally completely
non-interactive). Currently supports GELI and all ZFS vdev types. Also
performs validation on selections/settings providing error messages if
necessary, explaining (in plain language) what the issue is. Currently
the auto partitioning of naked disks only supports GPT and MBR (VTOC8
pending for sparc64), so is only available for i386/amd64 install.

Submitted by: Allan Jude <freebsd@allanjude.com>, myself
Reviewed by: Allan Jude <freebsd@allanjude.com>
Approved by: re (glebius)

After installation, dump 4096 bytes from /dev/random to /entropy in the
newly installed system. This should greatly increase the amount of
entropy available when SSH host keys are generated during first boot.

Reviewed by: markm, nathanw
Approved by: re (gjb)

Remove ftp5.se.f.o, as per request to -hubs@

Approved by: re (glebius)
MFC after: 3 days

- Remove the is (Iceland) mirror per mail from the admins.

Approved by: re
With hat: clusteradm@

Make sure bootonly.iso for -BETAs and -RCs use the releases/
directory on the FTP mirrors to fetch distributions, since
these are always pushed to releases/ during the release cycle.

MFC after: 3 days
X-MFC-To: stable/9, releng/9.2

Remove ftp2.tr.freebsd.org, it is out of date.

MFC after: 3 days

Update mirror list to more closely match
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/mirrors-ftp.html

MFC after: 1 week

If no resolv.conf has been made for the new system already and the install
media has one already, copy it in lieu of leaving things blank. This
reduces the foot-shooting potential for PXE installs that immediately
add packages.

Properly quote argument.

MFC after: 3 days

Remove some debugging code that snuck in.

Add a scripting frontend. Documentation and release(7) support coming later.

Scripts take the form:

------------------------------------
PARTITIONS=ada0
DISTRIBUTIONS="kernel.txz base.txz ports.txz"

#!/bin/sh
echo "sshd_enable=YES" >> /etc/rc.conf
pkg add puppet
echo "System setup complete"
------------------------------------

The second part of the script (beginning with #!) is run in the newly
installed system after the installation onto ada0 (with default partitioning,
see scriptedpart for more complicated scenarios) is complete.

Note that cpufreq(4) is not available on all hardware.

PR: bin/175139
MFC after: 2 weeks

Add missed BETA* to list.

Approved by: simon (implicit)
MFC after: 3 days
X-MFC-With: r244058

Get 'uname -r' earlier, so it can be used to determine what branch is
being run to set BSDINSTALL_DISTSITE accordingly. This change allows
non-RELEASE branches to use the FTP snapshots directory for bootonly.iso
installations.

Approved by: simon

Remove snapshots.jp.freebsd.org. It stopped working years ago.

Discussed with: hrs

- Remove snapshots.se.freebsd.org [1]
- Add ftp6.se.freebsd.org

Discussed with: brd [1]

Fix bug introduced by r241902 (MANIFEST uses TAB delimiter).

PR: bin/173140
Approved by: adrian (co-mentor)

Optimize syntax to use builtins and reduce unnecessary forking where possible.

Reviewed by: nwhitehorn
Approved by: adrian (co-mentor)

Add the IPv6 tag to ftp4.fr.freebsd.org.

Make the chroot shell more functional by providing devfs.

Reported by: Robert Simmons
MFC after: 4 days

Fix a whitespace nit

Reviewed by: bz

Prevent user astonishment by providing the shell option at the end, after
any installer-provided configuration files have been copied. This allows
users to edit their fstab, if desired, and to see what the installer has
placed in rc.conf.

Requested by: phk
MFC after: 1 day

Add a screen that asks if the user would like to enable crash dumps,
giving them a very brief description of the trade-offs. Whether the
user opts in or out add an entry to what will become /etc/rc.conf
explaining what dumpdev is and how to turn on/off crash dumps. The folks
who handle interacting with users submitting PRs have asked for this.

Reviewed by: nwhitehorn
MFC after: 1 day

Provide an error message and error handling if there are no network
interfaces in the system. This is a non-fatal error except when doing a
network installation.

PR: bin/161950
MFC after: 3 days

Escape the newline so we get a proper line continuation. Without this
the text of the menu selections doesn't get displayed properly and it
makes the installer appear to lock up for no obvious reason.

Reviewed by: nwhitehorn

Move "Exit" to the first entry in the list, so that it is the default
choice.

PR: bin/160913
MFC after: 3 days

Ask if you want to set the keymap before invoking kdbmap to prevent
confusion.

PR: bin/160913
MFC after: 3 days

Inquire about terminal type if starting from a serial console and do
not configure the host's networking if netbooted [1]. Also fix FTP
installations behind some firewalls [2].

PR: bin/159583 [2]
Reported by: stas [1]
Approved by: re (kib)

Fix WPA setup broken by changes to shell argument passing inside the
installer network configuration.

PR: bin/159558
Approved by: re (kib)

Allow WPA PSKs up to 63 characters, the max length according to
wpa_supplicant.conf(5). Up to now, the PSK was truncated
to 15 characters (the length of the input field).

PR: 159559
Submitted by: Gavin Chappell <gavin.chappell@gmail.com>
Approved by: re (kib)

Add ftp4.se.freebsd.org to the IPv4 and IPv6 mirror lists.

PR: 157210, 157211
Submitted by: Niclas Zeising <zeising@lysator.liu.se>
Approved by: re (kib)

Correct a typo.

Submitted by: gcooper
Reviewed by: nwhitehorn
Approved by: re (kib)

Fix two problems:
1) if no default gateway is given the route command will fail
and the script will abort with an error leading to an endless
loop unless configuration is cancelled or changed. [1]
2) In the IPv6 case add -inet6 to not alter the legacy IP default
route.

Reported by: Oleg Ginzburg (olevole olevole.ru) [1]
Tested by: Oleg Ginzburg (olevole olevole.ru) [1]
Approved by: re (kib)

Swap FTP paths to be MACHINE/MACHINE_ARCH instead of MACHINE_ARCH/MACHINE.
This follows what the output of make targets looks like, and reduces the
number of directories needed on the FTP server, since we now have only one
architecture with multiple machine types (i386, which has i386 and pc98),
but 3 machines with 9 architectures between them (powerpc, mips, and arm).

Requested by: kensmith, bz
Approved by: re(kensmith)

Make messages about typos appear on the screen instead of in the log file.

Reported by: lev
Approved by: re (kib)

Fix a bug that prevented docsinstall from being able to use DNS in most
cases and provide a better error handling mechanism during package
installation.

Approved by: re (kib)

Add back ftp4, a new machine has taken its place. ftp2 and ftp8 are both IPv6 capable
mirrors. cvsup4/ftp4 have already been added to the handbook.

Reviewed by: nathanw
Approved by: re (kib)

To inet or not two inet. Unbreak static IPv4 configuration. If you tried
you were stuck in an endless loop.

Reported by: kensmith
Approved by: re (kib) for BETA1
Pointy hat to: bz - wonder why I had not tested legacy IP thoroughly..

Run newaliases at the end of the install process.

Submitted by: kevlo

Per request of the docs team, install docs as packages, instead of via
the normal distfile mechanism. Thanks to Marc Fonvieille for the patch and
for putting up with me taking entirely too long to commit this!

Submitted by: blackend

Remove TODO which is not longer needed and the default.

Fix resolv.conf search list creation:
1) do not print out an empty "search ", things do not like it.
2) the search list is not comma separated.

Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Write the multi step netconfig to a temporary file and only move that
to the final name if netconfig was completely finished. This fixes
reentrance problems even better than r222611.

Suggested by: nwhitehorn
Reviewed by: nwhitehorn
Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Empty the network configuration only after the user decided to pick an
interface. Otherwise an accidental start of the netowrk configuration
and immediate cancel after the install has finished removes the previously
configured settings.

Discussed with: nwhitehorn
Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Fix opening a shell on the new system (prevent the shell's stderr from
ending up in the install log).

Contrary to the rc.conf framework, when manualy enabling IPv6 we have
to -ifdiabled ourselves.

Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

While doing it right for current configuration, fix the entry for rc.conf
adding the missing mandatory "inet6" keyword.

Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Split netconfig into three parts:
- netconfig - what auto will call which in turn will check for
IPv4 and IPv6 to be available and ask the user to configure it
by calling
- netconfig_ipv4 doing DHCP and static IPv4 addresses, and
- netconfig_ipv6 doing rtsol and static IPv6 addresses,
and then checking, querying and updating resolv.conf upon return.
Both DHCP and rtsol (in the future) might update resolv.conf already so
we seed ourselves from that file if available.

Reviewed by: nwhitehorn
Sponsored by: The FreeBSD Foundation
Sponsored by: iXsystems

Fix shell-based partitioning.

Fix mirror selection, which was broken by stderr redirection.

Reported by: Garrett Cooper

Update to reflect net install changes.

Make the keymap script continue to work now that stderr is being
redirected to the log file.

Rename the "Reboot" option to "Exit", which reflects what it actually
does. The rc.local wrapper script will also soon grow an option to use the
live environment after installation, so an immediate reboot isn't even
necessarily implied.

Check correctly for whether there are any available wireless networks. This
was tested on a train in the middle of an upstate New York swamp.

Committing while tired is never a good idea. Remove an unconditional error
exit left over from debugging.

Improve error handling.

Add mirror selection for FTP installs. To support TBEMD installs
(e.g. powerpc64), this looks in a slightly different default path than now,
specifying both $MACHINE and $MACHINE_ARCH:
MIRROR/pub/FreeBSD/releases/`uname -p`/`uname -m`/`uname -r`

How to handle TBEMD properly probably merits some more discussion, but,
since no such 9.0-CURRENT distfiles exist at the moment, all existing
mirrors presently fail anyway.

Mark any distfiles with no checksum entries in the manifest "Skipped"
instead of "Passed".

Add generation of an installation manifest containing SHA256 checksums as
well as package descriptions and add code in the installer to check the
checksums.

Correct typo.

Make installations work when no optional components are selected.

Add support for optional distributions, installation of the ports tree and
src. Fitting all of this on a single CD along with a full system requires
changing the default compression algorithm from gzip to xz.

Import bsdinstall. This is meant to be (eventually in conjunction with
pc-sysinstall) a replacement for sysinstall in the 9.0 release and beyond.
Currently supported platforms are sparc64, pc98, i386, amd64, powerpc, and
powerpc64. Integration into the build system will occur in the coming
weeks.

Merging with pc-sysinstall will use this code as a frontend, while
temporarily retaining the interactive partition editor here. This work
will be done in parallel with improvements on this code and release
integration.

Thanks to all who have provided testing and comments!