bsdinstall: Fix installation script splitting

This allows writing setup scripts that contain lines starting with
"#!", e.g., a shebang when creating a shell script using cat:

#!/bin/sh
echo "Populate rc.local"
cat >/etc/rc.local<<EOF
#!/bin/sh
echo booted | logger -s -t 'example'
EOF

Prevent accidentally running a setup script left behind by a
previous invocation of bsdinstall.

Reviewed by: imp, jrtc27
Differential Revision: https://reviews.freebsd.org/D43350

bsdinstall: avoid conflicts with fd 3

Throughout the bsdinstall script fd 3 is used by f_dprintf (set through
$TERMINAL_STDOUT_PASSTHRU). In several places in the bsdinstalls scripts,
we use fd 3 to juggle stdout when calling out to other tools, which can
cause the installer to fail with a "Bad file descriptor" error when
f_dprintf attempts to use it.

This commit replaces all constructs like this:

exec 3>&1
SOME_VARIABLE=$(some command 2>&1 1>&3)
exec 3>&-

With:

exec 5>&1
SOME_VARIABLE=$(some command 2>&1 1>&5)
exec 5>&-

PR: 273148
Reviewed by: corvink
Fixes: 1f7746d81f53447ac15cc99395bb714d4dd0a4da ("bsdinstall: stop messing with file descriptors")
MFC after: 1 week

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

bsdinstall/script: umount before zpool export

When running zpool export first, boot/efi and dev is still mounted so
zpool export fails. By running bsdinstall umount first the pool can be
cleanly exported.

Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D35114
Sponsored by: Beckhoff Automation GmbH & Co. KG
MFC After: 3 days

bsdinstall: stop messing with file descriptors

Throughout the bsdinstall script fd 3 is used by f_dprintf (set through
$TERMINAL_STDOUT_PASSTHRU). By closing file descriptor 3 here, the
final f_dprintf "Installation Completed ... does not work anymore.

By putting the code into a subshell, file descriptors can be edited
without interference with the calling script.

Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D35113
Sponsored by: Beckhoff Automation GmbH & Co. KG
MFC after: 3 days

bsdinstall: Avoid double-mounting /dev

After 34766aa8cb514472c571f8b0e90e833833acef51 we are mounting and
unmounting devfs elsewhere already.

Reviewed by: nwhitehorn
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D30877

bsdinstall: Fix typo (Instalation -> Installation).

Fix scripted installation from media without local distfiles.

The bsdinstall script target did not have the infrastructure to fetch
distfiles from a remote server the way the interactive installer does
on e.g. bootonly media. Solve this by factoring out the parts of the
installer that deal with fetching missing distributions into a new
install stage called 'fetchmissingdists', which is called by both the
interactive and scripted installer frontends.

In the course of these changes, cleaned up a few other issues with
the fetching of missing distribution files and added a warning if
fetching the MANIFEST file, which is used to verify the integrity of
the distribution files. We should at some point add cryptographic
signatures to MANIFEST so that it can be fetched safely if not present
on the install media (which it is for bootonly media).

Initial patch by: Vinícius Zavam
PR: 255659, 250928
Reviewed by: dteske
MFC after: 4 weeks
Differential Revision: https://reviews.freebsd.org/D27121

Fix scripted installs on EFI systems using ZFS root with zfsboot.

Unlike attended installations, scripted installs did not mount non-ZFS
partitions when ZFS root (via zfsboot) was selected. Since this included
the ESP, the EFI loader was not installed. Copy logic from the
attended-install path to make this work.

PR: 255824, 255081
MFC after: 1 week
Obtained from: Mark Huizer

Fix scripted installs on EFI systems after default mounting of the ESP.

Because the ESP mount point (/boot/efi) is in mtree, tar will attempt to
extract a directory at that point post-mount when the system is installed.
Normally, this is fine, since tar can happily set whatever properties it
wants. For FAT32 file systems, however, like the ESP, tar will attempt to
set mtime on the root directory, which FAT does not support, and tar will
interpret this as a fatal error, breaking the install (see
https://github.com/libarchive/libarchive/issues/1516). This issue would
also break scripted installs on bare-metal POWER8, POWER9, and PS3
systems, as well as some ARM systems.

This patch solves the problem in two ways:
- If stdout is a TTY, use the distextract stage instead of tar, as in
interactive installs. distextract solves this problem internally and
provides a nicer UI to boot, but requires a TTY.
- If stdout is not a TTY, use tar but, as a stopgap for 13.0, exclude
boot/efi from tarball extraction and then add it by hand. This is a
hack, and better solutions (as in the libarchive ticket above) will
obsolete it, but it solves the most common case, leaving only
unattended TTY-less installs on a few tier-2 platforms broken.

In addition, fix a bug with fstab generation uncovered once the tar issue
is fixed that umount(8) can depend on the ordering of lines in fstab in a
way that mount(8) does not. The partition editor now writes out fstab in
mount order, making sure umount (run at the end of scripted, but not
interactive, installs) succeeds.

PR: 254395
Reviewed by: gjb, imp
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D29380

bsdinstall: Use TMPDIR if set

Submitted by: Ryan Moeller <ryan@freqlabs.com>
Reviewed by: bcran, Nick Wolff <darkfiberiru@gmail.com>
Differential Revision: https://reviews.freebsd.org/D22979/

Add installer support for PS3 and PowerNV systems, also laying the
foundation for invoking efibootmgr as part of new-style EFI booting on
x86. On PS3 and PowerNV, which are booted using Linux kexec from petitboot
rather than by loader(8), install the kernel and the rest of /boot to a
FAT partition and set up the appropriate petitboot configuration file
there.

The new bootconfig installer stage can do platform-dependent modifications
more complex than partition layout and installation of boot blocks and can
be used to (as here) set up some special configuration files, run efibootmgr,
or boot0cfg.

MFC after: 1 month

bsdinstall: do not use distextract in scripted mode

It requires a tty, which might not be available in scripted installs. Instead
extract the sets manually using tar.

Reviewed by: tsoome
Sponsored by: Citrix Systems R&D
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D10736

bsdinstall: mount is not needed for the ZFS install case

Because the datasets are already mounted by zfsboot, and the mount script
doesn't know anything about ZFS. Also do not execute the "umount" script for
ZFS for the same reasons.

Reviewed by: dteske, tsoome
Sponsored by: Citrix Systems R&D
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D10738

Update copyright

MFC after: 3 days
X-MFC-to: stable/10

Better to reset trap and explicitly exit success

MFC after: 3 days
X-MFC-to: stable/10

Fix typo in debug/log statement.

Submitted by: Rick Miller <vmiller@hostileadmin.com>

Add zfsboot to the scripted interface of bsdinstall(8); oops!
NB: If the zfsboot variables ($ZFSBOOT_*) are set, a script is
assumed to want zfsboot module instead of scriptedpart module.

Submitted by: Loïc Brarda <loic.brarda@cern.ch>
Reviewed by: nwhitehorn@
MFC after: 3 days

A series of changes tested together as a whole:
+ Add a `-D FILE" command-line option for overriding the path to the
bsdinstall log file (BSDINSTALL_LOG env var).
+ Document new `-D FILE' in the man page for bsdinstall.
+ If FILE in `-D FILE' begins with a +, debug output goes to stdout
(interleaved between dialog(1) invocations/output) as well as to FILE
(minus the leading + of course).
+ If BSDINSTALL_LOG cannot be written, then debugging is disabled (except in
the case of a leading + in the pathname, wherein debug will still be
printed to stdout).
+ Update source code formatting style.
+ Fix a dangling participle ("Begun ..." -> "Began ...")
+ Rewrite the docsinstall script (was necessary to abate direct dependency
on BSDINSTALL_LOG (instead, use fault-tolerant bsdconfig framework which
displays appropriate errors for package management).
NB: docsinstall is still using pkg(8) after this change.
+ Add additional debug output for dhclient/rtsol/wpa_cliscan
+ Display script errors in a textbox rather than just on stdout
+ Update many coments.
+ Add new f_show_err() API call (like f_show_msg but changes the dialog
title to "Error")(see bsdconfig's `common.subr').
+ Add new f_eval_catch() API call for executing a command via eval but not
before logging the command to debug. Several example cases documented in
API header for function in bsdconfig's `common.subr'.
+ Fix dialog auto-sizing when launched as an rvalue to a pipe for indirected
scripts (previously would default to 24x80 sizing in this case, now it can
autosize to full size even when in a pipe chain).
+ Fix bug in f_snprintf if $format argument began with "-"; printf would
misinterpret as a flag. (this is in bsdcofig's `strings.subr').
+ Add accompanying f_sprintf() and f_vsprintf() to go along with already
existing f_snprintf() and f_vsnprintf() (see bsdconfig's `strings.subr').
+ Remove some unnecessary default ZFS datasets from the automatic "zfsboot"
script. Such as: /usr/ports/distfiles /usr/ports/packages /usr/obj /var/db
/var/empty /var/mail and /var/run (these can all be created as-needed once
the system is installed).
+ Remove setuid=off for /usr/home (as discussed from last round of CFT).
+ Fix some i18n string violations in "zfsboot".
+ Bolster debugging output in "zfsboot".
+ Fix some string quoting issues in "zfsboot".
+ Fix some variable scope issues in "zfsboot".
+ Change "Create" to "Install" in "zfsboot" main menu.
+ Increase error checking in "zfsboot" (type-check arguments and such).
+ Add call to "graid destroy" killing automatic metadata (part of the series
of pedantic destructions we do when bootstrapping a new/naked disk).
+ Make judicious use of new f_eval_catch() in "zfsboot".
+ Fixup some variable names for consistency (zfsboot).
+ Fix an underride syntax parameter expansion folly (zfsboot).
+ Confirm layout if not explicitly chosen when blindly proceeding (no
longer have to touch anything on the ZFS menu if it scares you, just
choose the omnibus "Install" option at the top and you'll be prompted to
select vdev type and disks in the layout confirmation dialog).
+ Change numbered menu items to alphabetic for more efficient navigation.
+ Consolidate vdev selection and disk selection into a single stateful
menu which performs validation and allows backing out to each previous
menu as you go deeper.
+ Redesign the ``Last Chance'' dialog (still using the same colors, but
make it conform to a tolerable width and make disks appear in a block-
quote style indented region).
+ Fix a bug wherein we used the a lowercase variable name by accident
(actual variable name declared as all-uppercase) at the time of
initializing fstab(5) (not believed to cause any issues though).
+ Update the geli setup infobox for each provider being initialized
(not just at the onset -- since each ``geli init'' causes kernel messages
to push our infobox off-screen).

Reviewed by: Allan Jude <freebsd@allanjude.com>
Discussed on: -current
MFC after: 3 days

After installation, dump 4096 bytes from /dev/random to /entropy in the
newly installed system. This should greatly increase the amount of
entropy available when SSH host keys are generated during first boot.

Reviewed by: markm, nathanw
Approved by: re (gjb)

If no resolv.conf has been made for the new system already and the install
media has one already, copy it in lieu of leaving things blank. This
reduces the foot-shooting potential for PXE installs that immediately
add packages.

Remove some debugging code that snuck in.

Add a scripting frontend. Documentation and release(7) support coming later.

Scripts take the form:

------------------------------------
PARTITIONS=ada0
DISTRIBUTIONS="kernel.txz base.txz ports.txz"

#!/bin/sh
echo "sshd_enable=YES" >> /etc/rc.conf
pkg add puppet
echo "System setup complete"
------------------------------------

The second part of the script (beginning with #!) is run in the newly
installed system after the installation onto ada0 (with default partitioning,
see scriptedpart for more complicated scenarios) is complete.