MFC r358418:

bsdconfig: remove the Italian mirror

ftp.it.FreeBSD.org has been down for several years.

PR: 244481
Reported by: xgeoplasma6@gmail.com

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Add bsdconfig `wifi' (aka `wireless' or `wlan') module

Approved by: re (gjb)
MFC after: 8 weeks
Relnotes: yes

Use dpv(1) in `bsdconfig packages'

META MODE: Update dependencies with 'the-lot' and add missing directories.

This is not properly respecting WITHOUT or ARCH dependencies in target/.
Doing so requires a massive effort to rework targets/ to do so. A
better approach will be to either include the SUBDIR Makefiles directly
and map to DIRDEPS or just dynamically lookup the SUBDIR. These lose
the benefit of having a userland/lib, userland/libexec, etc, though and
results in a massive package. The current implementation of targets/ is
very unmaintainable.

Currently rescue/rescue and sys/modules are still not connected.

Sponsored by: EMC / Isilon Storage Division

META MODE: These need object directories to handle staging.

Sponsored by: EMC / Isilon Storage Division

Step 1 of eliminating the "games" distribution: Move binaries to /usr/bin;
update paths; and include everything in the "base" distribution.

The "games" distribution being optional made sense when there were more
games and we had small disks; but the "games-like" games were moved into
the ports tree a dozen years ago and the remaining "utility-like" games
occupy less than 0.001% of my laptop's small hard drive. Meanwhile every
new user is confronted by the question "do you want games installed" when
they they try to install FreeBSD.

The next steps will be:

2. Removing punch card (bcd, ppt), phase-of-moon (pom), clock (grdc), and
caesar cipher (caesar, rot13) utilities. I intend to keep fortune, factor,
morse, number, primes, and random, since there is evidence that those are
still being used.

3. Merging src/games into src/usr.bin.

This change will not be MFCed.

Reviewed by: jmg
Discussed at: EuroBSDCon
Approved by: gjb (release-affecting changes)

Follow-up to r255036; remove beforeinstall directives from bsdconfig(8)
Makefile's, fixing concurrent installworld (`make -j17 installworld').

Thanks to: delphij, emaste
Reviewed by: delphij
MFC after: 3 days
X-MFC-to: stable/10, stable/9

Add example script `add_some_packages.sh', demonstrating how to install
a list of packages using the bsdconfig(8) API, providing dialog(8) based
user experience (versus plain console output were you to use pkg(8)
directly to install the same list of packages).

Remove example script `browse_packages_ftp.sh', made obsolete because
the digests.txz and packagesite.txz databases for pkg(8) are not
available via FTP (HTTP only to pkg.freebsd.org SRV hosts).

Update example script `browse_packages_http.sh', made to work with new
pkg(8) demonstrating how to generate a local package repository.

Fix a bug in `bsdconfig packages' where packages were listed twice. This
fix requires pkg(8) version 1.2.7_4 or higher.

NB: It is the introduction of pkg(8) 1.2.7_4 wherein I am also able to
drastically reduce the generation time of package dependencies prior to
the dialog display (by utilizing the new `-I' flag to pkg-rquery(8)).

While here, fix a positional argument nit for f_index_initialize() of
`packages/index.subr' include (the one and only argument is positional
argument 1 to state the by-ref handle, indicating the variable to set
in the caller's namespace; the nit I'm fixing here is that we were
querying positional argument 2 for this information incorrectly; caused
by a missing hunk back around SVN r257795).

Fix a bug in sysrc(8) described by PR bin/187458 "sysrc(8) silently and
unexpectedly bootstraps pkg". This was caused by an explicit entry in
`/usr/share/bsdconfig/common.subr' (used by sysrc(8)) that called pkg(8)
to populate the $PKG_ABI global (called in a way that allows implicit
bootstrap of pkg(8)). The solution to which was to find every place in
bsdconfig(8) that requires the $PKG_ABI global and add a layer of
protection by way of introducing the new API call f_musthavepkg_init()
(provided by new include, `/usr/share/bsdconfig/packages/musthavepkg.subr'
intented to mirror `/usr/share/bsdconfig/mustberoot.subr'). When the
$PKG_ABI global is required, you can use `f_musthavepkg_init' to cause
graceful premature termination in the event that pkg(8) has not yet been
bootstrapped, and thus cannot be used to populate $PKG_ABI.

NB: If running interactively ($nonInteractive is NULL or unset), the
f_musthavepkg_init() API call will attempt to bootstrap pkg(8), but only
if the user chooses "Yes" to a Yes/No dialog confirming it is ok to
bootstrap.

While here, simplify an if-conditional in `media/http.subr' include to
use sh(1) inline assignment-with-break.

Also, fix a bug in `media/http.subr' and `media/httpproxy.subr' wherein
the error messages for various HTTP failures were not finding their way
to the console (needed to pass stdout to pass-thru descriptor).

While here, remove the executable bit from `packages/categories.subr',
`packages/index.subr', and `packages/packages.subr' includes.

Fix issues where pkg(8) complains about PACKAGESITE being defined.
Previously, we would set $PACKAGESITE and export it. Now we only set
$PACKAGESITE for invocations of "pkg update" -- getting rid of all the
spurious warnings about PACKAGESITE being deprecated (it's still used
in the case of "pkg update" for simplicity versus having to configure
a config-file).

Remove the leading argument from invocations of f_index_initialize() in
`packages/packages.subr' include. The leading argument no longer means
what it used to, pre SVN r257995).

PR: bin/187458
Reviewed by: nwhitehorn
MFC after: 1 week
X-MFC-to: stable/10, stable/9

Sort messages.

Fix cosmetic typos (since the typo's in the variable names were copied
and pasted, it wasn't noticed).

Replace pkg-tools with pkgng.

Add an f_show_err() function -- acts pretty much the same as f_show_msg()
except it stylizes the msgbox widget to indicate an Error has occurred by
setting an appropriate title and default message.

Add keymap.subr for presenting custom keymap dialogs (with X11 support).

Approved by: re (glebius)

Remove ftp2.tr.freebsd.org, it is out of date.

MFC after: 3 days

Sync-up with bsdinstall(8) FTP mirrorselect changes.

SVN r224656:
Add back ftp4, a new machine has taken its place. ftp2 and ftp8 are both
IPv6 capable mirrors. cvsup4/ftp4 have already been added to the handbook.

NOTE: Except put IPv6 entries in IPv6 section at the top of the list

SVN r225467:
Add ftp4.se.freebsd.org to the IPv4 and IPv6 mirror lists.

SVN r235228:
Add the IPv6 tag to ftp4.fr.freebsd.org.

NOTE: Except dup entry into IPv6 section.

SVN r243832:
- Remove snapshots.se.freebsd.org [1]
- Add ftp6.se.freebsd.org

SVN r243854:
Remove snapshots.jp.freebsd.org. It stopped working years ago.

SVN r253543:
Update mirror list to more closely match
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/mirrors-ftp.html

MFC after: 1 week

Add the necessary code to uninstall packages (re-install still pending).
Both scripted access (packageDelete) and UI access have been tested
successfully with a variation of different situations including:
+ Uninstall a package which no other installed package depends
+ Uninstall multiple packages which no other installed packages depend
+ Uninstall multiple packages which depend on each other
+ Similar to above but when ordered removal requires tracing dependencies
+ Purposefully do things like uninstall a package that is not installed
+ Try to uninstall a package which other installed packages still depend
+ Try to uninstall multiple packages which other installed packages depend
+ And many more.

Add the necessary code to install packages (uninstall and re-install still
pending). Both scripted access (packageAdd) and UI access have been tested
successfully with a variation of different situations including:
+ Install a package with no dependencies
+ Install a package with dependencies that are already installed
+ Install a package with dependencies where some are already installed
+ Repeat each of the above from FTP and local Directory
+ Purposefully do things like try to install a package that does not exist
+ Try to install a package for which a dependency could not be loaded
+ Try to install a package that is in the INDEX bot not on the media
+ And many more.

Add a newline character to the end of the "Check URL again" error message
because long URLs do not induce extra height despite wrapping by dialog(1).
NOTE: For even longer lines, the cursor up/down keys work to scroll through

Merge r248313 from stable/9 sysinstall(8) to head bsdconfig(8):
Add support for installation directly via HTTP.

While we're here, remove the menu-item for Passive FTP (since moving to
ftp(1) and switching FTPMODE to `auto' by default -- see r251613 -- the
single remaining FTP menu-item works for both ftp.f.o and ftp-archive.f.o;
previously each requiring separately active versus passive both work with
the `auto' setting). In scripting you still have mediaSetFTPActive and
mediaSetFTPPassive but the remaining FTP menu-item uses mediaSetFTP which
defaults to `auto' (aforementioned SVN r251613).

Fix i18n violations in the package management module. A few words like
`packages', `installed', and `selected' were not internationalized.

Remove duplicated string.

Commit first portion of package module -- this includes the ability to view
categories, view packages, mark packages for installation, de-installation,
or re-installation, calculate and track dependencies, as well as ability to
review selections.

Still to come is the actual processing of selections (performing the
various actions associated with the user's selections, such as installing
dependencies first, then selections, etc.).

Alphabetize strings by variable names.

Fix copy/paste error.

Import media selection/preparation framework (sysinstall inspired). Makes
accessing files from various types of media nice and abstracted away from
the wet-work involved in preparing, validating, and initializing those
types of media. This will be used for the package management system module
and other modules that need access to files and want to allow the user to
decide where those files come from (either in a scripted fashion, prompted
fashion, or any combination thereof).

Heavily inspired by sysinstall and even uses the same reserved words so
that scripts are portable. Coded over months, tested continuously through-
out, and reviewed several times.

Some notes about the changes:
- Move network-setting acquisition/validation routines to media/tcpip.subr
- The options screen from sysinstall has been converted to a dialog menu
- The "UFS" media choice is renamed to "Directory" to reflect how sysinstall
treats the choice and a new [true] "UFS" media choice has been added that
acts on real UFS partitions (such as external disks with disklabels).
- Many more help files have been resurrected from sysinstall (I noticed that
some of the content seems a bit dated; I gave them a once-over but they
could really use an update).
- A total of 10 media choices are presented (via mediaGetType) including:
CD/DVD, FTP, FTP Passive, HTTP Proxy, Directory, NFS, DOS, UFS, Floppy, USB
- Novel struct/device management layer for managing the issue of passing
more information than can comfortably fit in an argument list.

Add support for scripting (sysinstall style).

Reviewed by: jilles

Resurrect usage.hlp and securelevel.hlp from sysinstall(8) and integrate them
into bsdconfig(8).

Approved by: adrian (co-mentor) (implicit)

More-accurately replicate the help system from sysinstall(8). However, also
improve upon the system by giving the user a "Help" button instead of requiring
the user to press F1.

NOTE: In FreeBSD-9 and higher, dialog(1) does not support the F1 hook, so the
mechanism for providing help to the user had to be changed to a button.

This now means we can resurrect *.hlp files from usr.sbin/sysinstall/help/ as-
is and reusing them as-needed in bsdconfig (holding to the goal of losing as
little functionality from sysinstall as possible).

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Ask to become root via sudo(8) and give user ability to save this preference.

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Spelling and whitespace corrections.

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Move major includes into /usr/share/bsdconfig for easy external access.

Reviewed by: adrian (co-mentor)
Approved by: adrian (co-mentor)

Import bsdconfig(8) as a replacement for the post-install abilities of
deprecated sysinstall(8). NOTE: WITH_BSDCONFIG is currently required.

Submitted by: Devin Teske (dteske), Ron McDowell <rcm@fuzzwad.org>
Reviewed by: Ron McDowell <rcm@fuzzwad.org>
Approved by: Ed Maste (emaste)