bhyve: Move lock of uart frontend to uart backend

Currently, lock of uart in bhyve is placed in frontend. There are some
problems about it:

1. If every frontend should has a lock, why not move it inside backend
as they all have same uart_softc.
2. If backend needs to modify the information of uart after initialize,
it will be impossible as backend cannot use lock. For example, if we
want implement a telnet support for uart in backend, It should wait
for connection when initialize. After some remote process connect it,
it needs to modify rfd and wfd in backend.

So I decide to move it to backend.

Reviewed by: corvink, jhb, markj
Differential Revision: https://reviews.freebsd.org/D44947

bhyve: Move device model-independent UART code into a separate file

Currently bhyve implements a ns16550-compatible UART in uart_emul.c.
This file also contains generic code to manage RX FIFOs and to handle
reading from and writing to a TTY. bhyve instantiates UARTs to
implement COM devices (via pci_lpc.c) and PCI UART devices.

The arm64 port will bring with it a PL011 device model which is used as
the default console (i.e., no COM ports). To simplify its integration,
add a UART "backend" layer which lets UART device models allocate an RX
FIFO and interact with TTYs without duplicating code. In particular,
code in uart_backend.* is to be shared among device models, and the
namespace for uart_emul.* is changed to uart_ns16550_*.

This is based on andrew@'s work in
https://github.com/zxombie/freebsd/tree/bhyvearm64 but I've made a
number of changes, particularly with respect to naming and source code
organization.

No functional change intended.

Reviewed by: corvink, jhb
MFC after: 1 week
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D40993

usr.sbin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

bhyve: default UART to 115200

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D41494

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

bhyve: Address some signed/unsigned comparison warnings

MFC after: 1 week

bhyve: clean up trailing whitespaces

Clean up trailing whitespaces. No functional changes.

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D33681

Refactor configuration management in bhyve.

Replace the existing ad-hoc configuration via various global variables
with a small database of key-value pairs. The database supports
heirarchical keys using a MIB-like syntax to name the path to a given
key. Values are always stored as strings. The API used to manage
configuation values does include wrappers to handling boolean values.
Other values use non-string types require parsing by consumers.

The configuration values are stored in a tree using nvlists. Leaf
nodes hold string values. Configuration values are permitted to
reference other configuration values using '%(name)'. This permits
constructing template configurations.

All existing command line arguments now set configuration values. For
devices, the "-s" option parses its option argument to generate a list
of key-value pairs for the given device.

A new '-o' command line option permits setting an individual
configuration variable. The key name is always given as a full path
of dot-separated components.

A new '-k' command line option parses a simple configuration file.
This configuration file holds a flat list of 'key=value' lines where
the 'key' is the full path of a configuration variable. Lines
starting with a '#' are comments.

In general, bhyve starts by parsing command line options in sequence
and applying those settings to configuration values. Once this is
complete, bhyve then begins initializing its state based on the
configuration values. This means that subsequent configuration
options or files may override or supplement previously given settings.

A special 'config.dump' configuration value can be set to true to help
debug configuration issues. When this value is set, bhyve will print
out the configuration variables as a flat list of 'key=value' lines.

Most command line argments map to a single configuration variable,
e.g. '-w' sets the 'x86.strictmsr' value to false. A few command
line arguments have less obvious effects:

- Multiple '-p' options append their values (as a comma-seperated
list) to "vcpu.N.cpuset" values (where N is a decimal vcpu number).

- For '-s' options, a pci.<bus>.<slot>.<function> node is created.
The first argument to '-s' (the device type) is used as the value of
a "device" variable. Additional comma-separated arguments are then
parsed into 'key=value' pairs and used to set additional variables
under the device node. A PCI device emulation driver can provide
its own hook to override the parsing of the additonal '-s' arguments
after the device type.

After the configuration phase as completed, the init_pci hook
then walks the "pci.<bus>.<slot>.<func>" nodes. It uses the
"device" value to find the device model to use. The device
model's init routine is passed a reference to its nvlist node
in the configuration tree which it can query for specific
variables.

The result is that a lot of the string parsing is removed from
the device models and centralized. In addition, adding a new
variable just requires teaching the model to look for the new
variable.

- For '-l' options, a similar model is used where the string is
parsed into values that are later read during initialization.
One key note here is that the serial ports use the commonly
used lowercase names from existing documentation and examples
(e.g. "lpc.com1") instead of the uppercase names previously
used internally in bhyve.

Reviewed by: grehan
MFC after: 3 months
Differential Revision: https://reviews.freebsd.org/D26035

Support COM3 and COM4 serial ports.

Submitted by: Jan Poctavek <janci@binaryparadise.com>, otis
Reviewed by: grehan (bhyve), imp, 0mp (manpages)
Differential Revision: https://reviews.freebsd.org/D28207

Initial support for bhyve save and restore.

Save and restore (also known as suspend and resume) permits a snapshot
to be taken of a guest's state that can later be resumed. In the
current implementation, bhyve(8) creates a UNIX domain socket that is
used by bhyvectl(8) to send a request to save a snapshot (and
optionally exit after the snapshot has been taken). A snapshot
currently consists of two files: the first holds a copy of guest RAM,
and the second file holds other guest state such as vCPU register
values and device model state.

To resume a guest, bhyve(8) must be started with a matching pair of
command line arguments to instantiate the same set of device models as
well as a pointer to the saved snapshot.

While the current implementation is useful for several uses cases, it
has a few limitations. The file format for saving the guest state is
tied to the ABI of internal bhyve structures and is not
self-describing (in that it does not communicate the set of device
models present in the system). In addition, the state saved for some
device models closely matches the internal data structures which might
prove a challenge for compatibility of snapshot files across a range
of bhyve versions. The file format also does not currently support
versioning of individual chunks of state. As a result, the current
file format is not a fixed binary format and future revisions to save
and restore will break binary compatiblity of snapshot files. The
goal is to move to a more flexible format that adds versioning,
etc. and at that point to commit to providing a reasonable level of
compatibility. As a result, the current implementation is not enabled
by default. It can be enabled via the WITH_BHYVE_SNAPSHOT=yes option
for userland builds, and the kernel option BHYVE_SHAPSHOT.

Submitted by: Mihai Tiganus, Flavius Anton, Darius Mihai
Submitted by: Elena Mihailescu, Mihai Carabas, Sergiu Weisz
Relnotes: yes
Sponsored by: University Politehnica of Bucharest
Sponsored by: Matthew Grooms (student scholarships)
Sponsored by: iXsystems
Differential Revision: https://reviews.freebsd.org/D19495

bhyve: add wrapper for debug printf statements

Add printf() wrapper to use CR/CRLF terminators depending on whether
stdio is mapped to a tty open in raw mode.
Try to use the wrapper everywhere.
For now we leave the custom DPRINTF/WPRINTF defined by device
models, but we may remove them in the future.

Reviewed by: grehan, jhb
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22657

usr.sbin/bhyve: don't leak a FD if the device is not a tty

Coverity CID: 1194193
Approved by: markj, jhb
Differential Revision: https://reviews.freebsd.org/D20934

Use separate descriptors in bhyve's stdio uart backend.

bhyve was previously using stdin for both reading and writing to the
console, which made it difficult to redirect console output. Use
stdin for reading and stdout for writing. This makes it easier to use
bhyve as a backend for syzkaller.

As a side effect, the change fixes a minor bug which would cause bhyve
to fail with ENOTCAPABLE if configured to use nmdm for com1 and stdio
for com2.

bhyveload already uses separate descriptors, as does the bvmcons driver.

Reviewed by: jhb
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D19788

Fix indentation.

No functional change intended.

Reviewed by: bcran, jhb, rgrimes
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D19786

bhyve(8): Fix uart emulation bug

THRE is always asserted in LSR reads, so REG_IER writes that raise
IER_ETXRDY must also set thre_int_pending.

Reported by: Illumos, according to emaste@
https://twitter.com/ed_maste/status/1106195949087584258
MFC after: 2 weeks

Revert r343077 until the license issues surrounding it can be resolved.

Approved by: core@

Fix broken uart on Win2016 guest.

Obtained from: Joyent (commit/2bf1a940afbd1382faff159e7c93c72779ca10f4)
MFC after: 3 weeks.

Use capsicum_helpers(3) that allow us to simplify the code and its functions
will return success when the kernel is built without support of
the capability mode.

It is important to note, that I'm taking a more conservative approach
with these changes and it will be done in small steps.

Reviewed by: jhb
MFC after: 6 weeks
Differential Revision: https://reviews.freebsd.org/D18744

Fix style(9) space vs tab.

Reviewed by: jhb
MFC after: 3 weeks.
Sponsored by: iXsystems Inc.
Differential Revision: https://reviews.freebsd.org/D15768

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

bhyve: avoid applying capsicum capabilities to file that was not opened

When using -l option targeting file that can't be opened (ie. nmdm module
is not loaded and /dev/nmdm* is specified) bhyve tries to apply capsicum
capabilities to a file that was not opened.

Enclose that code in an if statement and only run it on correctly opened
descriptor also providing meaningful message in case of an error.

Submitted by: Pawel Biernacki <pawel.biernacki@gmail.com>
Reviewed by: grehan, emaste
Sponsoied by: Mysterious Code Ltd.
Differential Revision: D12985

Capsicum support for bhyve(8).

Adds Capsicum sandboxing to bhyve.

Submitted by: Pawel Biernacki <pawel.biernacki@gmail.com>
Reviewed by: grehan, oshogbo
Approved by: emaste, grehan
Sponsored by: Mysterious Code Ltd.
Differential Revision: https://reviews.freebsd.org/D8290

Always assert DCD and DSR in bhyve's uart emulation.

The /etc/ttys entry for a serial console in FreeBSD/x86 is as follows:
ttyu0 "/usr/libexec/getty 3wire" vt100 onifconsole secure

The initial terminal type passed to getty(8) is "3wire" which sets the
CLOCAL flag. However reset(1) clears this flag and any programs that try
to open the terminal will hang waiting for DCD to be asserted.

Fix this by always asserting DCD and DSR in the emulated uart.

The following discussion on virtualization@ has more details:
https://lists.freebsd.org/pipermail/freebsd-virtualization/2015-June/003666.html

Reported by: jmg
Discussed with: grehan

Disable the 'uart_drain()' callback when the emulated receive FIFO is full.

Failing to do this will cause the kevent(2) notification to trigger
continuously and the bhyve(8) mevent thread will hog the cpu until the
characters on the backend tty device are drained.

Also, make the uart backend file descriptor non-blocking to avoid a
select(2) before every byte read from that backend.

Reviewed by: grehan

Use calloc() in favor of malloc + memset.

Reviewed by: neel

Open the uart emulation's backing tty in non-blocking mode.
This fixes the issue of bhyve appearing to halt when using
nmdm ports for the console, until a connection is made to
the other end.

bhyveload already does this.

Reported by: Many.
MFC after: 3 weeks.

Allow bhyve and bhyveload to attach to tty devices.

bhyveload: introduce the -c <device> parameter
to select a tty for output (or "stdio")

bhyve: allow the puc and lpc-com backends to
accept a tty in addition to "stdio"

When used in conjunction with the null-modem device,
nmdm(4), this allows attach/detach to the guest console
and multiple concurrent serial ports. kgdb on a serial
port is now functional.

Reviewed by: neel
Requested by: Almost everyone that has used bhyve
MFC after: 10.0

Add support for PCI-to-ISA LPC bridge emulation. If the LPC bus is attached
to a virtual machine then we implicitly create COM1 and COM2 ISA devices.

Prior to this change the only way of attaching a COM port to the virtual
machine was by presenting it as a PCI device that is mapped at the legacy
I/O address 0x3F8 or 0x2F8.

There were some issues with the original approach:
- It did not work at all with UEFI because UEFI will reprogram the PCI device
BARs and remap the COM1/COM2 ports at non-legacy addresses.
- OpenBSD GENERIC kernel does not create a /dev/console because it expects
the uart device at the legacy 0x3F8/0x2F8 address to be an ISA device.
- It was functional with a FreeBSD guest but caused the console to appear
on /dev/ttyu2 which was not intuitive.

The uart emulation is now independent of the bus on which it resides. Thus it
is possible to have uart devices on the PCI bus in addition to the legacy
COM1/COM2 devices behind the LPC bus.

The command line option to attach ISA COM1/COM2 ports to a virtual machine is
"-s <bus>,lpc -l com1,stdio".

The command line option to create a PCI-attached uart device is:
"-s <bus>,uart[,stdio]"

The command line option to create PCI-attached COM1/COM2 device is:
"-S <bus>,uart[,stdio]". This style of creating COM ports is deprecated.

Discussed with: grehan
Reviewed by: grehan
Submitted by: Tycho Nightingale (tycho.nightingale@pluribusnetworks.com)

M share/examples/bhyve/vmrun.sh
AM usr.sbin/bhyve/legacy_irq.c
AM usr.sbin/bhyve/legacy_irq.h
M usr.sbin/bhyve/Makefile
AM usr.sbin/bhyve/uart_emul.c
M usr.sbin/bhyve/bhyverun.c
AM usr.sbin/bhyve/uart_emul.h
M usr.sbin/bhyve/pci_uart.c
M usr.sbin/bhyve/pci_emul.c
M usr.sbin/bhyve/inout.c
M usr.sbin/bhyve/pci_emul.h
M usr.sbin/bhyve/inout.h
AM usr.sbin/bhyve/pci_lpc.c
AM usr.sbin/bhyve/pci_lpc.h