bhyve: Fix vq_getchain() error handling bugs in various device models

Reviewed by: grehan, khng
Approved by: so
Security: CVE-2021-29631
Security: FreeBSD-SA-21:13.bhyve

(cherry picked from commit 71fbc6faed62e8eb5864f7c40839740f5e9f5558)

Git Hash: eb01dfe6c01c700418a872ae520187a1292ea7da
Git Author: markj@FreeBSD.org

MFC r362612

bhyve: allow for automatic destruction on power-off

MFC 360399:
Update the cached MSI state when any MSI capability register is written.

bhyve uses cached copies of the MSI capability registers to generate
MSI interrupts for device models. Previously, these cached fields
were only set when the MSI capability control register was updated.
The Linux kernel recently adopted a change to deal with races in MSI
interrupt delivery that writes to the MSI capability address and data
registers to alter the destination of MSI interrupts without writing
to the MSI capability control register. bhyve was not updating its
cached registers for these writes and continued to send interrupts
with the old data value to the old address. Fix this by recomputing
the cached values for every write to any MSI capability register.

MFC 358394: Use stream_read() to read all 12 bytes of the RFB client version.

read() can return a short read, whereas stream_read() waits until the
full version string is read.

MFC 348694: Don't simulate PBA access if the PBA is in a separate BAR.

bhyve has to virtualize the MSI-X table to trap reads and writes to
that table and map those to virtual interrupts that it maps real host
interrupts on to. For the pending-bit-array (PBA), bhyve passes
accesses from the guest directly to the hardware.

bhyve's virtualization of the MSI-X table is done by intercepting all
reads and writes to the BAR holding the MSI-X table. However, if the
PBA is stored in the same BAR as the MSI-X table, accesses to the PBA
portion of this BAR have to be forwarded to the real BAR.

However, in the case that the PBA was stored in a separate BAR and
it's offset in that separate BAR overlapped with the portion of the
MSI-X table BAR that the table used, the handlers for the table BAR
would incorrectly think that some accesses were PBA reads and writes.
This caused a crash in bhyve when it indirected a NULL pointer. Fix
this case by never trying to handle PBA access if the PBA lives in a
separate BAR.

MFC 350618:
Validate guest-supplied length of headers for TSO transmit requests.

When transmitting a large TCP packet, the final transmit descriptor
includes the length of the protocol headers to be duplicated on each
segment. The device model was trusting the guest-supplied value
without validating it. A value of zero would result in the guest
being able to indirect a garbage pointer on the stack to overwrite
arbitrary memory in the bhyve process. A value that was non-zero but
too small for the requested parameters resulted in the device model
reading and writing values beyond the end of the on-stack buffer used
to hold the template header.

To fix, validate the supplied length and drop requests to transmit
packets that would overflow the header buffer. While here, initialize
the header pointer to NULL as a preventive measure so that any access
to an unallocated template header crashes they hypervisor
deterministically.

While here, only read the TCP sequence number if the packet being
split is a TCP packet. The e1000 logic supports a segmentation of UDP
frames, and while UDP segmentation requires this part of the header to
be valid (so there is no buffer overflow), only reading the field when
needed is cleaner.

admbugs: 918
Reported by: Reno Robert <renorobert@gmail.com>
Approved by: so
Security: CVE-2019-5609

bhyve: Fix resource leak when using strdup

MFC r340044 (araujo):

Fix resource leak when using strdup(3).

MFC r344160 (rgrimes):

In r340044 an attempt to quiet coverity warning cid 1357336
was incorrectly implemented leading to a possible double free.

It is possible for both the conditional free,
and the unconditional free added in r340044 to be done,
fix that by initializing uopt to NULL,
removing the conditional free,
and only using the unconditional free at the end.

CID: 1357336
Reported by: Coverity

MFC r349915 (seanc): usr.sbin/bhyve: initialize return value ...

... in xhci device interrupt handler

Coverity CID: 1357340

MFC r350244: bhyve: correct out-of-bounds read in XHCI device emulation

Add appropriate bounds checks on the epid and streamid fields in the
device doorbell registers.

admbugs: 919
Submitted by: jhb
Reported by: Reno Robert <renorobert@gmail.com>
Reviewed by: markj
Approved by: so
Security: out-of-bounds read

Add Capsicumification of the virtio_console device model.

This is a direct commit to stable/11. This change was missed when
merging virtio_console to 11 because the capsicum change and
virtio_console changes were merged in the opposite order of the
changes in head.

MFC 339911,339936,343075,343166,348592: Various AMD CPU-specific fixes.

339911:
Emulate machine check related MSR_EXTFEATURES to allow guest OSes to
boot on AMD FX Series.

339936:
Merge cases with upper block.
This is a cosmetic change only to simplify code.

343075:
vmm(4): Take steps towards multicore bhyve AMD support

vmm's CPUID emulation presented Intel topology information to the guest, but
disabled AMD topology information and in some cases passed through garbage.
I.e., CPUID leaves 0x8000_001[de] were passed through to the guest, but
guest CPUs can migrate between host threads, so the information presented
was not consistent. This could easily be observed with 'cpucontrol -i 0xfoo
/dev/cpuctl0'.

Slightly improve this situation by enabling the AMD topology feature flag
and presenting at least the CPUID fields used by FreeBSD itself to probe
topology on more modern AMD64 hardware (Family 15h+). Older stuff is
probably less interesting. I have not been able to empirically confirm it
is sufficient, but it should not regress anything either.

343166:
vmm(4): Mask Spectre feature bits on AMD hosts

For parity with Intel hosts, which already mask out the CPUID feature
bits that indicate the presence of the SPEC_CTRL MSR, do the same on
AMD.

Eventually we may want to have a better support story for guests, but
for now, limit the damage of incorrectly indicating an MSR we do not yet
support.

Eventually, we may want a generic CPUID override system for
administrators, or for minimum supported feature set in heterogenous
environments with failover. That is a much larger scope effort than
this bug fix.

348592:
Emulate the AMD MSR_LS_CFG MSR used for various Ryzen errata.

Writes are ignored and reads always return zero.

PR: 224476, 235010

bhyve: vtnet: fix locking on receive

The vsc_rx_ready and the RX virtqueue is protected by the rx_mtx lock.
However, pci_vtnet_ping_rxq() (currently called only once after each
device reset) accesses those without acquiring the lock.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D20609

MFC r349019

bhyve: move common code to net_utils.c

Both virtio_net and e82545 network frontends have code to validate and
generate MAC addresses. These functionalities are replicated in the two
files, so we move them in a separate compilation unit.

Reviewed by: rgrimes, bryanv, imp, kevans
Differential Revision: https://reviews.freebsd.org/D20626

MFC r348929

bhyve: virtio: introduce vq_kick_enable() and vq_kick_disable()

The VirtIO standard supports two schemes for notification suppression:
a notification enable bit and a more sophisticated one (event_idx) that
also supports delayed notifications. Currently bhyve fully supports
only the first scheme. This patch hides the notification suppression
internals by means of two inline routines, vq_kick_enable() and
vq_kick_disable(), and makes the code more readable.
Moreover, further improve readability by replacing the call to mb()
with a call to atomic_thread_fence_seq_cst(), which is already used
in virtio.c

Reviewed by: pmooney_pfmooney.com, bryanv
Differential Revision: https://reviews.freebsd.org/D20581

MFC r348834

bhyve: vtnet: simplify thread synchronization

On vtnet device reset it is necessary to wait for threads to stop TX and
RX processing. However, the rx_in_progress variable (used for to wait for
RX processing to stop) is actually useless, and can be removed. Acquiring
and releasing the RX lock is enough to synchronize correctly. Moreover,
it is possible to reset the device while holding both TX and RX locks, so
that the "resetting" variable becomes unnecessary for the RX thread, and
can be protected by the TX lock (instead of being volatile).

Reviewed by: jhb, markj
Differential Revision: https://reviews.freebsd.org/D20543

MFC r349600.

Correct name of vmm(4) pptdevs variable.

MFC r349196:
Make zlib encoding messages idempotent.

PR: 238333

MFC 347033:
Increase the VirtIO segment count to support modern Windows guests.

The Windows virtio driver ignores the advertized seg_max field and
assumes the host can accept up to 67 segments in indirect descriptors,
triggering an assert in the bhyve process.

This brings back r282922 but with a couple of changes:
- It raises the block interface segment limit to 128 instead of 67.
- Linux's virtio driver assumes that the segment limit is no
larger than the ring size. To avoid breaking Linux guests,
raise the VirtIO ring size to 128, and cap the VirtIO segment
limit at ring size - 2 (effectively 126).

Approved by: re (gjb)

MFC 345158: Fix uart emulation bug

THRE is always asserted in LSR reads, so REG_IER writes that raise
IER_ETXRDY must also set thre_int_pending.

Approved by: re (gjb)

MFC: r346717: Make bhyve SMBIOS table topology aware

When the CPU Topology was added to bhyve in r332298 the SMBIOS table was
missed, this table passes topology information to the system and was still
using the old concept of each vCPU is a socket with 1 core and 1 thread.
This code did not even try to use the old sysctl information to adjust
this data.

Correct that by building a proper SMBios table, mapping the > 254 cases to
0 per the SMBios 2.6 specification that is claimed by the structure.

Approved by: re (kib)

MFC: r332298,333712,334199,334216,334219 bhyve cpu topology

Approved by: re (gjb), bde/phk (mentor, implicit)

MFC: r347960: bhyve virtio needs barriers

Under certain tight race conditions, we found that the lack of a memory
barrier in bhyve's virtio handling causes it to miss a NO_NOTIFY state
transition on block devices, resulting in guest stall. The investigation
is recorded in OS-7613. As part of the examination into bhyve's use of
barriers, one other section was found to be problematic, but only on
non-x86 ISAs with less strict memory ordering. That was addressed in
this patch as well, although it was not at all a problem on x86.

PR: 231117
Submitted by: Patrick Mooney <patrick.mooney@joyent.com>
Reviewed by: jhb, kib, rgrimes
Approved by: re (gjb), jhb
Differential Revision: https://reviews.freebsd.org/D19501

MFC 342944: Fix spelling of identifier

MFC 340707:
Define AHCI_PORT_IDENT and increase by 1 the VTBLK_BLK_ID_BYTES
to avoid buffer accessed out of bounds, also switch to snprintf(3).

PR: 200859

MFC 334272: bhyve: guarantee NUL termination

Use strlcpy to guarantee NUL termination of the path to a
virtio console socket.

MFC 334271: bhyve: fix small memory leak in virtio console

MFC 333235:
Allow arbitrary numbers of columns for VNC server screen resolution.

The prior code only allowed multiples of 32 for the
numbers of columns. Remove this restriction to allow
a forthcoming UEFI firmware update to allow arbitrary
x,y resolutions.

(the code for handling rows already supported non mult-32 values)

MFC: r346715: Acpi MADT table correction for VM_MAXCPU > 21

The bhyve acpi MADT table was given a static space of 256 (0x100) bytes,
this is enough space to allow VM_MAXCPU to be 21, this patch changes that
so VM_MAXCPU can be of arbitrary value and not overflow the space by
actually calculating the space needed for the table.

PR: 212782
Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
Approved by: bde (mentor, implicit), jhb (maintainer)
Differential Revision: https://reviews.freebsd.org/D18815

MFC 325727:
bhyve: avoid applying capsicum capabilities to file that was not opened

When using -l option targeting file that can't be opened (ie. nmdm module
is not loaded and /dev/nmdm* is specified) bhyve tries to apply capsicum
capabilities to a file that was not opened.

Enclose that code in an if statement and only run it on correctly opened
descriptor also providing meaningful message in case of an error.

MFC 313732: bhyve: whitespace fix for r313727

Fix build with r346707.

MFC r339826 (by yuripv):
Provide basic descriptions for VMX exit reason (from "Intel 64 and IA-32
Architectures Software Developer’s Manual Volume 3"). Add the document
to SEE ALSO in bhyve.8 (and pet manlint here a bit).

MFC r346010:
Fix indentation.

MFC r346011:
Stop compiling bhyve(8) with -O0.

MFC r340046, r340050

r340046:
Add support ps/2 scancodes for NumLock, ScrollLock and numerical keypad
keys.

PR: 213835
Submitted by: Vasily Postnicov <ivan.zhmudo@gmail.com>
Relnotes: Yes
Sponsored by: iXsystems Inc.

r340050:
Remove printf for debug purpose forgotten on r340046.

Reported by: Dmitry Luhtionov <dmitryluhtionov@gmail.com>

MFC r341484

Always treat firmware request and response sizes as unsigned.

This fixes an incomplete bounds check on the guest-supplied request
size where a very large request size could be interpreted as a negative
value and not be caught by the bounds check.

Submitted by: jhb
Reported by: Reno Robert
Approved by: so
Security: FreeBSD-SA-18:14.bhyve
Security: CVE-2018-17160

MFC r339586:

In bhyve's fbuf emulation improve the overall "usage" message and
for the vga option, rather than printing the entire option string,
only print vga (as we do for everything else).

MFC r339681:

Allow the bhyve VNC server to listen on IPv6 for incoming connections.

Alternatively to IPv4 address:port this will allow to listen on IPv6
link-local (incl. scope), a specific address, or ::. Addresses have
to be given in RFC2732 format so that [::]:port parsing will work.

This patch also starts to introduce WITH_INET/INET6_SUPPORT to bhyve.

PR: 232018
Submitted by: Dave Rush (northwoodlogic.free gmail.com) (original)
Reviewed by: Dave Rush (updated verison)

MFC r337189:
bhyve: set title before entering capability mode

PR: 230082
Submitted by: Yuichiro NAITO <naito.yuichiro@gmail.com>

MFC r336188:

Improve bhyve exit(3) error code.

The bhyve(8) exit status indicates how the VM was terminated:

0 rebooted
1 powered off
2 halted
3 triple fault

The problem is when we have wrappers around bhyve that parses the exit
error code and gets an exit(1) for an error but interprets it as "powered off".
So to mitigate this issue and makes it less error prone for third part
applications, I have added a new exit code 4 that is "exited due to an error".

For now the bhyve(8) exit status are:
0 rebooted
1 powered off
2 halted
3 triple fault
4 exited due to an error

Reviewed by: @jhb
Sponsored by: iXsystems Inc.
Differential Revision: https://reviews.freebsd.org/D16161

MFC r335027, r335050

r335027:
When this code was introduced at r300829 the author forgot to add
the BSD license header that is the same as in its C header file.

Sponsored by: iXsystems Inc.

r335050:
While I was investigating CID 1194192 related with a resource leak on mrp memory
allocation, I could identify that actually we use this pointer on pci_emul.c as
well as on vga.c source file.

I have reworked the logic here to make it more readable and also add a warn to
explicit show the function where the memory allocation error could happen,
also sort headers.

Also CID 1194192 was marked as "Intentional".

Obtained from: TrueOS
Sponsored by: iXsystems Inc.

MFC r335025:

Add SPDX tags to bhyve(8).

Discussed with: rgrimes, pfg and mav.
Obtained from: TrueOS
Sponsored by: iXsystems Inc.

MFC r334307, r335103-r335104

r334307:
Simplify macros EFPRINTF and EFFLUSH. [0]
Also stdarg(3) says that each invocation of va_start() must be paired
with a corresponding invocation of va_end() in the same function. [1]

Reported by: Coverity
CID: 1194318[0] and 1194332[1]
Discussed with: jhb
Sponsored by: iXsystems Inc.
Differential Revision: https://reviews.freebsd.org/D15548

r335103:
Fix style(9) space vs tab.

Reviewed by: jhb
Sponsored by: iXsystems Inc.
Differential Revision: https://reviews.freebsd.org/D15774

r335104:
Fix style(9) space vs tab.

Reviewed by: jhb
Sponsored by: iXsystems Inc.
Differential Revision: https://reviews.freebsd.org/D15768

MFC r333622, r334019, r334084

r333622:
vq_getchain() can return -1 if some descriptor(s) are invalid and prints
a diagnostic message. So we do a sanity checking on the return value
of vq_getchain().

Spotted by: gcc49
Reviewed by: avg
Sponsored by: iXsystems Inc.
Differential Revision: https://reviews.freebsd.org/D15388

r334019:
Include atkbdc header where there are declared the prototype functions
atkbdc_event and atkbdc_init.

Sponsored by: iXsystems Inc.

r334084:
pthread_rwlock_unlock(3) returns 0 if successful, otherwise an error number
will be returned to indicate the error, so I'm applying an assert(3) to do
a sanity check of the return value.

Reported by: Coverity CID: 1391235, 1193654 and 1193651
Reviewed by: grehan
Sponsored by: iXsystems Inc.
Differential Revision: https://reviews.freebsd.org/D15533

MFC r326276:

various: general adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.

bhyve: actually call bhyve_caph_cache_catpages

Gcc noticed that bhyve_caph_cache_catpages was defined but unused.
It is a stable/11 copy of caph_cache_catpages in head.

Direct commit to stable/11. This bug does not apply to head.

Reviewed by: grehan
Approved by: markj (mentor)
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12212

MFC r321669:

bhyve/vga.c: fix atc_color_select_67 bit shift

MFC r321668:

bhyve/pci_e82545.c: squelch gcc warning for noreturn procedure

MFC r305898, r309120, r309121 (by jceel):
Add virtio-console support to bhyve.

Adds virtio-console device support to bhyve, allowing to create
bidirectional character streams between host and guest.

Syntax:
-s <slotnum>,virtio-console,port1=/path/to/port1.sock,anotherport=...

Maximum of 16 ports per device can be created. Every port is named
and corresponds to an Unix domain socket created by bhyve. bhyve
accepts at most one connection per port at a time.

Limitations:
- due to lack of destructors of in bhyve, sockets on the filesystem
must be cleaned up manually after bhyve exits
- there's no way to use "console port" feature, nor the console port
resize as of now
- emergency write is advertised, but no-op as of now

MFC r302850: Make PCI interupts allocation static when using bootrom (UEFI).

This makes factual interrupt routing match one shipped with UEFI firmware.
With old firmware this make legacy interrupts work reliable for functions 0
of PCI slots 3-6. Updated UEFI image fixes problem completely.

MFC r317542, r317543, r317543

317542 comment fix
317543 set rfb default port
317543 listen on localhost by default for rfb

MFC 313727, 317483
In addition, replace the missing caph routines with
small helper functions (bhyverun.c) or an open-coded
replacement (uart_emul.c)

313727 Capsicumize bhyve
317483 Allow CAP_MMAP_RW on memfd for PCI passthru

MFC r311699

ps2 mouse fixes, found by plan9/9front.

Reminded by: sevan

MFC r319487-r319488, r319682, r319968, r319995

r319487:
Add VNC Authentication support based on RFC6143 section 7.2.2.

Submitted by: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
Reworked by: myself
Reviewed by: grehan, rgrimes and jilles
Relnotes: Yes.
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D10818

r319488:
Bump manpage date.

r319682:
Make the VNC authentication build with LibreSSL on HardenedBSD and TrueOS.

PR: 219790
Submitted by: Shirkdog <mshirk@daemon-security.com>
Reviewed by: grehan and rgrimes
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D11071

r319968:
Initialize variables and use byteorder(9) instead of aliasing char array
buf via uint32_t pointer.

CID: 1375949
Reported by: Coverity, cem
Reviewed by: cem
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D11153

r319995:
Check if pthread_create(3) successfully created the thread prior to call
pthread_join(3). The variable tid is not yet initialized in case
the authentication fails at early stage, that would lead pthread_join be
called with an uninitialized variable.

CID: 1375950
Reported by: Coverity, cem
Reviewed by: cem
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D11150

MFC r307517, r314342, r316357, r317545, r317777:

r307517:
Typo fixed: arbitraty -> arbitrary.
PR: 213559

r314342:
bhyve: document virtio-console in the manpage

r316357:
Minor style improvements in bhyve.8
Replace "as of now" with "at present". As the change is a really minor one,
don't bump .Dd.

r317545:
Document raw framebuffer device and XHCI device configurations.

r317777:
Improve documentation of fbuf device.

Approved by: grehan (mentor)
Approved by: re (gjb)

MFC r318788:
bhyvegc_resize: make use of reallocarray(3) for bounds-checking.

Also add __FBSDID.

Reviewed by: grehan

DIRDEPS_BUILD: Update dependencies.

This is a direct commit since MFCing these changes is impractical.

Sponsored by: Dell EMC Isilon

MFC r316427, r316428: Add Log directory and SATA NCQ Send and Receive Log.

Those are used at least by Linux guests to detect queued TRIM support.

MFC r315715 Fix a type in bhyve's USB mouse emulation.

MFC r311702
Use correct PCI device id for virtio-rng.
This prevented the device from attaching with a
Windows guest (most other guests use the device type
for matching)

PR: 212711

MFH: r309295

bhyve: stability and performance improvement for dbgport

The TCP server implementation in dbgport does not track clients, so it
may try to write to a disconected socket resulting in SIGPIPE.
Avoid that by setting SO_NOSIGPIPE socket option.

Because dbgport emulates an I/O port to guest, the communication is done
byte by byte. Reduce latency of the TCP/IP transfers by using
TCP_NODELAY option. In my tests that change improves performance of
kgdb commands with lots of output (e.g. info threads) by two orders of
magnitude.

A general note. Since we have a uart emulation in bhyve, that can be
used for the console and gdb access to guests. So, bvmconsole and bvmdebug
could be de-orbited now. But there are many existing deployments that
still dependend on those.

Discussed with: julian, jhb
Sponsored by: Panzura

Merge r307937:
Fix unchecked array reference in the VGA device emulation code.

Submitted by: Ilja Van Sprundel <ivansprundel ioactive.com>
Patch by: tychon
Security: SA-16:32

accept4 actually expect SOCK_NONBLOCK and not O_NONBLOCK

Reported by: jhb
Pointyhat to: bapt

MFC r306554:

Use accept4 with O_NONBLOCK rather than accept + fcntl

MFC r305061
Invert calloc(3) argument order

MFC r303352
- Change the fbuf "vga" parameter to "vga=on|io|off".
"io" is the default, and allows VGA i/o registers to be
accessed. This is required by Win7/2k8 graphics guests that
use a combination of BIOS int10 and UEFI.
"off" disables all VGA i/o and mem accesses.
"on" is not yet hooked up, but will enable full VGA rendering.

OpenBSD/UEFI >= 5.9 graphics guests can be booted using "vga=off"

- Allow "rfb" to be used instead of "tcp" for the fbuf VNC
description. "tcp" will be removed at a future point and is
kept as an alias.

MFC r302972,r303349

r302972
Disallow interrupt requests on disabled endpoints.

r303349
Catch another case where an XHCI interrupt was being
injected without state being set up.

MFC r302546
Implement right shift/ctl, and convert the VNC/xorg scancode
of 0xff03 into right-alt.

MFC r302504, r302666, r302668, r302932, r302933:
Add emulation for Intel e1000 (e82545) network adapter.

The code was successfully tested with FreeBSD, Linux, Solaris and Windows
guests. This interface is predictably slower (about 2x) then virtio-net,
but it is very helpful for guests not supporting virtio-net by default.

Thanks to Jeremiah Lott and Peter Grehan for doing original heavy lifting.

MFC r302460: Add emulation for multiple (up to 16) MSI vectors for AHCI.

It was useless before, but may improve performance now if multiple devices
are configured and guest supports this feature.

Sponsored by: iXsystems, Inc.

MFC r302459: Allow AHCI controller to support up to 32 arbitrary devices.

While old syntax is still supported, new syntax looks like this:

-s 3,ahci,hd:/dev/zvol/XXX,hd:/dev/zvol/YYY,cd:/storage/ZZZ.iso

Sponsored by: iXsystems, Inc.

MFC r302957: Fix NCQ TRIM emulation.

When this code was written, there was no guests using it to test.

Approved by: re (kib)

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Fix CTASSERT issue in a more clean way

- Replace all CTASSERT macro instances with static_assert's.
- Remove the WRAPPED_CTASSERT macro; it's now an unnecessary obfuscation.
- Localize all static_assert's to the structures being tested.
- Sort some headers per-style(9).

Approved by: re (hrs)
Differential Revision: https://reviews.freebsd.org/D7130
MFC after: 1 week
X-MFC with: r302364
Reviewed by: ed, grehan (maintainer)
Submitted by: ed
Sponsored by: EMC / Isilon Storage Division

Fix gcc warning

Remove -Wunused-but-set-variable (`mopt`).

Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D7119
MFC after: 1 week
X-MFC with: r302332
Reported by: Jenkins
Reviewed by: grehan (maintainer)
Sponsored by: EMC / Isilon Storage Division

Fix gcc build errors with SSE 4.2 detection and gcc warnings

- Remove -Wunused-but-set-variable's (`len`, etc).
- Replace clang-specific tests in sse42_supported(..) with generic,
FreeBSD-supported CPU feature tests, using macros and functions
from machine/cpufunc.h and machine/specialreg.h . The previous method
for determining SSE4.2 availability was only compatible with clang.
- Sort #includes per style(9).

Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D7119
MFC after: 1 week
X-MFC with: r302332
Reported by: Jenkins
Reviewed by: grehan (maintainer)
Sponsored by: EMC / Isilon Storage Division

Fix gcc warnings

Remove unused function (`fifo_available`)

Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D7119
MFC after: 1 week
X-MFC with: r302332
Reported by: Jenkins
Reviewed by: grehan (maintainer)
Sponsored by: EMC / Isilon Storage Division

Fix gcc warnings

- Put parentheses around bitwise OR'ed values in the `FIELD_COPY(..)` and
`FIELD_REPLACE(..)` macros to mute warning from gcc 4.2.1.
- Remove -Wunused-but-set-variable's (`setup_addr`, `status_addr`).

Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D7119
MFC after: 1 week
X-MFC with: r302332
Reported by: Jenkins
Reviewed by: grehan (maintainer)
Sponsored by: EMC / Isilon Storage Division

Fix gcc warnings

Remove -Wunused-but-set-variable (`error`). Cast calls with
`(void)` to note that the return value is explicitly ignored.

Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D7119
MFC after: 1 week
Reported by: Jenkins
Reviewed by: grehan (maintainer)
Sponsored by: EMC / Isilon Storage Division

Fix gcc warnings

Add `WRAPPED_CTASSERT` macro by annotating CTASSERTs with __unused
to deal with -Wunused-local-typedefs warnings from gcc 4.8+.
All other compilers (clang, etc) use CTASSERT as-is. A more generic
solution for this issue will be proposed after ^/stable/11 is forked.

Consolidate all CTASSERTs under one block instead of inlining them in
functions.

Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D7119
MFC after: 1 week
Reported by: Jenkins
Reviewed by: grehan (maintainer)
Sponsored by: EMC / Isilon Storage Division

Fix gcc warnings

Put cfl/prdt under AHCI_DEBUG #defines as they are only used in
those cases.

Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D7119
MFC after: 1 week
Reported by: Jenkins
Reviewed by: grehan (maintainer)
Sponsored by: EMC / Isilon Storage Division

Fix gcc warnings

- Remove -Wunused-but-set-variable (newcpu)
- Always return VMEXIT_CONTINUE as the code always set retval to that value.

Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D7119
MFC after: 1 week
Reported by: Jenkins
Reviewed by: grehan (maintainer)
Sponsored by: EMC / Isilon Storage Division

Import bhyve_graphics into CURRENT. Thanks to all who tested
this on the branch.

Original commit message:
Initial bhyve native graphics support.

This adds emulations for a raw framebuffer device, PS2 keyboard/mouse,
XHCI USB controller and a USB tablet.

A simple VNC server is provided for keyboard/mouse input, and graphics
output.

A VGA emulation is included, but is currently disconnected until an
additional bhyve change to block out VGA memory is committed.

Credits:
- raw framebuffer, VNC server, XHCI controller, USB bus/device emulation
and UEFI f/w support by Leon Dang
- VGA, console/g, initial VNC server by tychon@
- PS2 keyboard/mouse jointly done by tychon@ and Leon Dang
- hypervisor framebuffer mem support by neel@

Tested by: Michael Dexter, in a number of revisions of this code.

With the appropriate UEFI image, FreeBSD, Windows and Linux guests can
installed and run in graphics mode using the UEFI/GOP framebuffer.

Approved by: re (gjb)

bhyve: improve memory size documentation

A couple of minor memory size option related nits:

- use common name 'memsize' (instead of 'max-size' or just 'size')
- bhyve: update usage with memsize unit suffix, drop legacy "MB"
unit
- bhyveload: update usage with memsize unit suffix
- bhyve(8): document default size
- bhyveload(8): use memsize formatting like it's done
in bhyve(8)

Reviewed by: wblock, grehan
Approved by: re (kib), wblock, grehan
Differential Revision: https://reviews.freebsd.org/D6952

Improve error message when failing to open a backing file

When bhyve cannot open a backing file, it now says explicitly which file
could not be opened

Note that the change has only be maed in block_if.c and not in
pci_virtio_block.c as the error will always be catched by the first

PR: 202321 (different patch)
Reviewed by: grehan
MFC after: 3 day
Sponsored by: Gandi.net
Differential Revision: https://reviews.freebsd.org/D6576

bhyve: consider the bogus case of a negative bar idx.

This is a followup to r297472 to squelch Coverity.

CID: 1194319

bhyve: replace uninitialized variable "offset".

Use io->io_req.br_offset instead.

Suggested by: grehan
CID: 1288937

Use MIN()/MAX() macros from sys/param.h.

Reviewed by: grehan
Differential Revision: https://reviews.freebsd.org/D6054

Don't use SYSDIR to avoid conflicts with existing usage.
Also, use SRCTOP to locate the top of the source tree
instead of a relative path.

PR: 208856

Update the bhyve(8) man page:

- Document powering off by sending SIGTERM signal
- Document exit codes

Reviewed by: wblock, neel
Approved by: wblock
Differential Revision: D5982

Always emit an error message on passthru configuration errors.

Previously, many errors (such as the PCI device not being attached
to the ppt(4) driver) resulted in bhyve silently exiting without
starting the virtual machine. Now any errors encountered when
configuring a virtual slot for a PCI passthru device should be noted
on stderr.

Reviewed by: neel
Differential Revision: https://reviews.freebsd.org/D5990

Merge the projects/release-pkg branch to head.

This allows packaging the base system with pkg(8), including
but not limited to providing the ability to provide upstream
binary update possibilities for non-tier-1 architectures.

This merge is a requirement of the 11.0-RELEASE, and as such,
thank you to everyone that has tested the project branch.

Documentation in build(7) etc. is still somewhat sparse, but
updates to those parts will follow.

Sponsored by: The FreeBSD Foundation

Final pass through bogus svn:mergeinfo removal.

Note, paths part of 'contrib' were left alone for now.

Sponsored by: The FreeBSD Foundation

Handle PBA that shares a page with MSI-X table for passthrough devices.

If the PBA shares a page with the MSI-X table, map the shared page via
/dev/mem and emulate accesses to the portion of the PBA in the shared
page by accessing the mapped page.

Reviewed by: grehan
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D5919

Allow the location of the kernel source tree to be overridden.
This makes it easier for the bhyve executable to be built
out of the tree.

bhyve: Remove unneeded variable ncq.

ncq was not being inititialized properly but it was not actually
necessary either, so make the code smaller by removing it.

CID: 1248842
Reviewed by: grehan

pci_emul_dior(): fix uninitialized scalar variable.

Prevent from returning an unitialized value in case the
ior size is unknown.

CID: 1194319
Reviewed by: grehan

Fix typo: nmd->cur_tx_ring should be used in pci_vtnet_netmap_writev()
The buffer length should be checked to avoid overflow, but there
is no API to get the slot length, so the hardcoded value is used.
Return the currently-first request chain back to the available
queue if there are no more packets.
Report the link as up if we managed to open vale port.
Use consistent coding style.

Submitted by: btw
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D5595

Upgrade our copies of clang, llvm, lldb and compiler-rt to 3.8.0
release.

Please note that from 3.5.0 onwards, clang, llvm and lldb require C++11
support to build; see UPDATING for more information.

Release notes for llvm and clang will soon be available here:
<http://llvm.org/releases/3.8.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.8.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste, Roman Divacky, Davide Italiano and Antoine Brodin
for their help.

Relnotes: yes

Cleanup unused-but-set-variable spotted by gcc-4.9.

Reviewed by: neel
Approved by: rodrigc (mentor)
Differential Revision: https://reviews.freebsd.org/D5042

Fix bhyve(1) operation on vmnet devices, broken in r293459.

Add netmap support for bhyve

Submitted by: btw
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D4826

Remove unused variable after r292981 to unbreak the build.

Clean up unused-but-set-variable spotted by gcc-4.9.

Reviewed by: grehan
Approved by: rodrigc (mentor)
Differential Revision: https://reviews.freebsd.org/D4734

Clean up unused-but-set-variable spotted by gcc-4.9.

Reviewed by: grehan
Approved by: bapt (mentor)
Differential Revision: https://reviews.freebsd.org/D4735

Improve collation string and locales support

Merge collation support from Illumos and DragonflyBSD.

Locales are now generated with the new localedef(1) tool from CLDR POSIX files.
The generated files are now identified as "BSD 1.0" format.

The libc now only read "BSD 1.0" locales definitions, all other version will be
set to "C"
The localedef(1) tool has been imported from Illumos and modified to use tree(3)
instead of the CDDL avl(3)
A set of tool created by edwin@ and extended by marino@ for dragonfly has been
added to be able to generate locales and the Makefiles from the vanilla CLDR
unicode databases + a universal UTF-8 charmap (by marino@)
Update the locales to unicode v27
Given our regex(3) does not support multibyte (yet) it has been forced to always
use locale C
Remove now unused colldef(1) and mklocale(1)
Finish implementing the numeric BSD extension for ctypes
The number of supported locales has grown from 175 to 250 locales. Among the new
locales: 6 Arabic locales (AE EG JO MA QA SA), Different variations of spanish
locales.
Added new 3 components locales for mn_Cyrl_MN, sr_Cyrl_RS sr_Latn_RS,
zh_Hans_CN, zh_Hant_HK and zh_Hant_TW. Some aliases has been for 2 components
version when possible.

Thanks: Garrett D'Amore (Illumos) who made sure all his work was done under
BSD license!, Edwin Groothuis (edwin@) for the work he made on tools to be able
to generate locales definition usable in freebsd sources out of vanilla CLDR
definitions, John Marino (DragonflyBSD) who first merge the Illumos work into
Dragonfly and spent hours tracking down bugs.

Merge mpsutil(8) branch

mpsutil(8)/mprutil(8) are new utilities for managing LSI Fusion-MPT
2/3 controllers (mps(4) and mpr(4))

For now only informational commands have been implemented.

This utility has been written by scottl@ [1] and polished by myself[2]

Submitted by: scottl
Discussed with: scottl
Relnotes: yes
Sponsored by: Netflix [1]
Sponsored by: Gandi.net [2]

Exit with a user-friendly message instead of tripping an assert
if vm_activate_cpu(..) fails when called from fbsdrun_addcpu(..)

MFC after: 1 week
PR: 203884
Reviewed by: grehan
Submitted by: William Orr <will@worrbase.com>

Fix a ton of speelling errors

arc lint is helpful

Reviewed By: allanjude, wblock, #manpages, chris@bsdjunk.com
Differential Revision: https://reviews.freebsd.org/D3337

Upgrade our copies of clang, llvm, lldb, compiler-rt and libc++ to 3.7.0
release.

Please note that from 3.5.0 onwards, clang, llvm and lldb require C++11
support to build; see UPDATING for more information.

Release notes for llvm and clang can be found here:
<http://llvm.org/releases/3.7.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.7.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste, Andrew Turner and Antoine Brodin for their help.

Exp-run: antoine
Relnotes: yes

Clean up some harmless unimplemented-command warning messages.

- Don't advertize trusted-computing capability in the Identify page.
This prevents Windows from issuing a TRUSTED_RECEIVE_DMA command.
- Windows will send down SMART and SECURITY_FREEZE_LOCK
even though smart and security capabilities were not advertized.
Send back a silent abort.

Reviewed by: mav

Fix post-test typo that snuck in.

Simple sysctl-like firmware query interface. Similar in operation
to the qemu one, and uses the same i/o ports but with different
messaging. Requires the 'bootrom' option to be enabled.

This is used by UEFI (and potentially other BIOSs/firmware) to
request information from bhyve. Currently, only the number of
vCPUs is made available, with more to follow.

A very large thankyou to Ben Perrault who helped out testing
an earlier version of this, and bhyve/Windows in general.

Reviewed by: tychon
Discussed with: neel
Sponsored by: Nahanni Systems

- Increase the max number of indirect descriptors to match
the largest that the Windows virtio driver can send down

- Always advertize indirect descriptors. The Illumos virtio
driver won't attach unless this capability is seen.

Reviewed by: neel

Use strlcpy() instead of strncpy() because subsequent mkstemps expects
the string be nul-terminated.

Reviewed by: neel
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D3685

Add simple (no-op) emulations for the CHECK_POWER_MODE,
READ_VERIFY and READ_VERIFY_EXT commands.

Reviewed by: mav

Another small typo.

MFC after: 3 days

Fix minor typo.

MFC after: 3 days

Manpage cleanup.

- new sentence -> new line
- fix manpage references
- fix macro usage
- fix a typo

MFC after: 1 week

Always assert DCD and DSR in bhyve's uart emulation.

The /etc/ttys entry for a serial console in FreeBSD/x86 is as follows:
ttyu0 "/usr/libexec/getty 3wire" vt100 onifconsole secure

The initial terminal type passed to getty(8) is "3wire" which sets the
CLOCAL flag. However reset(1) clears this flag and any programs that try
to open the terminal will hang waiting for DCD to be asserted.

Fix this by always asserting DCD and DSR in the emulated uart.

The following discussion on virtualization@ has more details:
https://lists.freebsd.org/pipermail/freebsd-virtualization/2015-June/003666.html

Reported by: jmg
Discussed with: grehan

Updated depends

add SO_REUSEADDR when starting debug port, lets you still bind when
a TIME_WAIT socket is still around...

Reviewed by: grehan
Review: https://reviews.freebsd.org/D2875

Restructure memory allocation in bhyve to support "devmem".

devmem is used to represent MMIO devices like the boot ROM or a VESA framebuffer
where doing a trap-and-emulate for every access is impractical. devmem is a
hybrid of system memory (sysmem) and emulated device models.

devmem is mapped in the guest address space via nested page tables similar
to sysmem. However the address range where devmem is mapped may be changed
by the guest at runtime (e.g. by reprogramming a PCI BAR). Also devmem is
usually mapped RO or RW as compared to RWX mappings for sysmem.

Each devmem segment is named (e.g. "bootrom") and this name is used to
create a device node for the devmem segment (e.g. /dev/vmm/testvm.bootrom).
The device node supports mmap(2) and this decouples the host mapping of
devmem from its mapping in the guest address space (which can change).

Reviewed by: tychon
Discussed with: grehan
Differential Revision: https://reviews.freebsd.org/D2762
MFC after: 4 weeks

Fix non-deterministic delays when accessing a vcpu that was in "running" or
"sleeping" state. This is done by forcing the vcpu to transition to "idle"
by returning to userspace with an exit code of VM_EXITCODE_REQIDLE.

MFC after: 2 weeks

The 'hostbridge' device exists to allow guests to infer msi/msix
capablity by advertising pcie capability.

Since the 'hostbridge' device isn't a true pci-to-pci bridge, and
doesn't actaully use the bridge configuration space layout, change
the header-type from type 1 to type 0 to avoid confusion.

Reviewed by: neel

Temporarily revert r282922 which bumped the max descriptors.

While there is no issued with the number of descriptors in
a virtio indirect descriptor, it's a guest's choice as to
whether indirect descriptors are used. For the case where
they aren't, the virtio block ring size is still 64 which
is less than the now reported max_segs of 67. This results
in an assertion in recent Linux guests even though it was
benign since they were using indirect descs.

The intertwined relationship between virtio ring size,
max seg size and blockif queue size will be addressed
in an upcoming commit, at which point the max descriptors
will again be bumped up to 67.

Bump the size of the blockif scatter-gather list to 67.

The Windows virtio driver ignores the advertized seg_max
field and assumes the host can accept up to 67 segments
in indirect descriptors, triggering an assert in the bhyve
process.

No objection from: mav
Reviewed by: neel
Reported and tested by: Leon Dang (ldang@nahannisys.com)
MFC after: 2 weeks

Set the subvendor field in config space to the vendor ID.
This is required by the Windows virtio drivers to correctly
match a device.

Submitted by: Leon Dang (ldang@nahannisys.com)
MFC after: 2 weeks

Allow configuration of the sector size advertised to the guest.

The default behavior is to infer the logical and physical sector sizes from
the block device backend. However older versions of Windows only work with
specific logical/physical combinations:
- Vista and Windows 7: 512/512
- Windows 7 SP1: 512/512 or 512/4096

For this reason allow the sector size to be specified using the following
block device option: sectorsize=logical[/physical]

Reported by: Leon Dang (ldang@nahannisys.com)
Reviewed by: grehan
MFC after: 2 weeks

Handling indirect descriptors is a capability of the host and
not one that needs to be negotiated. Use the host capabilities
field and not the negotiated field when verifying that indirect
descriptors are supported.

Found with the Redhat Windows viostor driver, which clears
the indirect capability in the negotiated caps and then starts
using them.

Reported and tested by: Leon Dang (ldang@nahannisys.com)
MFC after: 2 weeks

Allow byte reads of AHCI registers.

This is needed to support Windows guests that use byte reads to access certain
AHCI registers (e.g. PxTFD.Status and PxTFD.Error).

Reviewed by: grehan, mav
Reported by: Leon Dang (ldang@nahannisys.com)
Differential Revision: https://reviews.freebsd.org/D2469
MFC after: 2 weeks

Add memory barrier to r281764.

While race at this point may cause only a single packet delay and so was
not really reproduced, it is better to not have it at all.

MFC after: 1 week

Deprecate the 3-way return values from vm_gla2gpa() and vm_copy_setup().

Prior to this change both functions returned 0 for success, -1 for failure
and +1 to indicate that an exception was injected into the guest.

The numerical value of ERESTART also happens to be -1 so when these functions
returned -1 it had to be translated to a positive errno value to prevent the
VM_RUN ioctl from being inadvertently restarted. This made it easy to introduce
bugs when writing emulation code.

Fix this by adding an 'int *guest_fault' parameter and setting it to '1' if
an exception was delivered to the guest. The return value is 0 or EFAULT so
no additional translation is needed.

Reviewed by: tychon
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D2428

Reimplement queue freeze on error, added in r282429:

It is not required to use CLO to recover from task file error, it should
be enough to do only stop/start, that does not clear the PxTFD.STS.ERR.

MFC after: 13 days

Implement in-order execution of non-NCQ commands.

Using status updates in r282364, block queue on BSY, DRQ or ERR bits set.
This can be a performance penalization for non-NCQ commands, but it is
required for proper error recovery and standard compliance.

MFC after: 2 weeks

Implement basic PxTFD.STS.BSY reporting.

MFC after: 2 weeks

Initialize PxCMD on reset and make its read-only bits such.

MFC after: 2 weeks

Handle ATA_SEND_FPDMA_QUEUED as NCQ in ahci_port_stop().

MFC after: 1 week

Advertise an additional memory BAR in the "dummy" device emulation.

This is useful for testing the MOVS emulation when both the source and
destination addresses are in the MMIO space.

MFC after: 1 week

Implement the century byte in the RTC. Some guests require this field to be
properly set.

Reported by: Leon Dang (ldang@nahannisys.com)
MFC after: 2 weeks

Don't allow guest to modify readonly bits in the PCI config 'status' register.

Reported by: Leon Dang (ldang@nahannisys.com)
MFC after: 2 weeks

Reassign copyright statements on several files from Advanced
Computing Technologies LLC to Hudson River Trading LLC.

Approved by: Hudson River Trading LLC (who owns ACT LLC)
MFC after: 1 week

Don't set bits that should be zero for SATA devices.

Old value made Linux think that it is PATA device with SATA bridge.

MFC after: 2 weeks

Report link as up if tap device is not specified (black hole).

MFC after: 2 weeks

Report link as up only if we managed to open tap device.

It would be cool to report tap device status, but it has no such API.

MFC after: 2 weeks

Disable RX/TX queues notifications when not needed.

This reduces CPU load and doubles iperf throughput, reaching 2-3Gbit/s.

MFC after: 2 weeks
Sponsored by: iXsystems, Inc.

Workaround bhyve virtual disks operation on top of GEOM providers.

GEOM does not support scatter/gather lists in its I/Os. Such requests
are cut in pieces by physio(), that may be problematic, if those pieces
are not multiple of provider's sector size. If such case is detected,
move the data through temporary sequential buffer.

MFC after: 2 weeks

Make virtual AHCI more careful with I/O lengths.

MFC after: 2 weeks

If the number of guest vcpus is less than '1' then flag it as an error.

MFC after: 1 week

Prior to aborting due to an ioport error, it is always interesting to
see what the guest's %rip is.

Reviewed by: grehan

Fix overlinking in bhyve:

libvmmapi is actually needed to be linked to libutil, not bhyve nor bhyveload

Prior to aborting due to an instruction emulation error, it is always
interesting to see what the guest's %rip and instruction bytes are.

Reviewed by: grehan

Move legacy interrupt allocation for virtio devices to common code.
There are a number of assumptions about legacy interrupts always
being available in virtio so don't allow back-ends to make the
decision to support them.

This fixes the issue seen with virtio-rnd on OpenBSD. MSI-x vectors
were not being used, and the virtio-rnd backend wasn't allocating a
legacy interrupt resulting in a bhyve assert and guest exit.

Reported by: Julian Hsiao, madoka at nyanisore dot net
Reviewed by: neel
MFC after: 1 week

Add missing variable initialization.

Reported by: Coverity
CID: 1288938
MFC after: 3 days

Report that we may have write cache, and that we do support FLUSH.

FreeBSD guest driver does not use that legacy flag, but Linux seems does.

MFC after: 2 weeks

Increase S/G list size of 32 to 33 entries.

32 entries are not enough for the worst case of misaligned 128KB request,
that made FreeBSD to chunk large quests in odd pieces.

MFC after: 2 weeks

Pre-allocate one extra request per processing thread.

Processing threads call callbacks before freeing requests. As result,
new requests may arrive before old ones are freed.

MFC after: 2 weeks

According to Linux and QEMU, s/n equal to buffer is not zero-terminated.

This makes same s/n reported for both virtio and AHCI drivers.

MFC after: 2 weeks

Close potential race on blockif_close().

Reported by: vangyzen
MFC after: 2 weeks

Fix networking problem after r280026.

I've missed that network driver sometimes returns taken request back to
available queue without processing. Add new helper function for that case.

Reported by: flo
MFC after: 2 weeks

Give AHCI disk serial based on backing file path same as for virtio block.

It is still not good that they may intersect on different hosts, but that
is better then intersecting on the same host.

MFC after: 2 weeks

Rewrite virtio block device driver to work asynchronously and use the block
I/O interface.

Asynchronous operation, based on r280026 change, allows to not block virtual
CPU during I/O processing, that on slow/busy storage can take seconds.
Use of recently improved block I/O interface allows to process multiple
requests same time, that improves random I/O performance on wide storages.

Benchmarks of virtual disk, backed by ZVOL on RAID10 pool of 4 HDDs, show
~3.5 times random read performance improvements, while no degradation on
linear I/O. Guest CPU usage during test dropped from 100% to almost zero.

MFC after: 2 weeks

Upgrade our copy of clang, llvm and lldb to 3.6.0 release.

Please note that from 3.5.0 onwards, clang/llvm/lldb require C++11
support to build; see UPDATING for more information.

Release notes for llvm and clang can be found here:
<http://llvm.org/releases/3.6.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.6.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste for the lldb part of this upgrade.

Exp-run: antoine

Modify virtqueue helpers added in r253440 to allow queuing.

Original virtqueue design allows queued and out-of-order processing, but
helpers added in r253440 suppose only direct blocking in-order one.
It could be fine for network, etc., but it is a huge limitation for storage
devices.

Block delete capability for read-only devices.

Submitted by: neel
MFC after: 2 weeks

Give block I/O interface multiple (8) execution threads.

On parallel random I/O this allows better utilize wide storage pools.
To not confuse prefetcher on linear I/O, consecutive requests are executed
sequentially, following the same logic as was earlier implemented in CTL.

Benchmarks of virtual AHCI disk, backed by ZVOL on RAID10 pool of 4 HDDs,
show ~3.5 times random read performance improvements, while no degradation
on linear I/O.

MFC after: 2 weeks

Add checksums to identify data and NCQ command error log.

MFC after: 2 weeks

Slightly polish virtual AHCI CD reporting.

MFC after: 2 weeks

Fix NOP and IDLE commands for virtual AHCI disks.

MFC after: 2 weeks

Add support for NCQ variant of DSM TRIM for virtual AHCI disks.

The code is not really tested yet due to lack of initiator support.

Requested by: imp
MFC after: 2 weeks

Improve NCQ errors reporting for virtual AHCI disks.

While this implementation is still not perfect, previous was just broken.

MFC after: 2 weeks

Remove incorrect SERR register setting.

At this point we have nothing to report through that register.

MFC after: 2 weeks

Change prdbc value reporting.

MFC after: 2 weeks

Polish AHCI disk identify data and fix speed negotiation.

MFC after: 2 weeks

Add support for PIO variants of READ/WRITE commands for AHCI disks.

AHCI API hides all PIO specifics, so this functionality is almost free.

MFC after: 2 weeks

Use ahci_write_fis_d2h() for commands completion.

MFC after: 2 weeks

Add DSM TRIM command support for virtual AHCI disks.

It works only for virtual disks backed by ZVOLs and raw devices supporting
BIO_DELETE. Virtual disks backed by files won't report this capability.

MFC after: 2 weeks
Relnotes: yes

Add variable initialization missed by me and clang.

Reported by: grehan
MFC after: 2 weeks

Fix error translation broken in r279658.

Reported by: grehan
MFC after: 2 weeks

Implement cache flush for ahci-hd and for virtio-blk over device.

MFC after: 2 weeks

Add check for absent stripe size to r279652.

MFC after: 2 weeks

Report logical/physical sector sizes for virtual SATA disk.

MFC after: 2 weeks

Add support for TOPOLOGY feature of virtio block device.

Passing through physical block size/offset from underlying storage allows
guest to manage proper data and I/O alignment to improve performance.

MFC after: 2 weeks

Emulate MSR 0xC0011024 when running on AMD processors.

OpenBSD guests test bit 0 of this MSR to detect whether the workaround for
erratum 721 has been applied.

Reported by: Jason Tubnor (jason@tubnor.net)
MFC after: 1 week

Add "-u" option to bhyve(8) to indicate that the RTC should maintain UTC time.

The default remains localtime for compatibility with the original device model
in bhyve(8). This is required for OpenBSD guests which assume that the RTC
keeps UTC time.

Reviewed by: grehan
Pointed out by: Jason Tubnor (jason@tubnor.net)
MFC after: 2 weeks

Don't close a block context if it couldn't be opened,
for example if the backing file doesn't exist,
avoiding a null deref.

Reviewed by: neel
MFC after: 1 week.

Simplify instruction restart logic in bhyve.

Keep track of the next instruction to be executed by the vcpu as 'nextrip'.
As a result the VM_RUN ioctl no longer takes the %rip where a vcpu should
start execution.

Also, instruction restart happens implicitly via 'vm_inject_exception()' or
explicitly via 'vm_restart_instruction()'. The APIs behave identically in
both kernel and userspace contexts. The main beneficiary is the instruction
emulation code that executes in both contexts.

bhyve(8) VM exit handlers now treat 'vmexit->rip' and 'vmexit->inst_length'
as readonly:
- Restarting an instruction is now done by calling 'vm_restart_instruction()'
as opposed to setting 'vmexit->inst_length' to 0 (e.g. emulate_inout())
- Resuming vcpu at an arbitrary %rip is now done by setting VM_REG_GUEST_RIP
as opposed to changing 'vmexit->rip' (e.g. vmexit_task_switch())

Differential Revision: https://reviews.freebsd.org/D1526
Reviewed by: grehan
MFC after: 2 weeks

Upgrade our copy of clang, llvm and lldb to 3.5.0 release.

Please note that this version now requires C++11 support to build; see
UPDATING for more information.

Release notes for llvm and clang can be found here:
<http://llvm.org/releases/3.5.0/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.5.0/tools/clang/docs/ReleaseNotes.html>

Thanks to Ed Maste, Roman Divacky, Andrew Turner, Justin Hibbits and
Antoine Brodin for their invaluable help with this import.

Approved by: portmgr (antoine)
MFC after: 1 month

Replace bhyve's minimal RTC emulation with a fully featured one in vmm.ko.

The new RTC emulation supports all interrupt modes: periodic, update ended
and alarm. It is also capable of maintaining the date/time and NVRAM contents
across virtual machine reset. Also, the date/time fields can now be modified
by the guest.

Since bhyve now emulates both the PIT and the RTC there is no need for
"Legacy Replacement Routing" in the HPET so get rid of it.

The RTC device state can be inspected via bhyvectl as follows:
bhyvectl --vm=vm --get-rtc-time
bhyvectl --vm=vm --set-rtc-time=<unix_time_secs>
bhyvectl --vm=vm --rtc-nvram-offset=<offset> --get-rtc-nvram
bhyvectl --vm=vm --rtc-nvram-offset=<offset> --set-rtc-nvram=<value>

Reviewed by: tychon
Discussed with: grehan
Differential Revision: https://reviews.freebsd.org/D1385
MFC after: 2 weeks

Convert usr.sbin to LIBADD
Reduce overlinking

Fix improper .Fx macro usage.

Differential Revision: https://reviews.freebsd.org/D1158
Reviewed by: wblock@
MFC after: 1 month
Sponsored by: The FreeBSD Foundation

To allow a request to be submitted from within the callback routine of
a completing one increase the total by 1 but don't advertise it.

Reviewed by: grehan

Improve the ability to cancel an in-flight request by using an
interrupt, via SIGCONT, to force the read or write system call to
return prematurely.

Reviewed by: grehan

If the start bit, PxCMD.ST, is cleared and nothing is in-flight then
PxCI, PxSACT, PxCMD.CCS and PxCMD.CR should be 0.

Reviewed by: grehan

Add a comment explaining the intent behind the I/O reservation [0x72-0x77].

Move the ACPI PM timer emulation into vmm.ko.

This reduces variability during timer calibration by keeping the emulation
"close" to the guest. Additionally having all timer emulations in the kernel
will ease the transition to a per-VM clock source (as opposed to using the
host's uptime keep track of time).

Discussed with: grehan

Merge projects/bhyve_svm into HEAD.

After this change bhyve supports AMD processors with the SVM/AMD-V hardware
extensions.

More details available here:
https://lists.freebsd.org/pipermail/freebsd-virtualization/2014-October/002905.html

Submitted by: Anish Gupta (akgupt3@gmail.com)
Tested by: Benjamin Perrault (ben.perrault@gmail.com)
Tested by: Willem Jan Withagen (wjw@digiware.nl)

Support stopping and restarting the AHCI command list via toggling
PxCMD.ST from '1' to '0' and back. This allows the driver a chance to
recover if for instance a timeout occurred due to activity on the
host.

Reviewed by: grehan

Support Intel-specific MSRs that are accessed when booting up a linux in bhyve:
- MSR_PLATFORM_INFO
- MSR_TURBO_RATIO_LIMITx
- MSR_RAPL_POWER_UNIT

Reviewed by: grehan
MFC after: 1 week

iasl(8) expects integer fields in data tables to be specified as hexadecimal
values. Therefore the bit width of the "PM Timer Block" was actually being
interpreted as 50-bits instead of the expected 32-bit.

This eliminates an error message emitted by a Linux 3.17 guest during boot:
"Invalid length for FADT/PmTimerBlock: 50, using default 32"

Reviewed by: grehan
MFC after: 1 week

Implement the FLUSH operation in the virtio-block emulation.

This gets rid of the following error message during FreeBSD guest bootup:
"vtbd0: hard error cmd=flush fsbn 0"

Reported by: rodrigc
Reviewed by: grehan

Add new fields in the FADT, required by IASL 20140926-64.

The new IASL from the recent acpi-ca import will error out
if it doesn't see these new fields, which were previously
reserved.

Reported by: lme
Reviewed by: neel

Correct display of bhyve SMBIOS UUIDs with dmidecode by bumping the version.

The mixed little/big-endianness of SMBIOS UUIDs was clarified in v2.6
of the SMBIOS spec. dmidecode uses the reported version of SMBIOS to
determine the layout and what to byte-swap.

bhyve's SMBIOS reported as 2.4 though it implemented the 2.6-style of
memory layout. This resulted in dmidecode reporting a different
UUID than one passed in via the -U option.

Fix by exporting a version of 2.6.

Reviewed by: tychon
Reported by: julian
MFC after: 1 day

Restructure the MSR handling so it is entirely handled by processor-specific
code. There are only a handful of MSRs common between the two so there isn't
too much duplicate functionality.

The VT-x code has the following types of MSRs:

- MSRs that are unconditionally saved/restored on every guest/host context
switch (e.g., MSR_GSBASE).

- MSRs that are restored to guest values on entry to vmx_run() and saved
before returning. This is an optimization for MSRs that are not used in
host kernel context (e.g., MSR_KGSBASE).

- MSRs that are emulated and every access by the guest causes a trap into
the hypervisor (e.g., MSR_IA32_MISC_ENABLE).

Reviewed by: grehan

Update the bhyve(8) manual to reflect that it is no
longer considered 'experimental.'

Reviewed by: grehan
MFC after: 3 days
Sponsored by: The FreeBSD Foundation

Initialize 'bc_rdonly' to the right value.

Note that independent of this change a readonly disk file would still be
opened O_RDONLY and protected from writes by the guest.

Reviewed by: grehan

Allow vtnet operation without merged rx buffers.

NetBSD's virtio-net implementation doesn't negotiate
the merged rx-buffers feature. To support this, check
to see if the feature was negotiated, and then adjust
the operation of the receive path accordingly by using
a larger iovec, and a smaller rx header.
In addition, ignore writes to the (read-only) status byte.

Tested with NetBSD/amd64 5.2.2, 6.1.4 and 7-beta.

Reviewed by: neel, tychon
Phabric: D745
MFC after: 3 days

Add a callback to be notified about negotiated features.

Submitted by: luigi
Obtained from: Vincenzo Maffione, Universita` di Pisa
MFC after: 3 days

Set the 'inst_length' to '0' early on before any error conditions are detected
in the emulation of the task switch. If any exceptions are triggered then the
guest %rip should point to instruction that caused the task switch as opposed
to the one after it.

Fix a recursive lock acquisition in vi_reset_dev().

Reviewed by: grehan

Minor cleanup:
- Set 'pirq_cold' to '0' on the first PIRQ allocation.
- Make assertions stronger.

Reviewed by: jhb
CR: https://phabric.freebsd.org/D592

Support PCI extended config space in bhyve.

Add the ACPI MCFG table to advertise the extended config memory window.

Introduce a new flag MEM_F_IMMUTABLE for memory ranges that cannot be deleted
or moved in the guest's address space. The PCI extended config space is an
example of an immutable memory range.

Add emulation for the "movzw" instruction. This instruction is used by FreeBSD
to read a 16-bit extended config space register.

CR: https://phabric.freebsd.org/D505
Reviewed by: jhb, grehan
Requested by: tychon

Commands which encounter a fatal error shouldn't be marked as completed.
Furthermore, provide an indication of the current command so it can be
determined which one actually failed.

Reviewed by: grehan

Simplify the meaning of return values from the inout handlers. After this
change 0 means success and non-zero means failure.

This also helps to eliminate VMEXIT_POWEROFF and VMEXIT_RESET as return values
from VM-exit handlers.

CR: D480
Reviewed by: grehan, jhb

Reduce the proliferation of VMEXIT_RESTART in task_switch.c.

This is in preparation for further simplification of the return values from
VM exit handlers in bhyve(8).

Fix fault injection in bhyve.

The faulting instruction needs to be restarted when the exception handler
is done handling the fault. bhyve now does this correctly by setting
'vmexit[vcpu].inst_length' to zero so the %rip is not advanced.

A minor complication is that the fault injection APIs are used by instruction
emulation code that is shared by vmm.ko and bhyve. Thus the argument that
refers to 'struct vm *' in kernel or 'struct vmctx *' in userspace needs to
be loosely typed as a 'void *'.

Emulate instructions emitted by OpenBSD/i386 version 5.5:
- CMP REG, r/m
- MOV AX/EAX/RAX, moffset
- MOV moffset, AX/EAX/RAX
- PUSH r/m

Handle nested exceptions in bhyve.

A nested exception condition arises when a second exception is triggered while
delivering the first exception. Most nested exceptions can be handled serially
but some are converted into a double fault. If an exception is generated during
delivery of a double fault then the virtual machine shuts down as a result of
a triple fault.

vm_exit_intinfo() is used to record that a VM-exit happened while an event was
being delivered through the IDT. If an exception is triggered while handling
the VM-exit it will be treated like a nested exception.

vm_entry_intinfo() is used by processor-specific code to get the event to be
injected into the guest on the next VM-entry. This function is responsible for
deciding the disposition of nested exceptions.

Add emulation for legacy x86 task switching mechanism.

FreeBSD/i386 uses task switching to handle double fault exceptions and this
change enables that to work.

Reported by: glebius

Use the blockif CHS routine to create fake CHS values,
and then populate them in the identity page.

This fixes a divide-by-zero error at probe time with NetBSD.

MFC after: 1 week.

Add a call to synthesize a C/H/S value for block emulations
that require it (ahci). The algorithm used is from the VHD
specification.

Extend capabilities to 64-bits in preparation for some API changes.
The v1.0 virtio spec supports an extended size for guest/host
caps, but in practice 64-bits should last for a long time.

Use correct flag for event index.

Submitted by: luigi
Obtained from: Vincenzo Maffione, Universita` di Pisa
MFC after: 1 week

Add post-mortem debugging for "EPT Misconfiguration" VM-exit. This error
is hard to reproduce so try to collect all the breadcrumbs when it happens.

Reviewed by: grehan

Sort command flags in usage output and the manpages.

Set the version and date to fixed fields rather than using
preprocessor macros that don't allow reproducible builds.
As a side-effect, the date string is now spec-compliant.

root@bhyve:~ # dmidecode
# dmidecode 2.12
SMBIOS 2.4 present.
12 structures occupying 514 bytes.
Table at 0x000F101F.

Handle 0x0001, DMI type 0, 24 bytes
BIOS Information
Vendor: BHYVE
Version: 1.0
Release Date: 03/14/2014

Submitted by: des (original version)
Reviewed by: tychon
MFC after: 1 week

- Document -b to enable the bvmcons console (but mark it as deprecated
similar to -g.)
- Document -U to set the SMBIOS UUID.
- Add missing options to the usage output and to the manpage Synopsis.
- Don't claim that bvmdebug is amd64-only (it is also a device, not an
option).

Provide APIs to directly get 'lowmem' and 'highmem' size directly.

Previously the sizes were inferred indirectly based on the size of the mappings
at 0 and 4GB respectively. This works fine as long as size of the allocation is
identical to the size of the mapping in the guest's address space. However, if
the mapping is disjoint then this assumption falls apart (e.g., due to the
legacy BIOS hole between 640KB and 1MB).

use .Mt to mark up email addresses consistently (part2)

PR: 191174
Submitted by: Franco Fichtner <franco@lastsummer.de>

Fix typo and rename macro KDB_SYS_FLAG to KBD_SYS_FLAG.

Reviewed by: tychon

r267169 should apply to 64-bit BARs as well.

Reviewed by: neel

Remove blank lines.

Some devices (e.g. Intel AHCI and NICs) support quad-word access to
register pairs where two 32-bit registers make up a larger logical
size. Support those access by splitting the quad-word into two
double-words.

Reviewed by: grehan

Use MIN(a,b) from <sys/param.h> instead of rolling our own version.

Pointed out by: grehan

Limit the maximum number of back-to-back iterations of a "rep; ins/outs"
to 16. This is arbitrary and is used to ensure that a vcpu goes back into
the vm_run() loop to process interrupts or rendezvous events in a timely
fashion.

Found with: Coverity Scan
CID: 1216436

Activate vcpus from bhyve(8) using the ioctl VM_ACTIVATE_CPU instead of doing
it implicitly in vmm.ko.

Add ioctl VM_GET_CPUS to get the current set of 'active' and 'suspended' cpus
and display them via /usr/sbin/bhyvectl using the "--get-active-cpus" and
"--get-suspended-cpus" options.

This is in preparation for being able to reset virtual machine state without
having to destroy and recreate it.

Add segment protection and limits violation checks in vie_calculate_gla()
for 32-bit x86 guests.

Tested using ins/outs executed in a FreeBSD/i386 guest.

Fix issue with restarting an "insb/insw/insl" instruction because of a page
fault on the destination buffer.

Prior to this change a page fault would be detected in vm_copyout(). This
was done after the I/O port access was done. If the I/O port access had
side-effects (e.g. reading the uart FIFO) then restarting the instruction
would result in incorrect behavior.

Fix this by validating the guest linear address before doing the I/O port
emulation. If the validation results in a page fault exception being injected
into the guest then the instruction can now be restarted without any
side-effects.

Do the linear address calculation for the ins/outs emulation using a new
API function 'vie_calculate_gla()'.

While the current implementation is simplistic it forms the basis of doing
segmentation checks if the guest is in 32-bit protected mode.

Add libvmmapi functions vm_copyin() and vm_copyout() to copy into and out
of the guest linear address space. These APIs in turn use a new ioctl
'VM_GLA2GPA' to convert the guest linear address to guest physical.

Use the new copyin/copyout APIs when emulating ins/outs instruction in
bhyve(8).

Consolidate all the information needed by the guest page table walker into
'struct vm_guest_paging'.

Check for canonical addressing in vmm_gla2gpa() and inject a protection
fault into the guest if a violation is detected.

If the page table walk is restarted in vmm_gla2gpa() then reset 'ptpphys' to
point to the root of the page tables.

Check for alignment check violation when processing in/out string instructions.

Add emulation of the "outsb" instruction. NetBSD guests use this to write to
the UART FIFO.

The emulation is constrained in a number of ways: 64-bit only, doesn't check
for all exception conditions, limited to i/o ports emulated in userspace.

Some of these constraints will be relaxed in followup commits.

Requested by: grehan
Reviewed by: tychon (partially and a much earlier version)

Implement a PCI interrupt router to route PCI legacy INTx interrupts to
the legacy 8259A PICs.
- Implement an ICH-comptabile PCI interrupt router on the lpc device with
8 steerable pins configured via config space access to byte-wide
registers at 0x60-63 and 0x68-6b.
- For each configured PCI INTx interrupt, route it to both an I/O APIC
pin and a PCI interrupt router pin. When a PCI INTx interrupt is
asserted, ensure that both pins are asserted.
- Provide an initial routing of PCI interrupt router (PIRQ) pins to
8259A pins (ISA IRQs) and initialize the interrupt line config register
for the corresponding PCI function with the ISA IRQ as this matches
existing hardware.
- Add a global _PIC method for OSPM to select the desired interrupt routing
configuration.
- Update the _PRT methods for PCI bridges to provide both APIC and legacy
PRT tables and return the appropriate table based on the configured
routing configuration. Note that if the lpc device is not configured, no
routing information is provided.
- When the lpc device is enabled, provide ACPI PCI link devices corresponding
to each PIRQ pin.
- Add a VMM ioctl to adjust the trigger mode (edge vs level) for 8259A
pins via the ELCR.
- Mark the power management SCI as level triggered.
- Don't hardcode the number of elements in Packages in the source for
the DSDT. iasl(8) will fill in the actual number of elements, and
this makes it simpler to generate a Package with a variable number of
elements.

Reviewed by: tycho

Don't include the guest memory segments in the bhyve(8) process core dump.
This has not added a lot of value when debugging bhyve issues while greatly
increasing the time and space required to store the core file.

Passing the "-C" option to bhyve(8) will change the default and dump guest
memory in the core dump.

Requested by: grehan
Reviewed by: grehan

abort(3) the process in response to a VMEXIT_ABORT. This usually happens in
response to an unhandled VM exit or an unexpected error so a core is useful.

Remove unused macro VMEXIT_SWITCH.

Reviewed by: grehan

Disable the 'uart_drain()' callback when the emulated receive FIFO is full.

Failing to do this will cause the kevent(2) notification to trigger
continuously and the bhyve(8) mevent thread will hog the cpu until the
characters on the backend tty device are drained.

Also, make the uart backend file descriptor non-blocking to avoid a
select(2) before every byte read from that backend.

Reviewed by: grehan

Modify the "-p" option to be more flexible when associating a 'vcpu' with
a 'hostcpu'. The new format of the argument string is "vcpu:hostcpu".

This allows pinning a subset of the vcpus if desired.

It also allows pinning a vcpu to more than a single 'hostcpu'.

Submitted by: novel (initial version)

Remove misleading "addcpu" in an error message emitted by fbsdrun_deletecpu().

Pointed out by: novel

Re-adding an event to a kqueue modifies the parameters of the original event.
However, if the original knote had been disabled then it is not automatically
re-enabled.

Fix this by using EV_ADD to create an mevent and EV_ENABLE to enable it.

Adding a kevent for the first time implicitly enables it so existing callers
of mevent_add() don't need to change.

Reviewed by: grehan

Don't allow MPtable generation if there are multiple PCI hierarchies. This is
because there isn't a standard way to relay this information to the guest OS.

Add a command line option "-Y" to bhyve(8) to inhibit MPtable generation.

If the virtual machine is using PCI devices on buses other than 0 then it can
still use ACPI tables to convey this information to the guest.

Discussed with: grehan@

Add logic in the HLT exit handler to detect if the guest has put all vcpus
to sleep permanently by executing a HLT with interrupts disabled.

When this condition is detected the guest with be suspended with a reason of
VM_SUSPEND_HALT and the bhyve(8) process will exit.

Tested by executing "halt" inside a RHEL7-beta guest.

Discussed with: grehan@
Reviewed by: jhb@, tychon@

Ignore writes to microcode update MSR. This MSR is accessed by RHEL7 guest.
Add KTR tracepoints to annotate wrmsr and rdmsr VM exits.

Some Linux guests will implement a 'halt' by disabling the APIC and executing
the 'HLT' instruction. This condition was detected by 'vm_handle_hlt()' and
converted into the SPINDOWN_CPU exitcode . The bhyve(8) process would exit
the vcpu thread in response to a SPINDOWN_CPU and when the last vcpu was
spun down it would reset the virtual machine via vm_suspend(VM_SUSPEND_RESET).

This functionality was broken in r263780 in a way that made it impossible
to kill the bhyve(8) process because it would loop forever in
vm_handle_suspend().

Unbreak this by removing the code to spindown vcpus. Thus a 'halt' from
a Linux guest will appear to be hung but this is consistent with the
behavior on bare metal. The guest can be rebooted by using the bhyvectl
options '--force-reset' or '--force-poweroff'.

Reviewed by: grehan@

Allow a virtual machine to be forcibly reset or powered off. This is done
by adding an argument to the VM_SUSPEND ioctl that specifies how the virtual
machine should be suspended, viz. VM_SUSPEND_RESET or VM_SUSPEND_POWEROFF.

The disposition of VM_SUSPEND is also made available to the exit handler
via the 'u.suspended' member of 'struct vm_exit'.

This capability is exposed via the '--force-reset' and '--force-poweroff'
arguments to /usr/sbin/bhyvectl.

Discussed with: grehan@

Implement legacy interrupts for the AHCI device emulation
according to the method outlined in the AHCI spec.

Tested with FreeBSD 9/10/11 with MSI disabled,
and also NetBSD/amd64 (lightly).

Reviewed by: neel, tychon
MFC after: 3 weeks

Respect and track the enable bit in the PCI configuration address word.
Ignore writes, and return 0xff's, on config accesses when not set.
Behaviour now matches that seen on h/w.

Found with a NetBSD/amd64 guest.

Reviewed by: tychon
MFC after: 3 weeks

Provide a very basic stub for the 8042 PS/2 keyboard controller.

Reviewed by: jhb
Approved by: neel (co-mentor)

Use calloc() in favor of malloc + memset.

Reviewed by: neel

Factor out common ioport handler code for better hygiene -- pointed
out by neel@.

Approved by: neel (co-mentor)

Fix ACPI DSDT indentation cosmetic breakage introduced in r264631 --
pointed out by jhb@.

Approved by: grehan (co-mentor)

Respect the destination operand size of the 'Input from Port' instruction.

Approved by: grehan (co-mentor)

Add support for reading the PIT Counter 2 output signal via the NMI
Status and Control register at port 0x61.

Be more conservative about "catching up" callouts that were supposed
to fire in the past by skipping an interrupt if it was
scheduled too far in the past.

Restore the PIT ACPI DSDT entries and add an entry for NMISC too.

Approved by: neel (co-mentor)

Add support for emulating the slave PIC.

Reviewed by: grehan, jhb
Approved by: grehan (co-mentor)

Constrain the amount of data returned to what is actually available
not the size of the buffer.

Approved by: grehan (co-mentor)

Handle single-byte reads from the bvmcons port (0x220) by returning
0xff. Some guests may attempt to read from this port to identify
psuedo-PNP ISA devices. (The ie(4) driver in FreeBSD/i386 is one
example.)

Reviewed by: grehan

Add support for the virtio RNG entropy-source device.
Call through to /dev/random synchronously to fill
virtio buffers with RNG data.

Tested with FreeBSD-CURRENT and Ubuntu guests.

Submitted by: Leon Dang
Discussed with: markm
MFC after: 3 weeks
Sponsored by: Nahanni Systems

Add an ioctl to suspend a virtual machine (VM_SUSPEND). The ioctl can be called
from any context i.e., it is not required to be called from a vcpu thread. The
ioctl simply sets a state variable 'vm->suspend' to '1' and returns.

The vcpus inspect 'vm->suspend' in the run loop and if it is set to '1' the
vcpu breaks out of the loop with a reason of 'VM_EXITCODE_SUSPENDED'. The
suspend handler waits until all 'vm->active_cpus' have transitioned to
'vm->suspended_cpus' before returning to userspace.

Discussed with: grehan

Move the atpit device model from userspace into vmm.ko for better
precision and lower latency.

Approved by: grehan (co-mentor)

Use 'cpuset_t' to represent the vcpus active in a virtual machine.

Don't reissue in-flight commands.

Approved by: neel (co-mentor)

Though there currently isn't a way to insert new media into an ATAPI
drive, at least pretend to support Asynchronous Notification (AN) to
avoid a guest needlessly polling for it.

Approved by: grehan (co-mentor)

Support the bootloader's single 16-bit 'outw' access to the Divisor
Latch MSB and LSB registers.

Approved by: neel (co-mentor)

Replace the userspace atpic stub with a more functional vmm.ko model.

New ioctls VM_ISA_ASSERT_IRQ, VM_ISA_DEASSERT_IRQ and VM_ISA_PULSE_IRQ
can be used to manipulate the pic, and optionally the ioapic, pin state.

Reviewed by: jhb, neel
Approved by: neel (co-mentor)

Open the uart emulation's backing tty in non-blocking mode.
This fixes the issue of bhyve appearing to halt when using
nmdm ports for the console, until a connection is made to
the other end.

bhyveload already does this.

Reported by: Many.
MFC after: 3 weeks.

Add SMBIOS support.

A new option, -U, can be used to set the UUID in the System
Information (Type 1) structure. Manpage fix to follow.

Approved by: grehan (co-mentor)

Merge the projects/clang-sparc64 branch back to head. This brings in
several updates from the llvm and clang trunks to make the sparc64
backend fully functional.

Apart from one patch to sys/sparc64/include/pcpu.h which is still under
discussion, this makes it possible to let clang fully build world and
kernel for sparc64.

Any assistance with testing this on actual sparc64 hardware is greatly
appreciated, as there will unavoidably be bugs left.

Many thanks go to Roman Divacky for his upstream work on getting the
sparc64 backend into shape.

MFC after: 1 month

Document the "-a" and "-x" options to match the changes in r262236.

Reviewed by: grehan

Queue pending exceptions in the 'struct vcpu' instead of directly updating the
processor-specific VMCS or VMCB. The pending exception will be delivered right
before entering the guest.

The order of event injection into the guest is:
- hardware exception
- NMI
- maskable interrupt

In the Intel VT-x case, a pending NMI or interrupt will enable the interrupt
window-exiting and inject it as soon as possible after the hardware exception
is injected. Also since interrupts are inherently asynchronous, injecting
them after the hardware exception should not affect correctness from the
guest perspective.

Rename the unused ioctl VM_INJECT_EVENT to VM_INJECT_EXCEPTION and restrict
it to only deliver x86 hardware exceptions. This new ioctl is now used to
inject a protection fault when the guest accesses an unimplemented MSR.

Discussed with: grehan, jhb
Reviewed by: jhb

Fix virtio spec URL.

Submitted by: lwhsu
MFC after: 1 week

Avoid clobbering the counter mode when issuing a latch command.

Approved by: grehan (co-mentor)

Simplify APIC mode switching from MMIO to x2APIC. In part this is done to
simplify the implementation of the x2APIC virtualization assist in VT-x.

Prior to this change the vlapic allowed the guest to change its mode from
xAPIC to x2APIC. We don't allow that any more and the vlapic mode is locked
when the virtual machine is created. This is not very constraining because
operating systems already have to deal with BIOS setting up the APIC in
x2APIC mode at boot.

Fix a bug in the CPUID emulation where the x2APIC capability was leaking
from the host to the guest.

Ignore MMIO reads and writes to the vlapic in x2APIC mode. Similarly, ignore
MSR accesses to the vlapic when it is in xAPIC mode.

The default configuration of the vlapic is xAPIC. The "-x" option to bhyve(8)
can be used to change the mode to x2APIC instead.

Discussed with: grehan@

Add a check to validate that memory BARs of passthru devices are 4KB aligned.

Also, the MSI-x table offset is not required to be 4KB aligned so take this
into account when computing the pages occupied by the MSI-x tables.

Tweak the handling of PCI capabilities in emulated devices to remove
the non-standard zero capability list terminator. Instead, track
the start and end of the most recently added capability and use that
to adjust the previous capability's next pointer when a capability is
added and to determine the range of config registers belonging to
PCI capability registers.

Reviewed by: neel

Update bhyve(8) man page to describe the usage of the "-s" option to assign
bus numbers to emulated devices. Also add the restriction that the LPC bridge
emulation can only be configured on bus 0.

Reviewed by: grehan@

Allow PCI devices to be configured on all valid bus numbers from 0 to 255.

This is done by representing each bus as root PCI device in ACPI. The device
implements the _BBN method to return the PCI bus number to the guest OS.

Each PCI bus keeps track of the resources that is decodes for devices
configured on the bus: i/o, mmio (32-bit) and mmio (64-bit). These windows
are advertised to the guest via the _CRS object of the root device.

Bus 0 is treated specially since it consumes the I/O ports to access the
PCI config space [0xcf8-0xcff]. It also decodes the legacy I/O ports that
are consumed by devices on the LPC bus. For this reason the LPC bridge can
be configured only on bus 0.

The bus number can be specified using the following command line option
to bhyve(8): "-s <bus>:<slot>:<func>,<emul>[,<config>]"

Discussed with: grehan@
Reviewed by: jhb@

Provide an indication a "PIO Setup Device to Host FIS" occurred while executing
the IDENTIFY DEVICE and IDENTIFY PACKET DEVICE commands.

Also, provide an indication a "D2H Register FIS" occurred during a SET FEATURES
command.

Approved by: grehan (co-mentor)

Mark the I/O ports used by the bhyve console and debug devices as system
resources.

MFC after: 1 week

Enhance the support for PCI legacy INTx interrupts and enable them in
the virtio backends.
- Add a new ioctl to export the count of pins on the I/O APIC from vmm
to the hypervisor.
- Use pins on the I/O APIC >= 16 for PCI interrupts leaving 0-15 for
ISA interrupts.
- Populate the MP Table with I/O interrupt entries for any PCI INTx
interrupts.
- Create a _PRT table under the PCI root bridge in ACPI to route any
PCI INTx interrupts appropriately.
- Track which INTx interrupts are in use per-slot so that functions
that share a slot attempt to distribute their INTx interrupts across
the four available pins.
- Implicitly mask INTx interrupts if either MSI or MSI-X is enabled
and when the INTx DIS bit is set in a function's PCI command register.
Either assert or deassert the associated I/O APIC pin when the
state of one of those conditions changes.
- Add INTx support to the virtio backends.
- Always advertise the MSI capability in the virtio backends.

Submitted by: neel (7)
Reviewed by: neel
MFC after: 2 weeks

Remove support for legacy PCI devices. These haven't been needed since
support for LPC uart devices was added and it conflicts with upcoming
patches to add PCI INTx support.

Reviewed by: neel

Fix issue with stale fields from a recycled request pulled off the freelist.

Approved by: grehan (co-mentor)

Increase the block-layer backend maximum number of requests to match
the AHCI command queue depth. This allows a slew of commands issued
by a Linux guest to be absorbed without error.

Approved by: grehan (co-mentor)

Fix issue with the virtio descriptor region being truncated
if it was above 4GB. This was seen with CentOS 6.5 guests with
large RAM, since the block drivers are loaded late in the
boot sequence and end up allocating descriptor memory from
high addresses.

Reported by: Michael Dexter
MFC after: 3 days

virtio-block does not exist, the correct name is virtio-blk.

PR: 185573
Submitted by: Allan Jude
Facilitated by: Snow B.V.
MFC after: 3 days

Cosmetic change - switch over to vertical SRCS to make it
easier to keep files in alpha order.

Reviewed by: neel

Rework the DSDT generation code a bit to generate more accurate info about
LPC devices. Among other things, the LPC serial ports now appear as
ACPI devices.
- Move the info for the top-level PCI bus into the PCI emulation code and
add ResourceProducer entries for the memory ranges decoded by the bus
for memory BARs.
- Add a framework to allow each PCI emulation driver to optionally write
an entry into the DSDT under the \_SB_.PCI0 namespace. The LPC driver
uses this to write a node for the LPC bus (\_SB_.PCI0.ISA).
- Add a linker set to allow any LPC devices to write entries into the
DSDT below the LPC node.
- Move the existing DSDT block for the RTC to the RTC driver.
- Add DSDT nodes for the AT PIC, the 8254 ISA timer, and the LPC UART
devices.
- Add a "SuperIO" device under the LPC node to claim "system resources"
aling with a linker set to allow various drivers to add IO or memory
ranges that should be claimed as a system resource.
- Add system resource entries for the extended RTC IO range, the registers
used for ACPI power management, the ELCR, PCI interrupt routing register,
and post data register.
- Add various helper routines for generating DSDT entries.

Reviewed by: neel (earlier version)

Restructure the VMX code to enter and exit the guest. In large part this change
hides the setjmp/longjmp semantics of VM enter/exit. vmx_enter_guest() is used
to enter guest context and vmx_exit_guest() is used to transition back into
host context.

Fix a longstanding race where a vcpu interrupt notification might be ignored
if it happens after vmx_inject_interrupts() but before host interrupts are
disabled in vmx_resume/vmx_launch. We now called vmx_inject_interrupts() with
host interrupts disabled to prevent this.

Suggested by: grehan@

Extend the ACPI power management support to wire a virtual power button up
to SIGTERM when ACPI is enabled. Sending SIGTERM to the hypervisor when an
ACPI-aware OS is running will now trigger a soft-off allowing for a graceful
shutdown of the guest.
- Move constants for ACPI-related registers to acpi.h.
- Implement an SMI_CMD register with commands to enable and disable ACPI.
Currently the only change when ACPI is enabled is to enable the virtual
power button via SIGTERM.
- Implement a fixed-feature power button when ACPI is enabled by asserting
PWRBTN_STS in PM1_EVT when SIGTERM is received.
- Add support for EVFILT_SIGNAL events to mevent.
- Implement support for the ACPI system command interrupt (SCI) and assert
it when needed based on the values in PM1_EVT. Mark the SCI as active-low
and level triggered in the MADT and MP Table.
- Mark PCI interrupts in the MP Table as active-low in addition to level
triggered.

Reviewed by: neel

Use pthread_once() to replace a static integer initted flag.

Reviewed by: neel

Support soft power-off via the ACPI S5 state for bhyve guests.
- Implement the PM1_EVT and PM1_CTL registers required by ACPI.
The PM1_EVT register is mostly a dummy as bhyve doesn't support any
of the hardware-initiated events. The only bit of PM1_CNT that is
implemented are the sleep request bits (SPL_EN and SLP_TYP) which
request a graceful power off for S5. In particular, for S5, bhyve
exits with a non-zero value which terminates the loop in vmrun.sh.
- Emulate the Reset Control register at I/O port 0xcf9 and advertise
it as the reset register via ACPI.
- Advertise an _S5 package.
- Extend the in/out interface to allow an in/out handler to request
that the hypervisor trigger a reset or power-off.
- While here, note that all vCPUs in a guest support C1 ("hlt").

Reviewed by: neel (earlier version)

Extend the support for local interrupts on the local APIC:
- Add a generic routine to trigger an LVT interrupt that supports both
fixed and NMI delivery modes.
- Add an ioctl and bhyvectl command to trigger local interrupts inside a
guest. In particular, a global NMI similar to that raised by SERR# or
PERR# can be simulated by asserting LINT1 on all vCPUs.
- Extend the LVT table in the vCPU local APIC to support CMCI.
- Flesh out the local APIC error reporting a bit to cache errors and
report them via ESR when ESR is written to. Add support for asserting
the error LVT when an error occurs. Raise illegal vector errors when
attempting to signal an invalid vector for an interrupt or when sending
an IPI.
- Ignore writes to reserved bits in LVT entries.
- Export table entries the MADT and MP Table advertising the stock x86
config of LINT0 set to ExtInt and LINT1 wired to NMI.

Reviewed by: neel (earlier version)

mdoc: nuke whitespace.

Add a parameter to 'vcpu_set_state()' to enforce that the vcpu is in the IDLE
state before the requested state transition. This guarantees that there is
exactly one ioctl() operating on a vcpu at any point in time and prevents
unintended state transitions.

More details available here:
http://lists.freebsd.org/pipermail/freebsd-virtualization/2013-December/001825.html

Reviewed by: grehan
Reported by: Markiyan Kushnir (markiyan.kushnir at gmail.com)
MFC after: 3 days

Add an option to ignore accesses by the guest to unimplemented MSRs.

Also, ignore a couple of SandyBridge uncore PMC MSRs that Centos 6.4 writes
to during boot.

Reviewed by: grehan

Rename the ambiguously named 'vm_setup_msi()' and 'vm_setup_msix()' to
'vm_setup_pptdev_msi()' and 'vm_setup_pptdev_msix()' respectively.

It should now be clear that these functions operate on passthru devices.

Add an API to deliver message signalled interrupts to vcpus. This allows
callers treat the MSI 'addr' and 'data' fields as opaque and also lets
bhyve implement multiple destination modes: physical, flat and clustered.

Submitted by: Tycho Nightingale (tycho.nightingale@pluribusnetworks.com)
Reviewed by: grehan@

mdoc: sort SEE ALSO.

bhyve(8) man page.

mdoc formatting and much input and review from Warren Block (wblock@).

Reviewed by: many
MFC after: 3 days

If a vcpu disables its local apic and then executes a 'HLT' then spin down the
vcpu and destroy its thread context. Also modify the 'HLT' processing to ignore
pending interrupts in the IRR if interrupts have been disabled by the guest.
The interrupt cannot be injected into the guest in any case so resuming it
is futile.

With this change "halt" from a Linux guest works correctly.

Reviewed by: grehan@
Tested by: Tycho Nightingale (tycho.nightingale@pluribusnetworks.com)

Fix the processor table entry structure to use a fixed-width type for
32-bit fields so it is the correct size on amd64. Remove a workaround
for the broken structure from bhyve(8).

MFC after: 1 week

Add support for level triggered interrupt pins on the vioapic. Prior to this
commit level triggered interrupts would work as long as the pin was not shared
among multiple interrupt sources.

The vlapic now keeps track of level triggered interrupts in the trigger mode
register and will forward the EOI for a level triggered interrupt to the
vioapic. The vioapic in turn uses the EOI to sample the level on the pin and
re-inject the vector if the pin is still asserted.

The vhpet is the first consumer of level triggered interrupts and advertises
that it can generate interrupts on pins 20 through 23 of the vioapic.

Discussed with: grehan@

Allow bhyve and bhyveload to attach to tty devices.

bhyveload: introduce the -c <device> parameter
to select a tty for output (or "stdio")

bhyve: allow the puc and lpc-com backends to
accept a tty in addition to "stdio"

When used in conjunction with the null-modem device,
nmdm(4), this allows attach/detach to the guest console
and multiple concurrent serial ports. kgdb on a serial
port is now functional.

Reviewed by: neel
Requested by: Almost everyone that has used bhyve
MFC after: 10.0

The Data Byte Count (DBC) field of a Physical Region Descriptor
Table is 22 bits, with the bit 31 being the interrupt-on-completion
bit.

OpenBSD and UEFI set this bit, resulting in large block i/o lengths
being sent to bhyve and coredumping the process. Fix by masking off
the relevant 22 bits when using the DBC field as a length.

Reviewed by: Zhixiang Yu
Discussed with: Tycho Nightingale (tycho.nightingale@pluribusnetworks.com)
MFC after: 10.0

Fix discrepancy between the IOAPIC ID advertised by firmware tables and the
actual value read by the guest from the device. The IOAPIC ID is now set to
zero in both MPtable/ACPI tables as well as in the ioapic device emulation.

Pointed out by: grehan@

Add HPET device emulation to bhyve.

bhyve supports a single timer block with 8 timers. The timers are all 32-bit
and capable of being operated in periodic mode. All timers support interrupt
delivery using MSI. Timers 0 and 1 also support legacy interrupt routing.

At the moment the timers are not connected to any ioapic pins but that will
be addressed in a subsequent commit.

This change is based on a patch from Tycho Nightingale (tycho.nightingale@pluribusnetworks.com).

Add an ioctl to assert and deassert an ioapic pin atomically. This will be used
to inject edge triggered legacy interrupts into the guest.

Start using the new API in device models that use edge triggered interrupts:
viz. the 8254 timer and the LPC/uart device emulation.

Submitted by: Tycho Nightingale (tycho.nightingale@pluribusnetworks.com)