Revert r330897:

This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.

Revert with prejudice.

This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.

Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.

Requested by: gjb (re)

Partial merge of the SPDX changes

These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.

No objections from: pfg

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Various VirtIO improvements

PCI:
- Properly handle interrupt fallback from MSIX to MSI to legacy.
The host may not have sufficient resources to support MSIX,
so we must be able to fallback to legacy interrupts.
- Add interface to get the (sub) vendor and device IDs.
- Rename flags to VTPCI_FLAG_* like other VirtIO drivers.
Block:
- No longer allocate vtblk_requests from separate UMA zone.
malloc(9) from M_DEVBUF is sufficient. Assert segment counts
at allocation.
- More verbose error and debug messages.
Network:
- Remove stray write once variable.
Virtqueue:
- Shuffle code around in preparation of converting the mb()s to
the appropriate atomic(9) operations.
- Only walk the descriptor chain when freeing if INVARIANTS is
defined since the result is only KASSERT()ed.

Submitted by: Bryan Venteicher (bryanv@daemoninthecloset.org)

Do not include <sys/types.h> in the local headers. The .c files including
them have already included <sys/param.h> before these headers are included.

Catch up with Bryan Venteicher's virtio git repo:

a8af6270bd96be6ccd86f70b60fa6512b710e4f0
virtio_blk: Include function name in panic string

cbdb03a694b76c5253d7ae3a59b9995b9afbb67a
virtio_balloon: Do the notify outside of the lock

By the time we return from virtqueue_notify(), the descriptor
will be in the used ring so we shouldn't have to sleep.

10ba392e60692529a5cbc1e9987e4064e0128447
virtio: Use DEVMETHOD_END

80cbcc4d6552cac758be67f0c99c36f23ce62110
virtqueue: Add support for VIRTIO_F_RING_EVENT_IDX

This can be used to reduce the number of guest/host and
host/guest interrupts by delaying the interrupt until a
certain index value is reached.

Actual use by the network driver will come along later.

8fc465969acc0c58477153e4c3530390db436c02
virtqueue: Simplify virtqueue_nused()

Since the values just wrap naturally at UINT16_MAX, we
can just subtract the two values directly, rather than
doing 2's complement math.

a8aa22f25959e2767d006cd621b69050e7ffb0ae
virtio_blk: Remove debugging crud from 75dd732a

There seems to be an issue with Qemu (or FreeBSD VirtIO) that sets
the PCI register space for the device config to bogus values. This
only seems to happen after unloading and reloading the module.

d404800661cb2a9769c033f8a50b2133934501aa
virtio_blk: Use better variable name

75dd732a97743d96e7c63f7ced3c2169696dadd3
virtio_blk: Partially revert 92ba40e65

Just use the virtqueue to determine if any requests are
still inflight.

06661ed66b7a9efaea240f99f414c368f1bbcdc7
virtio_blk: error if allowed too few segments

Should never happen unless the host provides use with a
bogus seg_max value.

4b33e5085bc87a818433d7e664a0a2c8f56a1a89
virtio_blk: Sort function declarations

426b9f5cac892c9c64cc7631966461514f7e08c6
virtio_blk: Cleanup whitespace

617c23e12c61e3c2233d942db713c6b8ff0bd112
virtio_blk: Call disk_err() on error'd completed requests

081a5712d4b2e0abf273be4d26affcf3870263a9
virtio_blk: ASSERT the ready and inflight request queues are empty

a9be2631a4f770a84145c18ee03a3f103bed4ca8
virtio_blk: Simplify check for too many segments

At the cost of a small style violation.

e00ec09da014f2e60cc75542d0ab78898672d521
virtio_blk: Add beginnings of suspend/resume

Still not sure if we need to virtio_stop()/virtio_reinit()
the device before/after a suspend.

Don't start additional IO when marked as suspending.

47c71dc6ce8c238aa59ce8afd4bda5aa294bc884
virtio_blk: Panic when dealt an unhandled BIO cmd

1055544f90fb8c0cc6a2395f5b6104039606aafe
virtio_blk: Add VQ enqueue/dequeue wrappers

Wrapper functions managed the added/removing to the in-flight
list of requests.

Normally biodone() any completed IO when draining the virtqueue.

92ba40e65b3bb5e4acb9300ece711f1ea8f3f7f4
virtio_blk: Add in-flight list of requests

74f6d260e075443544522c0833dc2712dd93f49b
virtio_blk: Rename VTBLK_FLAG_DETACHING to VTBLK_FLAG_DETACH

7aa549050f6fc6551c09c6362ed6b2a0728956ef
virtio_blk: Finish all BIOs through vtblk_finish_bio()

Also properly set bio_resid in the case of errors. Most geom_disk
providers seem to do the same.

9eef6d0e6f7e5dd362f71ba097f2e2e4c3744882
Added function to translate VirtIO status to error code

ef06adc337f31e1129d6d5f26de6d8d1be27bcd2
Reset dumping flag when given unexpected parameters

393b3e390c644193a2e392220dcc6a6c50b212d9
Added missing VTBLK_LOCK() in dump handler

Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org

Catch up with Bryan Venteicher's virtio Hg repo:

c162516
Remove vtblk_sector_size

c162515
Wrap long license lines

c162514
Remove vtblk_unit

c162513
Wrap long lines in the license.

c162512
Remove verbose messages when link goes up/down.

A similar message is printed elsewhere as a result of
if_link_state_change().

c162511
Explicity compare pointer to NULL

c162510
Allocate the mac filter table at attach time.

c162509
Add real BSD licenses to the header files copied from Linux.

The chases upstream changes made in Linux awhile ago.

c162508
Only notify if we actually dequeued something.

c162507
Change a couple of if () { KASSERT(...) } to just KASSERTs.

In non-debug kernels, the if() { } probably get optomized
away, but I guess this is clearer.

c162506
Remove VIRTIO_BLK_F_TOPOLOGY fields in the config.

TOPOLOGY has since been removed from the spec, and the FreeBSD
didn't really do anything with the fields anyways.

c162505
Move vtblk_enqueue_request() outside the locks when getting the ident.

c162504
Remove soon to be uneeded trylock during dump [1].
http://lists.freebsd.org/pipermail/freebsd-current/2011-November/029226.html

c162503
Remove emtpy line

c162502
Drop frame if cannot allocate a vtnet_tx_header.

If we don't, we set OACTIVE, but if there are no
other frames in flight, vtnet_txeof() will never
be called to unset OACTIVE. The interface would
have to be down/up'ed in order to become usable.

We could be cuter here and only do this if the
virtqueue is emtpy, but its probably not worth
the complication.

c162501
Start mbuf replacement loop at 1 for clarity

Obtained from: Bryan Venteicher bryanv at daemoninthecloset dot org

Import virtio base, PCI front-end, and net/block/balloon drivers.
Tested on Qemu/KVM, VirtualBox, and BHyVe.

Currently built as modules-only on i386/amd64. Man pages not yet hooked
up, pending review.

Submitted by: Bryan Venteicher bryanv at daemoninthecloset dot org
Reviewed by: bz
MFC after: 4 weeks or so