MFC r328083,328096,328116,328119,328120,328128,328135,328153,328157,
328166,328177,328199,328202,328205,328468,328470,328624,328625,328627,
328628,329214,329297,329365:

Meltdown mitigation by PTI, PCID optimization of PTI, and kernel use of IBRS
for some mitigations of Spectre.

Tested by: emaste, Arshan Khanifar <arshankhanifar@gmail.com>
Discussed with: jkim
Sponsored by: The FreeBSD Foundation

MFC 324487

hyperv/vmbus: Add tunable to pin/unpin event tasks.

Event tasks are pinned to their respective CPU by default, in the same
fashion as they were.

Unpin the event tasks by setting hw.vmbus.pin_evttask to 0, if certain
CPUs serve special purpose.

Sponsored by: Microsoft

MFC 322488
hyperv: Update copyright for the files changed in 2017

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D11982

MFC 318136

hyperv/vmbus: Reorganize vmbus device tree

For GEN1 Hyper-V, vmbus is attached to pcib0, which contains the
resources for PCI passthrough and SR-IOV. There is no
acpi_syscontainer0 on GEN1 Hyper-V.

For GEN2 Hyper-V, vmbus is attached to acpi_syscontainer0, which
contains the resources for PCI passthrough and SR-IOV. There is
no pcib0 on GEN2 Hyper-V.

The ACPI VMBUS device now only holds its _CRS, which is empty as
of this commit; its existence is mainly for upward compatibility.

Device tree structure is suggested by jhb@.

Tested-by: dexuan@
Collabrated-wth: dexuan@
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D10565

MFC 310651

hyperv/vmbus: Nuke unnecessary critical sections.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8906

MFC 309310,309311,309316,309318

309310
hyperv/hn: Nuke the unused TX taskqueue CPU binding tunable.

It was an experimental tunable, and is now deemed to be road blocker
for further changes. Time to retire it.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8654

309311
hyperv/hn: Allow multiple TX taskqueues.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8655

309316
hyperv/vmbus: Add DEVMETHOD to map cpu to event taskq.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8658

309318
hyperv/hn: Allow TX to share event taskqueues.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8659

MFC 309240,309242,309244,309245,309670

309240
hyperv/vmbus: Add result polling support for xact API.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8633

309242
hyperv/vmbus: Add result polling support for message Hypercall API.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8634

309244
hyperv/vmbus: Add exec cancel support for message Hypercall API.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8635

309245
hyperv/vmbus: Use poll/cancel APIs to wait for the CHOPEN response.

Since hypervisor does not respond CHOPEN to a revoked channel.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8636

309670
hyperv/vmbus: Use pause if possible.

This makes booting on Hyper-V w/ small # of vCPUs work properly.

Reported by: Hongxiong Xian <v-hoxian microsoft com>, Hongjiang Zhang <honzhan microsoft com>
Sponsored by: Microsoft

MFC 309236,309237

309236
hyperv/vmbus: Make sure that the allocated GPADL is not zero.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8631

309237
hyperv/vmbus: Stringent GPADL parameter assertion.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8632

r310348

hyperv: Unbreak EARLY_AP_STARUP Hyper-V bootstrap by using intrhook

Properly working pause and friends are required.

Sponsored by: Microsoft

MFC: 308723-308725,308793-308795,309127

Approved by: sephe (mentor)

r308723
hyperv/vmbus: add a new method to get vcpu_id

vcpu_id is host's representation of guest CPU.
We get the mapping between vcpu_id and FreeBSD kernel's cpu id when VMBus
driver is loaded. Later, when a driver, like the coming pcib driver, talks
to the host and needs to refer to a guest CPU, the driver must use the
vcpu_id.

Reviewed by: jhb, sephe
Approved by: sephe (mentor)
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8410

r308724
hyperv/vmbus: add new vmbus methods to support PCIe pass-through

The new methods will be used by the coming pcib driver.

Reviewed by: sephe
Approved by: sephe (mentor)
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8409

r308725
hyperv/pcib: enable PCIe pass-through (a.k.a. Discrete Device Assignment)

The feature enables us to pass through physical PCIe devices to FreeBSD VM
running on Hyper-V (Windows Server 2016) to get near-native performance with
low CPU utilization.

The patch implements a PCI bridge driver to support the feature:

1) The pcib driver talks to the host to discover device(s) and presents
the device(s) to FreeBSD's pci driver via PCI configuration space (note:
to access the configuration space, we don't use the standard I/O port
0xCF8/CFC method; instead, we use an MMIO-based method supplied by Hyper-V,
which is very similar to the 0xCF8/CFC method).

2) The pcib driver allocates resources for the device(s) and initialize
the related BARs, when the device driver's attach method is invoked;

3) The pcib driver talks to the host to create MSI/MSI-X interrupt
remapping between the guest and the host;

4) The pcib driver supports device hot add/remove.

Reviewed by: sephe
Approved by: sephe (mentor)
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8332

r308793
hyperv/pcib: Fix the build for some kernel configs

Add the dependency on pci explicitly for the pcib and vmbus drivers.
The related Makefiles are updated accordingly too.

Reviewed by: sephe
Approved by: sephe (mentor)
Sponsored by: Microsoft

r308794
hyperv/vmbus,pcib: Add MODULE_DEPEND on pci

We'd better add this dependency explicitly, though usually the pci
driver is built into the kernel by default.

Reviewed by: sephe
Approved by: sephe (mentor)
Sponsored by: Microsoft

r308795
hyperv/pcib: change the file name: pcib.c -> vmbus_pcib.c

This makes the file name and the variable naming in the file consistent.

Reviewed by: sephe
Approved by: sephe (mentor)
Sponsored by: Microsoft

r309127
hyperv/vmbus,pcib: unbreak build in case NEW_PCIB is undefined

vmbus_pcib requires NEW_PCIB, but in case that's not defined, we at
least shouldn't break build.

Reviewed by: sephe
Approved by: sephe (mentor)
Sponsored by: Microsoft

MFC 307952,307953,308278

307952
hyperv/vmbus: Add missing white space.

Submitted by: QianYue You <t-youqi microsoft com>
Sponsored by: Microsoft

307953
hyperv/vmbus: Implement vmbus_chan_printf.

And use it for vmbus channel logging, which can log the channel
owner's name properly, instead of vmbus0.

Submitted by: QianYue You <t-youqi microsoft com>
Sponsored by: Microsoft

308278
hyperv/vmbus: Reset ch_dev, once the child is deleted.

So it will not be mis-used later on, e.g. in vmbus_chan_printf().

Submitted by: dexuan
Reported by: dexuan
Sponsored by: Microsoft

MFC 306360,306387,306389

306360
hyperv/vmbus: Add dynamic device add and remove support

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8008

306387
hyperv/vmbus: Add functions to test RX/TX bufring emptiness

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8044

306389
hyperv/vmbus: Add function to drain channel interrupt task.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8045

MFC 303945,303947-303949,303989,303992,303998,304001,304002,304109,304111

303945
hyperv/vmbus: Add macro to get channel packet data length.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7455

303947
hyperv/vmbus: Add APIs for various types of transactions.

Reviewed by: Jun Su <junsu microsoft com>
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7456

303948
hyperv/hn: Switch to vmbus xact APIs for NVS initialization

Reviewed by: Jun Su <junsu microsoft com>
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7457

303949
hyperv/vmbus: Use xact APIs to implement post message Hypercall APIs

Avoid code duplication.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7458

303989
hyperv/hn: Simplify NDIS configuration.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7466

303992
hyperv/hn: Simplify NDIS initialization.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7467

303998
hyperv/hn: Switch to vmbus xact APIs for NVS RXBUF connection.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7469

304001
hyperv/hn: Switch to vmbus xact APIs for NVS chimney buffer connection.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7470

304002
hyperv/hn: Simplify RXBUF disconnection.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7472

304109
hyperv/hn: Simplify chimney sending buffer disconnection.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7479

304111
hyperv/hn: Switch to vmbus xact APIs for sub-channel alloc request.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7480

MFC 303421,303422,303470-303473

303421
hyperv/vmbus: Avoid unnecessary mb()

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7333

303422
hyperv/vmbus: Inclusion cleanup

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7334

303470
hyperv/vmbus: Reindent function declarations.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7344

303471
hyperv/vmbus: Forward declare static functions

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7346

303472
hyperv/vmbus: Move driver glue to the beginning of the files

Just as most of other drivers do. And move sysinit function close
to its SYSINIT.

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7347

303473
hyperv/vmbus: Revoke unnecessary exposure of vmbus softc

Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D7348

MFC 303283

hyperv/vmbus: Rename hv_vmbus_priv.h to vmbus_brvar.h

It only contains bufring related bits for a while.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7281

MFC 303066-303072,303127-303129,303131

303066
hyperv/vmbus: Get rid of unnecessary definition.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7240

303067
hyperv/vmbus: Move IC register definition to Hyper-V utilities

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7241

303068
hyperv/vmbus: Channel struct field rename

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7242

303069
hyperv/vmbus: Pass channel as the first argument for channel callback

The prepares to kill device private fields in channel struct, which
are not flexible and extensible.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7243

303070
hyperv/vmbus: Deprecate the device private data in channel struct

They are neither flexible nor extensible.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7244

303071
hyperv/vmbus: Hide channel struct definition.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7245

303072
hyperv/vmbus: Save MNF trigger location instead of MNF trigger index.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7246

303127
hyperv/vmbus: Save event flag location and evet flag mask.

This avoids unnecessary access to the vmbus_softc struct on sending path.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7257

303128
hyperv/vmbus: Reorder channel fields.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7258

303129
hyperv/vmbus: Shuffle function declaration and macro definition.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7259

303131
hyperv: hv_vmbus_channel -> vmbus_channel

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7260

MFC 302873,302874

302873
hyperv/vmbus: Set vcpuid to 0, if MSR_HV_VP_INDEX does not exist.

Mainly for compatibility. While I'm here, rename cpuid related
fields in hv_vmbus_channel.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7141

302874
hyperv/vmbus: Field rename

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7146

MFC 302867-302870

302867
hyperv/vmbus: Remove unused struct

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7127

302868
hyperv/vmbus: Function rename

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7129

302869
hyperv/vmbus: Remove unused function definition/declaration.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7131

302870
hyperv/vmbus: Use iovec for bufring scatter/gather list.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7134

MFC 302864

hyperv/vmbus: Merge hv_channel_mgmt.c into hv_channel.c

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7126

MFC 302816-302818

302816
hyperv/vmbus: Release vmbus channel lock before detach devices

Device detach method may sleep.

While I'm here, rename the function, fix indentation and function
comment.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7110

302817
hyperv/vmbus: Field renaming to reflect reality

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7111

302818
hyperv/vmbus: Fix the racy channel close.

It is not safe to iterate the sub-channel list w/o lock on the
close path, while it's even more difficult to hold the lock
and iterate the sub-channel list. We leverage the
vmbua_{get,rel}_subchan() functions to solve this dilemma.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7112

MFC 302808-302815

302808
hyperv/vmbus: Alloc/Free monitor param in vmbus channel alloc/free.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7101

302809
hyperv/vmbus: Move device register and channel free to the caller.

This paves the way for more cleanup/disentangle.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7102

302810
hyperv/vmbus: Move new channel scan notification to device register

And nuke now unnecessary function indirection.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7103

302811
hyperv/vmbus: Cleanup vmbus_chan_msgproc_choffer

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7104

302812
hyperv/vmbus: Nuke the channel open state.

Channel is either opened or not-opened.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7105

302813
hyperv/vmbus: Cleanup vmbus_chan_add()

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7106

302814
hyperv/vmbus: Use sub-channel index to detect primary channel

In case that VMBUS_CHAN_ISPRIMARY is needed in the early place of
channel setup.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7108

302815
hyperv/vmbus: Only add primary channels to vmbus channel list

- Make the vmbus_chan_add more straightforward.
- Partially fix the hv_vmbus_release_unattached_channels().

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7109

MFC 302733,302737,302801-302806

302733
hyperv/vmbus: Remove unused code

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7089

302737
hyperv/vmbus: Cleanup channel rescind

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7090

302801
hyperv/vmbus: Remove unused bits

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7091

302802
hyperv: hv_guid -> struct hyperv_guid.

This paves way for the further cleanup/disentangle.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7092

302803
hyperv/vmbus: Move channel offer message definition to vmbus_reg.h

- Avoid bit fields.
- Avoid unnecessary indirection.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7093

302804
hyperv/vmbus: Switch to vmbus channel message macros

Prepare for more cleanup.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7094

302805
hyperv/vmbus: Remove unused bits

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7095

302806
hyperv/vmbus: Get rid of rel{_id,id}, use channel id consistently.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7100

MFC 302723,302726,302731

302723
hyperv: All Hypercall parameters have same alignment requirement.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7086

302726
hyperv: Signal event input parameter is shared w/ MNF

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7087

302731
hyperv/vmbus: Reorganize MNF event sending.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7088

MFC 302698-302704,302706

302698
hyperv/vmbus: Add vmbus method for GUID base device probing.

Reduce the exposure of hv_device.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7024

302699
hyperv/vmbus: All ivars are read-only; nuke unnecessary write_ivar

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7025

302700
hyperv/vmbus: Add channel ivar accessor.

This makes life easier during the transition period to nuke the hv_device.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7026

302701
hyperv/stor: Avoid the hv_device and nuke the broken get_stor_device

This paves way to nuke the hv_device, which is actually an unncessary
indirection.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7027

302702
hyperv/util: Avoid the hv_device

This paves way to nuke the hv_device, which is actually an unncessary
indirection.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7028

302703
hyperv/vmbus: Deprecate the usage of hv_device.

This paves way to nuke the hv_device, which is actually an unncessary
indirection.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7032

302704
hyperv/hn: Avoid the hv_device

This paves way to nuke the hv_device, which is actually an unncessary
indirection.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7033

302706
hyperv: Get rid of hv_device, which is unnecessary indirection.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7034

MFC 302693-302697

302693
hyperv/vmbus: Make channel id a field of hv_vmbus_channel.

This prepares to remove the unnecessary offer message embedding in
hv_vmbus_channel.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7014

302694
hyperv/vmbus: Make subchan index a field of hv_vmbus_channel.

This prepares to remove the unnecessary offer message embedding in
hv_vmbus_channel.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7015

302695
hyperv/vmbus: Add flags field into hv_vmbus_channel for MNF indication

This prepares to remove the unnecessary offer message embedding in
hv_vmbus_channel.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7019

302696
hyperv/vmbus: Add type/instance guid fields into hv_vmbus_channel

This prepares to remove the unnecessary offer message embedding in
hv_vmbus_channel.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7020

302697
hyperv/vmbus: Remove the embedded offer message from hv_vmbus_channel

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7021

MFC 302636-302638

302636
hyperv/vmbus: Move channel map to vmbus_softc

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6982

302637
hyperv/vmbus: Remove needed bits

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7002

302638
hyperv/vmbus: Destroy channel list lock upon attach failure and detach.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D7003

MFC 302617-302621,302623,302629-302631

302617
hyperv/vmbus: Flatten channel message response processing.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6914

302618
hyperv/vmbus: Avoid tx_evtflags setting code duplication.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6915

302619
hyperv/vmbus: Busdma-fy Hypercall signal event input parameter.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6916

302620
hyperv: Nuke unused stuffs

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6917

302621
hyperv/vmbus: Don't be oversmart in default cpu selection.

Pin the channel to cpu0 by default. Drivers having special channel-cpu
mapping requirement should call vmbus_channel_cpu_{set,rr}() themselves.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6918

302623
hyperv/vmbus: Minor renaming

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6919

302629
hyperv/vmbus: Rework vmbus version accessing.

Instead of global variable, vmbus version is accessed through
a vmbus DEVMETHOD now.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6953

302630
hyperv/vmbus: Move GPADL index into vmbus_softc

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6954

302631
hyperv/vmbus: Move channel list to vmbus_softc

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6956

MFC 302607-302612

302607
hyperv/vmbus: Use post message Hypercall APIs for channel open

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6876

302608
hyperv/vmbus: Remove unnecessary check and unapplied comment

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6877

302609
hyperv/vmbus: Use post message Hypercall APIs for GPADL connect.

This also fixes memory leakge if sub-connect messages are needed.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6878

302610
hyperv/vmbus: Use post message Hypercall APIs for channel close

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6906

302611
hyperv/vmbus: Use post message Hypercall APIs for GPA disconnect

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6912

302612
hyperv: Nuke unused stuffs

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6913

MFC 302543-302545,302547,302549,302554,302556,302557,302559,302606

302543
hyperv/vmbus: Use post message Hypercall APIs for channel request

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6831

302544
hyperv/hn: Add tunable to allow tcp_lro_queue_mbuf()

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6841

302545
hyperv/vmbus: Function renaming.

And pass vmbus_softc to vmbus_doattach()

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6842

302547
hyperv/vmbus: Explicitly assign channel message process array.

While I'm here, remove the useless message type from message process
array, which is not used and serves no purposes at all.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6858

302549
hyperv/vmbus: Add sysctl to expose vmbus version.

Requested by: Hongxiong Xian <v-hoxian microsoft com>
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6860

302554
hyperv/vmbus: Use post message Hypercall APIs for unload

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6861

302556
hyperv/vmbus: Create channel synchronously.

The device probe/attach has been move to a different thread, so the
reasons to create the channel asynchronously are no longer valid.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6862

302557
hyperv/vmbus: Save vmbus softc to channels.

So that we don't need to access the global vmbus softc.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6863

302559
hyperv/vmbus: Embed channel detach task in channel itself.

GC work queue stuffs.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6864

302606
hyperv/vmbus: Reorganize vmbus scan process.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6875

MFC 302540

hyperv/vmbus: Implement a new set of APIs for post message Hypercall

And use this new APIs for Initial Contact post message Hypercall.
More post message Hypercalls will be converted.

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6830

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

hyperv/vmbus: Change tx_evtflags type to u_long to match vmbus_evtflags

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6745

hyperv/vmbus: Busdma-fy MNF and event flags.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6744

hyperv/vmbus: Constify channel message

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6708

hyperv/vmbus: Factor out channel message processing

This paves the way for further cleanup.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6707

hyperv/vmbus: Define type for channel messages.

And fix message processing; only channel messages are supported.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6706

hyperv: Rename some cleaned up/almost cleaned up files

MFC after: 1 week
Sponsored by: Microsoft OSTC

hyperv/vmbus: Redefine event flags.

- Nuke unnecessary union.
- Avoid convoluted macro indirection.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6671

hyperv: Move guid2str from vmbus file to hyperv file

- Use uint8_t for GUID byte array.
- Define GUID string length.
- Break long lines.
- Nuke unnecessary stack variable.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6640

hyperv/vmbus: White space cleanup

No functional changes

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6637

hyperv/vmbus: Redefine SynIC message.

- Avoid unnecessary indirection.
- Avoid bit fields.
- Use __packed.

Reviewed by: Jun Su <junsu microsoft com>
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6636

hyperv/vmbus: Move global vmbus id array to stack.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6603

hyperv/vmbus: Rename ISR functions

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6601

hyperv/vmbus: Process event timer before checking events

And update comment.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6600

hyperv/vmbus: Move SINT settings to vmbus_var.h

While I'm here remove the event timer's dependency on hv_vmbus_priv.h

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6574

hyperv: Test features before enabling optional functionalities

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6571

hyperv/vmbus: Move MSR EOM to hyperv_reg.h

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6567

hyperv/vmbus: Rework SynIC setup and teardown

- Avoid bit fields.
- Fix SINT setup (preserve required bits).

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6529

hyperv/vmbus: Nuke unnecessary MSR read

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6528

hyperv/vmbus: Pass vmbus softc to vmbus_synic_setup

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6527

hyperv/vmbus: Minor style and white space cleanup

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6523

hyperv/vmbus: Move two global flags into vmbus softc

And pack them into one flag field.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6522

hyperv/vmbus: Move vcpuid into vmbus softc per-cpu data

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6521

hyperv/vmbus: Move event/message taskqueue/task to vmbus softc

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6520

hyperv/vmbus: Allocate/setup IDT vector after all ISR resources are ready

And release IDT vector before releasing ISR resources on interrupt
teardown path. We still have some work to do on the interrupt tearing
down path.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6519

hyperv/vmbus: Check hyperv_dmamem_alloc return value

Though it is highly unlikely this function would fail w/ BUS_DMA_WAITOK,
we had better to check its return value; better safe then sorry here.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6518

hyperv/vmbus: Free message taskqueue during interrupt teardown

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6506

hyperv/vmbus: Factor out functions for vmbus interrupt set/teardown

This paves way for further cleanup and fix.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6505

hyperv/vmbus: Git rid of sc version of pcpu data extraction macro

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6503

hyperv/vmbus: Use busdma(9) for messages and event flags

And
- Move message and event flags to vmbus_softc per-cpu data.
- Get rid of hv_setup_arg, which serves no purpose now.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6502

hyperv/vmbus: Move SynIC setup/teardown from hyperv file to vmbus file

Avoid unnecessary exposure.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6501

hyperv/vmbus: Rename local variable and break long lines

No functional changes.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6500

hyperv/vmbus: Pass vmbus_softc and curcpu to hv_vmbus_isr()

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6498

hyperv/vmbus: Move vmbus interrupt counter into vmbus softc

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6497

hyperv/vmbus: Move IDT vector to vmbus_softc

Prepare to get rid of the hv_setup_arg.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6449

hyperv/vmbus: Get rid of vmbus_devp

While I'm here, nuke useless print in vmbus_attach().

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6447

hyperv/vmbus: Declare Synic message and event w/ proper types

Avoid ugly casts.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6446

hyperv: Move Hypercall setup to an early place.

It does not belong to the vmbus.

While I'm here rework the Hypercall setup, e.g. use busdma(9)
and avoid bit fields.

Discussed with: Jun Su <junsu microsoft com>
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6445

hyperv/vmbus: Use consistent device description as other devices

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6418

hyperv/vmbus: Minor function definition style fixup

Reviewed by: Jun Su <junsu microsoft com>
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6417

hyperv/vmbus: Fix SYSINIT function prototype and usage.

Reviewed by: Jun Su <junsu microsoft com>
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6416

hyperv/vmbus: Reindent and cleanup devmethods.

While I'm here, use DEVMETHOD_END.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6415

hyperv/vmbus: Staticize vmbus_devclass

Reviewed by: Jun Su <junsu microsoft com>
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6414

hyperv/vmbus: Nuke unnecessary function indirection

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6411

hyperv/vmbus: Remove useless modevent handler

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6410

hyperv/vmbus: Function renaming vmbus_msg_swintr -> vmbus_msg_task

It is not an SWI handler for a long time.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6409

hyperv/vmbus: Utilize curcpu

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6408

hyperv/vmbus: Minor white space and style cleanup

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6407

hyperv/vmbus: Avoid two unnecessary protocol checks on isr handling path

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6405

hyperv/vmbus: Reduce the # of event loops by recording event flag count

Use vmbus softc to save vmbus per-cpu data. More stuffs will be moved
into vmbus softc.

MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6403

Add an EARLY_AP_STARTUP option to start APs earlier during boot.

Currently, Application Processors (non-boot CPUs) are started by
MD code at SI_SUB_CPU, but they are kept waiting in a "pen" until
SI_SUB_SMP at which point they are released to run kernel threads.
SI_SUB_SMP is one of the last SYSINIT levels, so APs don't enter
the scheduler and start running threads until fairly late in the
boot.

This change moves SI_SUB_SMP up to just before software interrupt
threads are created allowing the APs to start executing kernel
threads much sooner (before any devices are probed). This allows
several initialization routines that need to perform initialization
on all CPUs to now perform that initialization in one step rather
than having to defer the AP initialization to a second SYSINIT run
at SI_SUB_SMP. It also permits all CPUs to be available for
handling interrupts before any devices are probed.

This last feature fixes a problem on with interrupt vector exhaustion.
Specifically, in the old model all device interrupts were routed
onto the boot CPU during boot. Later after the APs were released at
SI_SUB_SMP, interrupts were redistributed across all CPUs.

However, several drivers for multiqueue hardware allocate N interrupts
per CPU in the system. In a system with many CPUs, just a few drivers
doing this could exhaust the available pool of interrupt vectors on
the boot CPU as each driver was allocating N * mp_ncpu vectors on the
boot CPU. Now, drivers will allocate interrupts on their desired CPUs
during boot meaning that only N interrupts are allocated from the boot
CPU instead of N * mp_ncpu.

Some other bits of code can also be simplified as smp_started is
now true much earlier and will now always be true for these bits of
code. This removes the need to treat the single-CPU boot environment
as a special case.

As a transition aid, the new behavior is available under a new kernel
option (EARLY_AP_STARTUP). This will allow the option to be turned off
if need be during initial testing. I plan to enable this on x86 by
default in a followup commit in the next few days and to have all
platforms moved over before 11.0. Once the transition is complete,
the option will be removed along with the !EARLY_AP_STARTUP code.

These changes have only been tested on x86. Other platform maintainers
are encouraged to port their architectures over as well. The main
things to check for are any uses of smp_started in MD code that can be
simplified and SI_SUB_SMP SYSINITs in MD code that can be removed in
the EARLY_AP_STARTUP case (e.g. the interrupt shuffling).

PR: kern/199321
Reviewed by: markj, gnn, kib
Sponsored by: Netflix

hyperv/et: Make Hyper-V event timer a device.

Submitted by: Jun Su <junsu microsoft com>
Reviewed by: sephe, Dexuan Cui <decui microsoft com>
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5957

hyperv: Update copyright to 2016 for the files Microsoft changed in 2016

Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D6039

hyperv/vmbus: Make device probe/attach synchronous w/ vmbus attach/SYSINIT

Discussed with: Jun Su <junsu microsoft com>, Dexuan Cui <decui microsoft com>
MFC after: 1 week
Sponsored by: Microsoft OSTC

hyperv: Deprecate HYPERV option by moving Hyper-V IDT vector into vmbus

Submitted by: Jun Su <junsu microsoft com>
Reviewed by: jhb, kib, sephe
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5910

hyperv/vmbus: Merge duplicated version check for events

Submitted by: Jun Su <junsu microsoft com>
Reviewed by: sephe
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5911

hyperv: Replace 0 w/ NULL

Submitted by: pfg
MFC after: 1 week
Sponsored by: Microsoft OSTC

hyperv: Use lapic_{alloc,free}_ipi to allocate private interrupt vector

Suggested by: jhb
Reviewed by: Dexuan Cui <decui microsoft com>, Jun Su <junsu microsoft com>
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5850

hyperv: Typo in r297634

Noticed by: hiren
MFC after: 1 week
Sponsored by: Microsoft OSTC

hyperv: Use mb() instead of atomic_thread_fence_seq_cst()

Since atomic_thread_fence_seq_cst() will become compiler fence on UP kernel.

Reviewed by: kib, Dexuan Cui <decui microsoft com>
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5852

hyperv/vmbus: Create per-cpu fast taskqueue for msg handling

Using one taskqueue does not work, since the EOM MSR must be written
on the msg's owner CPU.

Noticed by: Jun Su <junsu microsoft com>
Discussed with: Jun Su <junsu microsoft com>, Dexuan Cui <decui microsoft com>
MFC after: 1 week
Sponsored by: Microsoft OSTC

hyperv/vmbus: Remove NULL check for taskqueue_create_fast(M_WAITOK)

Submitted by: Jun Su <junsu microsoft com>
Reviewed by: Dexuan Cui <decui microsoft com>, sephe
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5215

hyperv/vmbus: Use taskqueue_fast for non-performance critical messages

This gets rid of the per-cpu SWIs.

Submitted by: Jun Su <junsu microsoft com>
Reviewed by: Dexuan Cui <decui microsoft com>, sephe
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5215

hyperv/evttimer: Use an independent message slot so that it can work

Using the same message slot as the other types of the messages has
the side effect that the event timer message could be deferred to
the swi threads to run (lacking of trapframe and the original code
didn't even handle that, so the event timer was actually broken).

As of this commit we use an independent message slot for event timer,
so that we could handle all of event timer messages in the interrupt
handler directly. Note, the message slot for event timer is still
bind to the same interrupt vector as the other types of messages.

Submitted by: Jun Su <junsu microsoft com>
Reviewed by: sephe
Discussed with: Jun Su <junsu microsoft com>, Dexuan Cui <decui microsoft com>
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5696

hyperv/vmbus: Implement bus_child_pnpinfo_str method

Submitted by: Jun Su <junsu microsoft com>
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5669

hyperv: Factor out snprinf_hv_guid()

Submitted by: Ju Sun <junsu microsoft com>
Reviewed by: Dexuan Cui <decui microsoft com>, sephe
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5651

hyperv: Use proper fence function to keep store-load order for msgs

sfence only makes sure about the store-store order, which is not
sufficient here. Use atomic_thread_fence_seq_cst() as suggested
jhb and kib (a locked op in the nutshell, which should have the

Reviewed by: jhb, kib, Jun Su <junsu microsoft com>
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5436

hyperv: Use WAITOK in the places where we can wait

And convert rndis non-hot path spinlock to mutex.

Submitted by: Jun Su <junsu microsoft com>
Reviewed by: adrian, sephe
Approved by: adrian (mentor)
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D5081

hyperv: Use standard taskqueue instead of hv_work_queue

HyperV code was ported from Linux. There is an implementation of
work queue called hv_work_queue. In FreeBSD, taskqueue could be
used for the same purpose. Convert all the consumer of hv_work_queue
to use taskqueue, and remove work queue implementation.

Submitted by: Jun Su <junsu microsoft com>
Reviewed by: adrian, Hongjiang Zhang <honzhan microsoft com>
Approved by: adrian (mentor)
MFC after: 1 week
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D4963

hyperv/vmbus: Event handling code refactor.

- Use taskqueue instead of swi for event handling.
- Scan the interrupt flags in filter
- Disable ringbuffer interrupt mask in filter to ensure no unnecessary
interrupts.

Submitted by: Jun Su <junsu microsoft com>
Reviewed by: adrian, sephe, Dexuan <decui microsoft com>
Approved by: adrian (mentor)
MFC after: 2 weeks
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D4920

hyperv: add interrupt counters

Submitted by: Howard Su <howard0su gmail com>
Reviewed by: royger, Dexuan Cui <decui microsoft com>, adrian
Approved by: adrian (mentor)
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D4693

hyperv: implement an event timer

Submitted by: Howard Su <howard0su@gmail.com>
Reviewed by: delphij, royger, adrian
Approved by: adrian (mentor)
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D4676

hyperv: use x86 generic code to do the hypervisor detection

This is first step to move the generic part of HV code into kernel instead
of module, so that it is possible to use hypercall to implement some other
paravirtualization code in the kernel.

Submitted by: Howard Su <howard0su@gmail.com>
Reviewed by: royger, delphij, adrian
Approved by: adrian (mentor)
Sponsored by: Microsoft OSTC
Differential Revision: https://reviews.freebsd.org/D3072

hyperv: vmbus: run non-blocking message handlers in vmbus_msg_swintr()

We'll remove the per-channel control_work_queue because it can't properly
do serialization of message handling, e.g., when there are 2 NIC devices,
vmbus_channel_on_offer() -> hv_queue_work_item() has a race condition:
for an SMP VM, vmbus_channel_process_offer() can run concurrently on
different CPUs and if the second NIC's
vmbus_channel_process_offer() -> hv_vmbus_child_device_register() runs
first, the second NIC's name will be hn0 and the first NIC's name will
be hn1!

We can fix the race condition by removing the per-channel control_work_queue
and run all the message handlers in the global
hv_vmbus_g_connection.work_queue -- we'll do this in the next patch.

With the coming next patch, we have to run the non-blocking handlers
directly in the kernel thread vmbus_msg_swintr(), because the special
handling of sub-channel: when a sub-channel (e.g., of the storvsc driver)
is received and being handled in vmbus_channel_on_offer() running on the
global hv_vmbus_g_connection.work_queue, vmbus_channel_process_offer()
invokes channel->sc_creation_callback, i.e., storvsc_handle_sc_creation,
and the callback will invoke hv_vmbus_channel_open() -> hv_vmbus_post_message
and expect a further reply from the host, but the handling of the further
messag can't be done because the current message's handling hasn't finished
yet; as result, hv_vmbus_channel_open() -> sema_timedwait() will time out
and th device can't work.

Also renamed the handler type from hv_pfn_channel_msg_handler to
vmbus_msg_handler: the 'pfn' and 'channel' in the old name make no sense.

Submitted by: Dexuan Cui <decui microsoft com>
Reviewed by: royger
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D4596

Microsoft vmbus, storage and other related driver enhancements for HyperV.
- Vmbus multi channel support.
- Vector interrupt support.
- Signal optimization.
- Storvsc driver performance improvement.
- Scatter and gather support for storvsc driver.
- Minor bug fix for KVP driver.
Thanks royger, jhb and delphij from FreeBSD community for the reviews
and comments. Also thanks Hovy Xu from NetApp for the contributions to
the storvsc driver.

PR: 195238
Submitted by: whu
Reviewed by: royger, jhb, delphij
Approved by: royger
MFC after: 2 weeks
Relnotes: yes
Sponsored by: Microsoft OSTC

More BUS_PROBE_NOWILDCARD sweeping. Some devices here (if_ath_ahb and siba)
resist easy conversion since they implement a great deal of their attach
logic inside probe(). Some of this could be fixed by moving it to attach(),
but some requires something more subtle than BUS_PROBE_NOWILDCARD.

Centralize the detection logic for the Hyper-V hypervisor.

Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
Reviewed by: gibbs, grehan
Approved by: re (gjb)

sys/sys/systm.h:
* Add a new VM_GUEST type, VM_GUEST_HV (HyperV guest).

sys/dev/hyperv/vmbus/hv_vmbus_drv_freebsd.c:
sys/dev/hyperv/vmbus/hv_hv.c:
sys/dev/hyperv/stordisengage/hv_ata_pci_disengage.c:
* Set vm_guest to VM_GUEST_HV and use that on other HyperV related
devices instead of cloning the cpuid hypervisor check.
* Cleanup the vmbus_identify function.

In sys/dev/hyperv, fix a number of gcc warnings about usage of anonymous
union members in strict C99, by giving them names. While here, add some
FreeBSD keywords where they were missing.

Approved by: re (gjb)
Reviewed by: grehan

Import Hyper-V paravirtualized drivers from projects/hyperv
branch into head.

Approved by: re@ (hrs)
Obtained from: Microsoft, NetApp, and Citrix.

Latest update from Microsoft.

Obtained from: Microsoft Hyper-v dev team

Microsoft have changed their policy on how the hyper-v code will
be pulled into FreeBSD. From now, FreeBSD will be considered the
upstream repo.

First step: move the drivers away from the contrib area and into
the base system.

A follow-on commit will include the drivers in the amd64 GENERIC kernel.

Import driver source from hyperv-20130627 vendor branch.

Import driver source from hyperv-20130502 vendor branch.

Tag hyperv 20130502

Initial import of the Microsoft HyperV 'enlightened' drivers.

From https://github.com/FreeBSDonHyper-V/VendorBranchForFreeBSDonHyper-V
rev: 99eaa0ddb0485c9d76046664100f6beb1a0a0c58