MFC r283291: don't use CALLOUT_MPSAFE with callout_init()

The main purpose of this MFC is to reduce conflicts for other merges.
Parts of the original change have already "trickled down" via individual MFCs.

MFC r303561: Wrap previous MSIX workaround into #ifndef EARLY_AP_STARTUP.

With EARLY_AP_STARTUP we can successfully negotiate MSIX earlier.

MFC r303554: Block MSIX negotiation until SMP started and IRQ reshuffled.

MFC r303510: Clear scratchpad after MSIX negotiation to not leak garbage.

MFC r303429, r303437, r303551:
Once more refactor KPI between NTB hardware and consumers.

New design allows hardware resources to be split between several consumers.
For example, one BAR can be dedicated for remote memory access, while other
resources can be used for packet transport for virtual Ethernet interface.
And even without resource split, this code allows to specify which consumer
driver should attach the hardware.

From some points this makes the code even closer to Linux one, even though
Linux does not provide the described flexibility.

MFC r303266: Postpone ntb_get_msix_info() till we need to negotiate MSIX.

Calling it earlier increases the window when MSIX info may change.
This change does not solve the problem completely, but seems logical.
Complete solution should probably include link reset in case of MSIX
remap to trigger new negotiation, but we have no way to get notified
about that now.

MFC r302622 (by sephe): ntb: Fix LINT

MFC r302531: Revert odd change, setting limit registers before base.

I don't know what errata is mentioned there, I was unable to find it, but
setting limit before the base simply does not work at all. According to
specification attempt to set limit out of the present window range resets
it to zero, effectively disabling it. And that is what I see in practice.

Fixing this properly disables access for remote side to our memory until
respective xlat is negotiated and set. As I see, Linux does the same.

MFC r302530: Fix wrong copy/paste in r302510.

MFC r302510:
Simplify MSIX MW BAR xlat setup, and don't forget to unlock its limit.

The last fixes SB01BASE_LOCKUP workaround after driver reload.

MFC r302508: Disable SB01BASE_LOCKUP workaround when split BARs disabled.

For some reason hack with sending MSI-X interrupts by writing to remote
LAPIC memory works only for 32-bit BARs, that are available only if split
BARs mode is enabled in BIOS. If it is not, complain loudly and fall back
to less efficient workaround.

MFC r302493: Reimplement doorbell register emulation for NTB_SB01BASE_LOCKUP.

This allows at least first three doorbells to work very close to normal
hardware, properly signaling events to upper layers without spurious or
lost events. Doorbells above the first three may still report spurious
events due to lack of reliable information, but they are rarely used.

MFC r302491: Switch ctx_lock from mutex to rmlock.

It is odd idea to serialize different MSI-X vectors. Use of rmlocks
here allows them to execute in parallel, but still protects ctx.
If upper layers require any additional serialization -- they can
do it by themselves.

MFC r302484: NewBus'ify NTB subsystem.

This follows NTB subsystem modularization in Linux, tuning it to FreeBSD
native NewBus interfaces. This change allows to support different types
of hardware with different drivers, support multiple NTB instances in a
system, ntb_transport module use for needs other then if_ntb, etc.

Sponsored by: iXsystems, Inc.

MFC r302483: Remove some dead code found by Clang static analyzer.

MFC r302482: Fix NTB_SDOORBELL_LOCKUP workaround.

Since SBARxSZ register can be write-once, it can be unusable for disabling
the SBAR. For such case also set SBARxBASE to zero to not intersect with
config BAR.

MFC r301293:
When negotiating NTB_SB01BASE_LOCKUP workaround, don't try to limit the
BAR size to 1MB. According to Xeon v3 specifications and my tests, that
size register is write-once and so not writeable after BIOS written it.

Instead of that, make the code work with BAR of any sufficient size,
properly calculating offset within its base. It also simplifies the code.

Sponsored by: iXsystems, Inc.

MFC r301292: When negotiating MSIX parameters, give other head time to see
our NTB_MSIX_RECEIVED status, before making upper layers overwrite it.

This is not completely perfect, but now it works better then before.

Sponsored by: iXsystems, Inc.

MFC r295618,r300100,r300531:

r295618 (by cem):

NTB: workaround for high traffic hardware hang

This patch comes from Dave Jiang's Linux tree, davejiang/ntb. It hasn't
been accepted into Linus' tree, so I do not have an authoritative SHA1
to point at. Original commit log:

=====================================================================
A hardware errata causes the NTB to hang when heavy bi-directional
traffic in addition to the usage of BAR0/1 (where the registers reside,
including the doorbell registers to trigger interrupts).

This workaround is only available on Haswell and Broadwell platform.
The workaround is to enable split BAR in the BIOS to allow the 64bit
BAR4 to be split into two 32bit BAR4 and BAR5. The BAR4 shall be pointed
to LAPIC region of the remote host. We will bypass the db mechanism and
directly trigger the MSIX interrupts. The offsets and vectors are
exchanged during transport scratch pad negotiation. The scratch pads are
now overloaded in order to allow the exchange of the information. This
gets around using the doorbell and prevents the lockup with additional
pcode changes in BIOS.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
=====================================================================

Notable changes in the FreeBSD version of this patch:
* The MSIX BAR is configurable, like hw.ntb.b2b_mw_idx (msix_mw_idx).
The Linux version of the patch only uses BAR4.
* MSIX negotiation aborts if the link goes down.

Obtained from: Linux (Dual BSD/GPL driver)

r300100 (by cem):

ntb_hw(4): Add sysctls for administrative/test link config, state

dev.ntb_hw.0.admin_up=0/1: Like ifconfig UP/DOWN.
dev.ntb_hw.0.active=0/1: Like ifconfig 'status'

r300531 (by cem):

ntb_hw(4): Only record the first three MSIX vectors

Don't overrun the msix_data array by reading the (unused) link state
interrupt information.

Add respective tunables to all sysctls.

This is direct commit to stable/10, since head does not need it any more.

MFC 103 ntb(4) patches by cem@ up to r295487.

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Remove contractions.

Approved by: jimharris
Sponsored by: Intel

Workaround an issue with hardware by accessing remote device through mem
window.

Approved by: jimharris
Sponsored by: Intel

Simplify register access macros by removing one level of indirection.

Approved by: jimharris
Sponsored by: Intel

Implement workaround for IvyTown 4K BAR size issue.

Approved by: jimharris
Sponsored by: Intel

Simplifying bus alloc resource call since we only need the default values.

Approved by: jimharris
Sponsored by: Intel

Add support for per device features and workarounds.

Approved by: jimharris
Sponsored by: Intel

Restructure the PCI bar initialization code in anticipation of upcoming
bug fixes.

Approved by: jimharris
Sponsored by: Intel

Throw a bit to enable the link to come up on Xeon.

Approved by: jimharris
Sponsored by: Intel

Add some logging to ntb link up.

Approved by: jimharris
Sponsored by: Intel

Add a new driver to support the Intel Non-Transparent Bridge(NTB).

The NTB allows you to connect two systems with this device using a PCI-e
link. The driver is made of two modules:
- ntb_hw which is a basic hardware abstraction layer for the device.
- if_ntb which implements the ntb network device and the communication
protocol.

The driver is limited at the moment to CPU memcpy instead of using DMA, and
only Back-to-Back mode is supported. Also the network device isn't full
featured yet. These changes will be coming soon. The DMA change will also
bring in the ioat driver from the project branch it is on now.

This is an initial port of the GPL/BSD Linux driver contributed by Jon Mason
from Intel. Any bugs are my contributions.

Sponsored by: Intel
Reviewed by: jimharris, joel (man page only)
Approved by: jimharris (mentor)