Copy stable/11@r303970 to releng/11.0 as part of the 11.0-RELEASE
cycle.

Prune svn:mergeinfo from the new branch, and rename it to RC1.

Update __FreeBSD_version.

Use the quarterly branch for the default FreeBSD.conf pkg(8) repo and
the dvd1.iso packages population.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

ntb_hw(4): Add sysctls for administrative/test link config, state

dev.ntb_hw.0.admin_up=0/1: Like ifconfig UP/DOWN.
dev.ntb_hw.0.active=0/1: Like ifconfig 'status'

Reviewed by: ngie
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D6429

NTB: workaround for high traffic hardware hang

This patch comes from Dave Jiang's Linux tree, davejiang/ntb. It hasn't
been accepted into Linus' tree, so I do not have an authoritative SHA1
to point at. Original commit log:

=====================================================================
A hardware errata causes the NTB to hang when heavy bi-directional
traffic in addition to the usage of BAR0/1 (where the registers reside,
including the doorbell registers to trigger interrupts).

This workaround is only available on Haswell and Broadwell platform.
The workaround is to enable split BAR in the BIOS to allow the 64bit
BAR4 to be split into two 32bit BAR4 and BAR5. The BAR4 shall be pointed
to LAPIC region of the remote host. We will bypass the db mechanism and
directly trigger the MSIX interrupts. The offsets and vectors are
exchanged during transport scratch pad negotiation. The scratch pads are
now overloaded in order to allow the exchange of the information. This
gets around using the doorbell and prevents the lockup with additional
pcode changes in BIOS.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
=====================================================================

Notable changes in the FreeBSD version of this patch:
* The MSIX BAR is configurable, like hw.ntb.b2b_mw_idx (msix_mw_idx).
The Linux version of the patch only uses BAR4.
* MSIX negotiation aborts if the link goes down.

Obtained from: Linux (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division

NTB: WC/WB isn't enough; set MMR region as UC

And expose vm_memattr_t of current mapping to consumers (as well as the
ability to change it to one of UC, WB, WC).

After short discussion with: jhb (but no review)
Sponsored by: EMC / Isilon Storage Division

if_ntb: Add Xeon link watchdog register writes

This feature is disabled by default. To enable it, tune
hw.if_ntb.enable_xeon_watchdog to non-zero.

If enabled, writes an unused NTB register every second to demonstrate to
a hardware watchdog that the NTB device is still alive. Most machines
with NTB will not need this -- you know who you are.

Sponsored by: EMC / Isilon Storage Division

NTB: Expose 32-bit BAR limits to consumers

32-bit BARs can only address memory mapped in the low 32 bits of
physical RAM. Expose this as a 'plimit' out parameter from
ntb_mw_get_range().

Fix if_ntb to allocate memory within this limit.

Sponsored by: EMC / Isilon Storage Division

ntb_hw: Add programmatic interface to enable/disable WC

Enable users to enable/disable WC on memory windows programmatically.

Sponsored by: EMC / Isilon Storage Division

ntb: Use caddr_t to simplify pointer arithmetic

Sponsored by: EMC / Isilon Storage Division

NTB: Add device introspection sysctl hierarchy

This should export all of the same information as the Linux ntb_hw_intel
debugfs info file, but with a bit more structure, in the sysctl tree
rooted at 'dev.ntb_hw.<N>.debug_info'.

Raw registers are marked as OPAQUE because reading them on some hardware
revisions may cause a hard lockup (NTB errata). They can be read with
'sysctl -x dev.ntb_hw.<N>.debug_info.registers'. On Xeon platforms,
some additional registers are available under 'registers.xeon_stats' and
'registers.xeon_hw_err'. They are exported as big-endian values so that
the 'sysctl -x' output is legible.

Shrink the feature mask to 32 bits so we can use the %b formatter in
'debug_info.features'.

Sponsored by: EMC / Isilon Storage Division

NTB: Add ntb_db_vector_mask() missed in r289546

This is the last one.

Obtained from: Linux (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division

if_ntb: MFV e26a5843: Move MW/DB management to if_ntb

This is the last e26a5843 patch. The general thrust of the rewrite was
to move more responsibility for Memory Window and Doorbell interrupt
management from the ntb_hw driver to if_ntb.

A number of APIs have been added, removed, or replaced. The old
DB callback mechanism has been excised. Instead, callers (if_ntb) are
responsible for configuring MWs and handling their interrupts more
directly.

This adds a tunable, hw.ntb.max_mw_size, allowing users to limit the
size of memory windows used by if_ntb (identical to the Linux modparam
of the same name).

Despite attempts to keep mechanical name changes to separate commits,
some have snuck in here. At least the driver should be much more
similar to the latest Linux one now -- making porting fixes easier.

Authored by: Allen Hubbe
Obtained from: Linux (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division

if_ntb: Rename things to match Linux driver

No functional change. Part of the huge rewrite (e26a5843).

Obtained from: Linux (e26a5843) (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division

NTB: "Split ntb_hw_intel and ntb_transport drivers"

This Linux commit was more or less a rewrite. Unfortunately, the commit
log does not give a lot of context for the rewrite. I have tried to
faithfully follow the changes made upstream, including matching function
names where possible, while churning the FreeBSD driver as little as
possible.

This is the bulk of the rewrite. There are two groups of changes to
follow in separate commits: fleshing out the rest of the changes to
xeon_setup_b2b_mw(), and some changes to if_ntb.

Yes, this is a big patch (3 files changed, 416 insertions(+), 237
deletions(-)), but the Linux patch was 13 files changed, 2,589
additions(+) and 2,195 deletions(-).

Original Linux commit log:
Change ntb_hw_intel to use the new NTB hardware abstraction layer.

Split ntb_transport into its own driver. Change it to use the new NTB
hardware abstraction layer.

Authored by: Allen Hubbe
Obtained from: Linux (e26a5843) (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division

NTB: Rename some variables/functions to match Linux

No functional change.

Still part of the huge e26a5843 rewrite. I'm trying to make it less of
a complete rewrite in the FreeBSD version of the driver. Still, it
helps if our names match Linux.

Obtained from: Linux (e26a5843) (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division

NTB: Rename some constants to match Linux

No functional change.

Obtained from: Linux (part of e26a5843) (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division

NTB: MFV ab760a0c: Add split BAR support for Haswell

On the Haswell platform, a split BAR option to allow creation of 2 32bit
BARs (4 and 5) from the 64bit BAR 4. Adding support for this new option.

Authored by: Dave Jiang
Obtained from: Linux (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division

NTB: Add variable number MW, DB CB support code

This is a follow-up to r289208: "Xeon Errata Workaround."

Add logic to support a variable number of memory windows and doorbell
callbacks. This was added to the Linux driver in the "Xeon Errata
Workaround" commit, but I skipped it because it didn't look neccessary
at the time. It is needed for future Haswell split-BAR support, so
bring it in now.

A new tunable was added for if_ntb, 'hw.ntb.max_num_clients'. By
default, it is set to zero -- infer the number of clients from the
number of memory windows available from the hardware. Any other
positive value can specify a different number of clients, limited by the
number of doorbell callbacks available (4 under MSI-X, or 15 (Xeon) or
34 (SoC) under legacy INTx).

Obtained from: Linux (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division

NTB: MFV 403c63cb: client event cleanup

Provide a better event interface between the client and transport.

Authored by: Jon Mason
Obtained from: Linux (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division

NTB: MFV e8aeb60c: Disable interrupts and poll under high load

Authored by: Jon Mason
Obtained from: Linux (Dual BSD/GPL driver)
Sponsored by: EMC / Isilon Storage Division

NTB: MFV 49793889: Rename Variables for NTB-RP

Many variable names in the NTB driver refer to the primary or secondary
side. However, these variables will be used to access the reverse case
when in NTB-RP mode. Make these names more generic in anticipation of
NTB-RP support.

Authored by: Jon Mason
Obtained from: Linux
Sponsored by: EMC / Isilon Storage Division

NTB: Update pci ids

Add JSF, HSX, BDX ids; add two additional Xeon errata flags while we're
here.

Obtained from: Linux
Sponsored by: EMC / Isilon Storage Division

NTB: MFV 948d3a65: Xeon Errata Workaround

There is a Xeon hardware errata related to writes to SDOORBELL or B2BDOORBELL
in conjunction with inbound access to NTB MMIO Space, which may hang the
system. To workaround this issue, use one of the memory windows to access the
interrupt and scratch pad registers on the remote system. This bypasses the
issue, but removes one of the memory windows from use by the transport. This
reduction of MWs necessitates adding some logic to determine the number of
available MWs.

Since some NTB usage methodologies may have unidirectional traffic, the ability
to disable the workaround via modparm has been added.

See BF113 in
http://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/xeon-c5500-c3500-spec-update.pdf
See BT119 in
http://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/xeon-e5-family-spec-update.pdf

Authored by: Jon Mason
Obtained from: Linux
Sponsored by: EMC / Isilon Storage Division

NTB: Add hw.ntb sysctl node

Add some logging to ntb link up.

Approved by: jimharris
Sponsored by: Intel

Add a new driver to support the Intel Non-Transparent Bridge(NTB).

The NTB allows you to connect two systems with this device using a PCI-e
link. The driver is made of two modules:
- ntb_hw which is a basic hardware abstraction layer for the device.
- if_ntb which implements the ntb network device and the communication
protocol.

The driver is limited at the moment to CPU memcpy instead of using DMA, and
only Back-to-Back mode is supported. Also the network device isn't full
featured yet. These changes will be coming soon. The DMA change will also
bring in the ioat driver from the project branch it is on now.

This is an initial port of the GPL/BSD Linux driver contributed by Jon Mason
from Intel. Any bugs are my contributions.

Sponsored by: Intel
Reviewed by: jimharris, joel (man page only)
Approved by: jimharris (mentor)