MFC 9a47ae044b48:
Bump driver versions for mlx5en(4) and mlx4en(4).

Sponsored by: Mellanox Technologies // NVIDIA Networking

Git Hash: e87e3e82f3a062856118ed42751b498277eb09a5
Git Author: hselasky@FreeBSD.org

MFC r367719:
Make mlx5_cmd_exec_cb() a safe API in mlx5core.

APIs that have deferred callbacks should have some kind of cleanup
function that callers can use to fence the callbacks. Otherwise things
like module unloading can lead to dangling function pointers, or worse.

The IB MR code is the only place that calls this function and had a
really poor attempt at creating this fence. Provide a good version in
the core code as future patches will add more places that need this
fence.

Linux commit:
e355477ed9e4f401e3931043df97325d38552d54

Sponsored by: Mellanox Technologies // NVIDIA Networking

MFC r367717:
Enhance the mlx5_core_create_cq() function in mlx5core.

Enhance mlx5_core_create_cq() to get the command out buffer from the
callers to let them use the output.

Linux commit:
38164b771947be9baf06e78ffdfb650f8f3e908e

Sponsored by: Mellanox Technologies // NVIDIA Networking

MFC r367715:
Fix error handling order in create_kernel_qp in mlx5ib.

Make sure order of cleanup is exactly the opposite of initialization.

Linux commit:
f4044dac63e952ac1137b6df02b233d37696e2f5

Sponsored by: Mellanox Technologies // NVIDIA Networking

MFC r362953:
Infiniband clients must be attached and detached in a specific order in ibcore.

Currently the linking order of the infiniband, IB, modules decide in which
order the clients are attached and detached. For example one IB client may
use resources from another IB client. This can lead to a potential deadlock
at shutdown. For example if the ipoib is unregistered after the ib_multicast
client is detached, then if ipoib is using multicast addresses a deadlock may
happen, because ib_multicast will wait for all its resources to be freed before
returning from the remove method.

Fix this by using module_xxx_order() instead of module_xxx().

Differential Revision: https://reviews.freebsd.org/D23973
Sponsored by: Mellanox Technologies

MFC r358220:
Fix broken MLX5_IB_INDEX() macro in mlx5ib(4).

The index should be computed as distance from arg[0] and not
the beginning of struct mlx5_ib_congestion .

While at it fix a use of zero length array to avoid depending
on undefined compiler behaviour.

Sponsored by: Mellanox Technologies

MFC r352998:
Bump driver version for mlx5core, mlx5en(4) and mlx5ib(4).

Sponsored by: Mellanox Technologies

MFC r352957:
Update warning and error print formats in mlx5ib.

Submitted by: slavash@
Sponsored by: Mellanox Technologies

MFC r352956:
Fix reported max SGE calculation in mlx5ib.

Add the 512 bytes limit of RDMA READ and the size of remote address to the max
SGE calculation.

Submitted by: slavash@
Linux commit: 288c01b746aa
Sponsored by: Mellanox Technologies

MFC r347325:
Bump the Mellanox driver version numbers and the FreeBSD version number.

Sponsored by: Mellanox Technologies

MFC r347304:
Always return success for RoCE modify port in mlx5ib.

CM layer calls ib_modify_port() regardless of the link layer.

For the Ethernet ports, qkey violation and Port capabilities
are meaningless. Therefore, always return success for ib_modify_port
calls on the Ethernet ports.

Linux Commit:
ec2558796d25e6024071b6bcb8e11392538d57bf

Submitted by: slavash@
Sponsored by: Mellanox Technologies

MFC r347303:
Add support for new rates to mlx5ib.

Submitted by: slavash@
Sponsored by: Mellanox Technologies

MFC r347299:
Add support for 200Gb ethernet speeds to mlx5core.

Submitted by: slavash@
Sponsored by: Mellanox Technologies

MFC r347259:
Remove unused module parameter in mlx5ib.

Sponsored by: Mellanox Technologies

MFC r347258:
Make sure to error out when arming the CQ fails in mlx4ib and mlx5ib.

Sponsored by: Mellanox Technologies

MFC r347251:
Import Linux code to implement mlx5_ib_disassociate_ucontext() in mlx5ib.

Submitted by: kib@
Sponsored by: Mellanox Technologies

MFC r341587:
mlx4/mlx5: Updated driver version to 3.5.0

Sponsored by: Mellanox Technologies

MFC r341571:
mlx5ib: Set default active width and speed when querying port.

Make sure the active width and speed is set in case the
translate_eth_proto_oper() function doesn't recognize the
current port operation mask.

Linux commit:
7672ed33c4c15dbe9d56880683baaba4227cf940

Sponsored by: Mellanox Technologies

MFC r341570:
mlx5ib: Make sure the congestion work timer does not escape the drain procedure.

If the mlx5_ib_read_cong_stats() function was running when mlx5ib was unloaded,
because this function unconditionally restarts the timer, the timer can still
be pending after the delayed work has been cancelled. To fix this simply loop
on the delayed work cancel procedure as long as it returns non-zero.

Sponsored by: Mellanox Technologies

MFC r341569:
mlx5ib: Fix null pointer dereference in mlx5_ib_create_srq

Although "create_srq_user" does overwrite "in.pas" on some paths, it
also contains at least one feasible path which does not overwrite it.

Sponsored by: Mellanox Technologies

MFC r341568:
mlx5ib: Fix sign extension in mlx5_ib_query_device

"fw_rev_min(dev->mdev)" with type "unsigned short" (16 bits, unsigned) is
promoted in "fw_rev_min(dev->mdev) << 16" to type "int" (32 bits, signed), then
sign-extended to type "unsigned long" (64 bits, unsigned). If
"fw_rev_min(dev->mdev) << 16" is greater than 0x7FFFFFFF, the upper bits of the
result will all be 1.

Sponsored by: Mellanox Technologies

MFC r341567:
mlx5: Fix driver version location

Driver description should be set by core and not by the Ethernet driver.

Sponsored by: Mellanox Technologies

MFC r341557:
mlx5: Add SRQ fixes from Linux

Combine multiple fixes from Linux to SRQ.
Linux commits:
c73b791 IB/mlx5: Assign SRQ type earlier
0fd27a8 IB/mlx5: Fix out-of-bound access
c2b37f7 IB/mlx5: Fix integer overflows in mlx5_ib_create_srq
d63c467 RDMA/mlx5: Fix memory leak in mlx5_ib_create_srq() error path

Sponsored by: Mellanox Technologies

MFC r341554:
mlx5: Raise fatal IB event when sys error occurs

All other mlx5_events report the port number as 1 based, which is how FW
reports it in the port event EQE. Reporting 0 for this event causes
mlx5_ib to not raise a fatal event notification to registered clients
due to a seemingly invalid port.

All switch cases in mlx5_ib_event that go through the port check are
supposed to set the port now, so just do it once at variable
declaration.

Linux commit:
aba462134634b502d720e15b23154f21cfa277e5

Sponsored by: Mellanox Technologies

MFC r341553:
mlx5: Fix integer overflow while resizing CQ

The user can provide very large cqe_size which will cause to integer
overflow.

Linux commit:
28e9091e3119933c38933cb8fc48d5618eb784c8

Sponsored by: Mellanox Technologies

MFC r336395:
Update version information for the mlx5ib module.

Sponsored by: Mellanox Technologies

MFC r336394:
Don't pass unsupported events to ibcore from mlx5ib.

Sponsored by: Mellanox Technologies

MFC r336393:
Use static device naming instead of dynamic one in mlx5ib.

When resetting mlx5core instances it can happen that the order of attach and
detach for mlx5ib instances is changed. Take the unit number for mlx5_%d from
the parent PCI device, similarly to what is done in mlx5en(4), so that there
is a direct relationship between mce<N> and mlx5_<N>.

Sponsored by: Mellanox Technologies

MFC r336392:
Implement support for Differentiated Service Code Point, DSCP, in mlx5en(4).

The DSCP feature is controlled using a set of sysctl(8) fields under
the qos sysctl directory entry for mlx5en(4).

For Routable RoCE QPs, the DSCP should be set in the QP's address path.
The DSCP's value is derived from the traffic class.

Linux commit:
ed88451e1f2d400fd6a743d0a481631cf9f97550

Sponsored by: Mellanox Technologies

MFC r336372:
Add support for prio-tagged traffic for RDMA in ibcore.

When receiving a PCP change all GID entries are reloaded.
This ensures the relevant GID entries use prio tagging,
by setting VLAN present and VLAN ID to zero.

The priority for prio tagged traffic is set using the regular
rdma_set_service_type() function.

Fake the real network device to have a VLAN ID of zero
when prio tagging is enabled. This is logic is hidden inside
the rdma_vlan_dev_vlan_id() function which must always be used
to retrieve the VLAN ID throughout all of ibcore and the
infiniband network drivers.

The VLAN presence information then propagates through all
of ibcore and so incoming connections will have the VLAN
bit set. The incoming VLAN ID is then checked against the
return value of rdma_vlan_dev_vlan_id().

Sponsored by: Mellanox Technologies

MFC r330648:
Add support for explicit congestion notification, ECN, to mlx5ib(4).

ECN configuration and statistics is available through a set of sysctl(8)
nodes under sys.class.infiniband.mlx5_X.cong . The ECN configuration
nodes can also be used as loader tunables.

Sponsored by: Mellanox Technologies

MFC r330647:
Use the autogenerated interface file for all commands in mlx5core.

This patch accumulates the following Linux commits:
- 90b3e38d048f09b22fb50bcd460cea65fd00b2d7
mlx5_core: Modify CQ moderation parameters
- 09a7d9eca1a6cf5eb4f9abfdf8914db9dbd96f08
mlx5_core: QP/XRCD commands via mlx5 ifc
- 1a412fb1caa2c1b77719ccb5ed8b0c3c2bc65da7
mlx5_core: Modify QP commands via mlx5 ifc
- ec22eb53106be1472ba6573dc900943f52f8fd1e
mlx5_core: MKey/PSV commands via mlx5 ifc
- 73b626c182dff06867ceba996a819e8372c9b2ce
mlx5_core: EQ commands via mlx5 ifc
- 20ed51c643b6296789a48adc3bc2cc875a1612cf
mlx5_core: Access register and MAD IFC commands via mlx5 ifc
- a533ed5e179cd15512d40282617909d3482a771c
mlx5_core: Pages management commands via mlx5 ifc
- b8a4ddb2e8f44f872fb93bbda2d541b27079fd2b
mlx5_core: Add MLX5_ARRAY_SET64 to fix BUILD_BUG_ON
- af1ba291c5e498973cc325c501dd8da80b234571
mlx5_core: Refactor internal SRQ API
- b06e7de8a9d8d1d540ec122bbdf2face2a211634
mlx5_core: Refactor device capability function
- c4f287c4a6ac489c18afc4acc4353141a8c53070
mlx5_core: Unify and improve command interface

Submitted by: Matthew Finlay <matt@mellanox.com>
Sponsored by: Mellanox Technologies

MFC r330606:
Implement missing query for current port rate in mlx5ib(4).

This is a direct commit.

Sponsored by: Mellanox Technologies

MFC r330662:
Set correct SL in completion for RoCE in mlx5ib(4).

There is a difference when parsing a completion entry between Ethernet
and IB ports. When link layer is Ethernet the bits describe the type of
L3 header in the packet. In the case when link layer is Ethernet and VLAN
header is present the value of SL is equal to the 3 UP bits in the VLAN
header. If VLAN header is not present then the SL is undefined and consumer
of the completion should check if IB_WC_WITH_VLAN is set.

While that, this patch also fills the vlan_id field in the completion if
present.

linux commit 12f8fedef2ec94c783f929126b20440a01512c14

Sponsored by: Mellanox Technologies

MFC r330597:
Disable unsupported disassociate ucontext functionality in mlx5ib(4).

Sponsored by: Mellanox Technologies

MFC r330508:
Optimize ibcore RoCE address handle creation from user-space.

Creating a UD address handle from user-space or from the kernel-space,
when the link layer is ethernet, requires resolving the remote L3
address into a L2 address. Doing this from the kernel is easy because
the required ARP(IPv4) and ND6(IPv6) address resolving APIs are readily
available. In userspace such an interface does not exist and kernel
help is required.

It should be noted that in an IP-based GID environment, the GID itself
does not contain all the information needed to resolve the destination
IP address. For example information like VLAN ID and SCOPE ID, is not
part of the GID and must be fetched from the GID attributes. Therefore
a source GID should always be referred to as a GID index. Instead of
going through various racy steps to obtain information about the
GID attributes from user-space, this is now all done by the kernel.

This patch optimises the L3 to L2 address resolving using the existing
create address handle uverbs interface, retrieving back the L2 address
as an additional user-space information structure.

This commit combines the following Linux upstream commits:

IB/core: Let create_ah return extended response to user
IB/core: Change ib_resolve_eth_dmac to use it in create AH
IB/mlx5: Make create/destroy_ah available to userspace
IB/mlx5: Use kernel driver to help userspace create ah
IB/mlx5: Report that device has udata response in create_ah

Sponsored by: Mellanox Technologies

MFC r303505, r303506, r303512, r303513, r303646, r320418, r323082,
r326169, r326563, r326649, r326716, r326764, r326765 and r329222:

RoCE/infiniband/iWarp upgrade to Linux 4.9 for kernel and userspace.
This commit merges projects/bsd_rdma_4_9 to 11-stable.

Compatibility wrappers have been made for existing 11-stable ibcore
APIs, including ib_reg_phys_mr().
Refer to "sys/ofed/include/rdma/ib_verbs_compat.h" for more information.

The iw_cxgb driver has not been updated and has been disconnected from
the build.

Sponsored by: Mellanox Technologies

MFC r326169 and r326563:
RoCE/infiniband upgrade to Linux v4.9 for kernel and userspace.

List of kernel sources used:
============================

1) kernel sources were cloned from git://github.com/torvalds/linux.git
Top commit 69973b830859bc6529a7a0468ba0d80ee5117826 - tag: v4.9, linux-4.9

2) krping was cloned from https://github.com/larrystevenwise/krping
Top commit 292a2f1abf0348285e678a82264740d52e4dcfe4

List of userspace sources used:
===============================

1) rdma-core was cloned from https://github.com/linux-rdma/rdma-core.git
Top commit d65138ef93af30b3ea249f3a84aa6a24ba7f8a75

2) OpenSM was cloned from git://git.openfabrics.org/~halr/opensm.git
Top commit 85f841cf209f791c89a075048a907020e924528d

3) libibmad was cloned from git://git.openfabrics.org/~iraweiny/libibmad.git
Tag 1.3.13 with some additional patches from Mellanox.

4) infiniband-diags was cloned from git://git.openfabrics.org/~iraweiny/infiniband-diags.git
Tag 1.6.7 with some additional patches from Mellanox.

NOTES:
======

1) The mthca driver has been removed from userspace.
2) All GPLv2 only sources have been removed and where applicable
rewritten from scratch under a BSD license.
3) List of fully supported drivers in userspace and kernel:
a) iw_cxgbe (Chelsio)
b) mlx4ib (Mellanox)
c) mlx5ib (Mellanox)
4) WITH_OFED=YES is still required by make in order to build
OFED userspace and kernel code.
5) Full support has been added for routable RoCE, RoCE v2.

MFC r326649:
Disconnect OFED after r326169 broke all DIRDEPS support for it.

MFC r326716:
Correctly define the unordered_map namespace in ofed/libibnetdisc .

This should fix ofed/libibnetdisc compilation with C-compilers
different from clang and GCC v4.2.1.

Submitted by: kib
Sponsored by: Mellanox Technologies

MFC r326764:
ofed: Remove duplicated symbols from the version file.

ld.bfd accepts multiple listing of the same symbol in the version script.
lld is stricter and errors out. Since arm64 and sometimes amd64 use lld,
we should correct this cosmetic issue.

Sponsored by: Mellanox Technologies
Reviewed by: hselasky
Differential revision: https://reviews.freebsd.org/D13329

MFC r326765:
ofed: Define barriers for mips and arm.

I used the strongest barriers available on the architectures, so if
the future analysis show that it is excessive, the barriers could be
relaxed. Still, it is unlikely that it is meaningful to run IB on 32bit
ARM or current MIPS machines, so the change is to make WITH_OFED to pass
tinderbox.

Sponsored by: Mellanox Technologies
Reviewed by: hselasky
Differential revision: https://reviews.freebsd.org/D13329

MFC r303505:
sdp: Use an mbufq for received control packets.

This is simpler than the hand-rolled queue, and fixes a use-after-free.

Sponsored by: EMC / Isilon Storage Division

MFC r303506:
sdp: Destroy the PCB lock before freeing to the zone.

Sponsored by: EMC / Isilon Storage Division

MFC r303512:
sdp: Use malloc(9) instead of the Linux compat layer.

SDP transmit and receive rings are always created in a sleepable context,
so we can use M_WAITOK and remove error checks.

Sponsored by: EMC / Isilon Storage Division

MFC r303513:
sdp: Destroy the RDMA ID after destroying the connection's queue pair.

This is the ordering documented by rdma_destroy_qp(). Also add a useful
KASSERT to sdp_pcbfree().

Sponsored by: EMC / Isilon Storage Division

MFC r303646:
ipoib: Bound the number of egress mbufs buffered during pathrec lookups.

In pathological situations where the master subnet manager becomes
unresponsive for an extended period, we may otherwise end up queuing all
of the system's mbufs while waiting for a response to a path record lookup.

This addresses the same issue as commit 1e85b806f9 in Linux.

Reviewed by: cem, ngie
Sponsored by: EMC / Isilon Storage Division

MFC r329222:
Import the mthca kernel side infiniband driver from Linux 4.9 and fix
compilation under FreeBSD. The mthca driver was temporarily removed as
part of the Linux 4.9 RoCE/infinband upgrade.

Top commit in Linux source tree:
69973b830859bc6529a7a0468ba0d80ee5117826

Sponsored by: Mellanox Technologies

MFC r320418. Note that the socket lock _is_ the same as so_rcv's lock
in 11 and this is a no-op in this branch.

Sponsored by: Chelsio Communications

MFC r323082:
cxgbe/iw_cxgbe: Set TCP_NODELAY before initiating connection so that
t4_tom picks it up right away. This is less work than waiting for
the connection to be established before applying the setting.

Sponsored by: Chelsio Communications

MFC r330606:
Implement support for querying the current port rate in mlx5core.
The mlx5ib(4) part will be merged separately.

- Factor out port speed definitions into new port.h header file,
similarly as done in Linux upstream.
- Correct two existing port speed definitions in mlx5en according to
Linux upstream.

Sponsored by: Mellanox Technologies

MFC r325807:
Make sure the ib_wr_opcode enum is signed by adding a negative dummy element.
Different compilers may optimise the enum type in different ways. This ensures
coherency when range checking the value of enums in ibcore.

Sponsored by: Mellanox Technologies

MFC r325662:
Add more and update existing mlx5 core firmware structure definitions and bits.
This change is part of coming ibcore and mlx5ib updates.

Sponsored by: Mellanox Technologies

MFC r324792:
The remote DMA TCP portspace selector, RDMA_PS_TCP, is used for both
iWarp and RoCE in ibcore. The selection of RDMA_PS_TCP can not be used
to indicate iWarp protocol use. Backport the proper IB device
capabilities from Linux upstream to distinguish between iWarp and
RoCE. Only allocate the additional socket required for iWarp for RDMA
IDs when at least one iWarp device present. This resolves
interopability issues between iWarp and RoCE in ibcore

Reviewed by: np @
Differential Revision: https://reviews.freebsd.org/D12563
Sponsored by: Mellanox Technologies

MFC r324491:
Use common rdma_ip2gid() function instead of custom mlx5_ip2gid() one.

Sponsored by: Mellanox Technologies

MFC r322810 and r322830:
Add new mlx5ib(4) driver to the kernel source tree which supports
Remote DMA over Converged Ethernet, RoCE, for the ConnectX-4 series of
PCI express network cards.

There is currently no user-space support and this driver only supports
kernel side non-routable RoCE V1. The krping kernel module can be used
to test this driver. Full user-space support including RoCE V2 will be
added as part of the ongoing upgrade to ibcore from Linux 4.9. Otherwise
this driver is feature equivalent to mlx4ib(4). The mlx5ib(4) kernel
module will only be built when WITH_OFED=YES is specified.

Sponsored by: Mellanox Technologies

Add new mlx5ib(4) driver to the kernel source tree which supports
Remote DMA over Converged Ethernet, RoCE, for the ConnectX-4 series of
PCI express network cards.

There is currently no user-space support and this driver only supports
kernel side non-routable RoCE V1. The krping kernel module can be used
to test this driver. Full user-space support including RoCE V2 will be
added as part of the ongoing upgrade to ibcore from Linux 4.9. Otherwise
this driver is feature equivalent to mlx4ib(4). The mlx5ib(4) kernel
module will only be built when WITH_OFED=YES is specified.

MFC after: 2 weeks
Sponsored by: Mellanox Technologies