MFC 9a47ae044b48:
Bump driver versions for mlx5en(4) and mlx4en(4).

Sponsored by: Mellanox Technologies // NVIDIA Networking

Git Hash: e87e3e82f3a062856118ed42751b498277eb09a5
Git Author: hselasky@FreeBSD.org

MFC r367555:
Include GID type when deleting GIDs from HW table under RoCE in mlx4ib.
Refer to the Linux commit mentioned below for a more detailed description.

Linux commit:
a18177925c252da7801149abe217c05b80884798

Requested by: Isilon
Sponsored by: Mellanox Technologies // NVIDIA Networking

MFC r362953:
Infiniband clients must be attached and detached in a specific order in ibcore.

Currently the linking order of the infiniband, IB, modules decide in which
order the clients are attached and detached. For example one IB client may
use resources from another IB client. This can lead to a potential deadlock
at shutdown. For example if the ipoib is unregistered after the ib_multicast
client is detached, then if ipoib is using multicast addresses a deadlock may
happen, because ib_multicast will wait for all its resources to be freed before
returning from the remove method.

Fix this by using module_xxx_order() instead of module_xxx().

Differential Revision: https://reviews.freebsd.org/D23973
Sponsored by: Mellanox Technologies

MFC r362271:
Allow multicast packets to be received in promiscious mode, in mlx4en(4).

Make sure we disable the multicast filter in promiscious mode aswell as when
the all multicast flag is set.

Found by: Tycho Nightingale <tychon@freebsd.org>
Sponsored by: Mellanox Technologies

MFC r362201:

Avoid trying to toggle TSO twice

Remove TSO from the toggle mask when automatically disabled by TXCKSUM* in
various NIC drivers.

Reviewed by: hselasky, np, gallatin, jpaetzel
Approved by: mav (mentor)
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25120

MFC r347325:
Bump the Mellanox driver version numbers and the FreeBSD version number.

Sponsored by: Mellanox Technologies

MFC r347258:
Make sure to error out when arming the CQ fails in mlx4ib and mlx5ib.

Sponsored by: Mellanox Technologies

MFC r345011:
Eliminate useless warning message when reading sysctl node in mlx4core.

Sponsored by: Mellanox Technologies

MFC r345010:
Improve support for switching to and from command polling mode in mlx4core.

Make sure the enter and leave polling routines can be called multiple times
with same setting. Ignore setting polling or event mode twice. This fixes a
deadlock during shutdown if polling mode was already selected.

Sponsored by: Mellanox Technologies

MFC r344920:
Teardown ifnet after stopping port in the mlx4en(4) driver.

mlx4_en_stop_port() calls mlx4_en_put_qp() which can refer the link level
address of the network interface, which in turn will be freed by the
network interface detach function. Make sure the port is stopped
before detaching the network interface.

Sponsored by: Mellanox Technologies

MFC r344919:
Don't hold state lock while detaching network device instance in mlx4en(4).

It can happen during shutdown that the lock will recurse when the mlx4en(4)
instance is part of a lagg interface. Call ether_ifdetach() unlocked.

Backtrace:
panic(): _sx_xlock_hard: recursed on non-recursive sx &mdev->state_lock
_sx_xlock_hard()
_sx_xlock()
mlx4_en_ioctl()
if_setlladdr()
lagg_port_destroy()
lagg_port_ifdetach()
if_detach()
mlx4_en_destroy_netdev()
mlx4_en_remove()
mlx4_remove_device()
mlx4_unregister_device()
mlx4_unload_one()
mlx4_shutdown()
linux_pci_shutdown()
bus_generic_shutdown()

Sponsored by: Mellanox Technologies

MFC r341587:
mlx4/mlx5: Updated driver version to 3.5.0

Sponsored by: Mellanox Technologies

MFC r341552:
mlx4en: Optimise reception of small packets.

Copy small packets like TCP ACKs into a new mbuf
reusing the existing mbuf to receive a new ethernet
frame. This avoids wasting buffer space for
small sized packets.

Sponsored by: Mellanox Technologies

MFC r341551:
mlx4: Make sure default VNET is set when adding a new interface.

Adding an interface might be done outside the device_attach() routine
and will then cause a panic, due to the VNET not being defined.

Sponsored by: Mellanox Technologies

MFC r341550:
mlx4en: Remove duplicate statistics variable assignment.

The "priv->pkstats.rx_dropped" is written twice in a row.

Sponsored by: Mellanox Technologies

MFC r341549:
mlx4en: Add support for receiving all data using one or more MCLBYTES sized mbufs.
Also when the MTU is greater than MCLBYTES.

Sponsored by: Mellanox Technologies

MFC r341547:
mlx4en: Remove the DRBR and associated logic in the transmit path.

The hardware queues are deep enough currently and using the DRBR and associated
callbacks only leads to more task switching in the TX path. The is also a race
setting the queue_state which can lead to hung TX rings.

Sponsored by: Mellanox Technologies

MFC r341546:
mlx4en: Add driver version to sysctl desc

Sponsored by: Mellanox Technologies

MFC r341545:
mlx4: Add board identifier and firmware version to sysctl

In last mlx4 update (r325841) we lost the sysctl to show the
firmware version for mlx4 devices.
Add both board identifier and firmware version under:
sys.device.mlx4_core0.hw sysctl node.

Sponsored by: Mellanox Technologies

MFC r341544:
mlx4core: Add checks for invalid port numbers.

Sponsored by: Mellanox Technologies

MFC r341543:
mlx4: Zero initialize device capabilities to avoid use of uninitialized fields.

Sponsored by: Mellanox Technologies

MFC r341542:
mlx4core: Avoid multiplication overflow by casting multiplication.

Sponsored by: Mellanox Technologies

MFC r339235:
Add missing steering rules for virtual function, VF, in mlx4en(4) driver.

When acting as a VF it is required to add steering rules for all unicast
addresses. Even if promiscious mode is selected. Else incoming data packets
will be dropped.

Sponsored by: Mellanox Technologies

MFC r336372:
Add support for prio-tagged traffic for RDMA in ibcore.

When receiving a PCP change all GID entries are reloaded.
This ensures the relevant GID entries use prio tagging,
by setting VLAN present and VLAN ID to zero.

The priority for prio tagged traffic is set using the regular
rdma_set_service_type() function.

Fake the real network device to have a VLAN ID of zero
when prio tagging is enabled. This is logic is hidden inside
the rdma_vlan_dev_vlan_id() function which must always be used
to retrieve the VLAN ID throughout all of ibcore and the
infiniband network drivers.

The VLAN presence information then propagates through all
of ibcore and so incoming connections will have the VLAN
bit set. The incoming VLAN ID is then checked against the
return value of rdma_vlan_dev_vlan_id().

Sponsored by: Mellanox Technologies

MFC r331797:

Use an accessor function to access ifr_data.

This fixes 32-bit compat (no ioctl command defintions are required
as struct ifreq is the same size).

Reviewed by: kib
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14900

MFC r330596:
Bump version information in mlx4ib(4).

Sponsored by: Mellanox Technologies

MFC r330595:
The mlx4ib(4) should not be loaded before the ibcore is initialized.

Sponsored by: Mellanox Technologies

MFC r330594:
Disable unsupported disassociate ucontext functionality in mlx4ib(4).

Sponsored by: Mellanox Technologies

MFC r330508:
Optimize ibcore RoCE address handle creation from user-space.

Creating a UD address handle from user-space or from the kernel-space,
when the link layer is ethernet, requires resolving the remote L3
address into a L2 address. Doing this from the kernel is easy because
the required ARP(IPv4) and ND6(IPv6) address resolving APIs are readily
available. In userspace such an interface does not exist and kernel
help is required.

It should be noted that in an IP-based GID environment, the GID itself
does not contain all the information needed to resolve the destination
IP address. For example information like VLAN ID and SCOPE ID, is not
part of the GID and must be fetched from the GID attributes. Therefore
a source GID should always be referred to as a GID index. Instead of
going through various racy steps to obtain information about the
GID attributes from user-space, this is now all done by the kernel.

This patch optimises the L3 to L2 address resolving using the existing
create address handle uverbs interface, retrieving back the L2 address
as an additional user-space information structure.

This commit combines the following Linux upstream commits:

IB/core: Let create_ah return extended response to user
IB/core: Change ib_resolve_eth_dmac to use it in create AH
IB/mlx5: Make create/destroy_ah available to userspace
IB/mlx5: Use kernel driver to help userspace create ah
IB/mlx5: Report that device has udata response in create_ah

Sponsored by: Mellanox Technologies

MFC r303505, r303506, r303512, r303513, r303646, r320418, r323082,
r326169, r326563, r326649, r326716, r326764, r326765 and r329222:

RoCE/infiniband/iWarp upgrade to Linux 4.9 for kernel and userspace.
This commit merges projects/bsd_rdma_4_9 to 11-stable.

Compatibility wrappers have been made for existing 11-stable ibcore
APIs, including ib_reg_phys_mr().
Refer to "sys/ofed/include/rdma/ib_verbs_compat.h" for more information.

The iw_cxgb driver has not been updated and has been disconnected from
the build.

Sponsored by: Mellanox Technologies

MFC r326169 and r326563:
RoCE/infiniband upgrade to Linux v4.9 for kernel and userspace.

List of kernel sources used:
============================

1) kernel sources were cloned from git://github.com/torvalds/linux.git
Top commit 69973b830859bc6529a7a0468ba0d80ee5117826 - tag: v4.9, linux-4.9

2) krping was cloned from https://github.com/larrystevenwise/krping
Top commit 292a2f1abf0348285e678a82264740d52e4dcfe4

List of userspace sources used:
===============================

1) rdma-core was cloned from https://github.com/linux-rdma/rdma-core.git
Top commit d65138ef93af30b3ea249f3a84aa6a24ba7f8a75

2) OpenSM was cloned from git://git.openfabrics.org/~halr/opensm.git
Top commit 85f841cf209f791c89a075048a907020e924528d

3) libibmad was cloned from git://git.openfabrics.org/~iraweiny/libibmad.git
Tag 1.3.13 with some additional patches from Mellanox.

4) infiniband-diags was cloned from git://git.openfabrics.org/~iraweiny/infiniband-diags.git
Tag 1.6.7 with some additional patches from Mellanox.

NOTES:
======

1) The mthca driver has been removed from userspace.
2) All GPLv2 only sources have been removed and where applicable
rewritten from scratch under a BSD license.
3) List of fully supported drivers in userspace and kernel:
a) iw_cxgbe (Chelsio)
b) mlx4ib (Mellanox)
c) mlx5ib (Mellanox)
4) WITH_OFED=YES is still required by make in order to build
OFED userspace and kernel code.
5) Full support has been added for routable RoCE, RoCE v2.

MFC r326649:
Disconnect OFED after r326169 broke all DIRDEPS support for it.

MFC r326716:
Correctly define the unordered_map namespace in ofed/libibnetdisc .

This should fix ofed/libibnetdisc compilation with C-compilers
different from clang and GCC v4.2.1.

Submitted by: kib
Sponsored by: Mellanox Technologies

MFC r326764:
ofed: Remove duplicated symbols from the version file.

ld.bfd accepts multiple listing of the same symbol in the version script.
lld is stricter and errors out. Since arm64 and sometimes amd64 use lld,
we should correct this cosmetic issue.

Sponsored by: Mellanox Technologies
Reviewed by: hselasky
Differential revision: https://reviews.freebsd.org/D13329

MFC r326765:
ofed: Define barriers for mips and arm.

I used the strongest barriers available on the architectures, so if
the future analysis show that it is excessive, the barriers could be
relaxed. Still, it is unlikely that it is meaningful to run IB on 32bit
ARM or current MIPS machines, so the change is to make WITH_OFED to pass
tinderbox.

Sponsored by: Mellanox Technologies
Reviewed by: hselasky
Differential revision: https://reviews.freebsd.org/D13329

MFC r303505:
sdp: Use an mbufq for received control packets.

This is simpler than the hand-rolled queue, and fixes a use-after-free.

Sponsored by: EMC / Isilon Storage Division

MFC r303506:
sdp: Destroy the PCB lock before freeing to the zone.

Sponsored by: EMC / Isilon Storage Division

MFC r303512:
sdp: Use malloc(9) instead of the Linux compat layer.

SDP transmit and receive rings are always created in a sleepable context,
so we can use M_WAITOK and remove error checks.

Sponsored by: EMC / Isilon Storage Division

MFC r303513:
sdp: Destroy the RDMA ID after destroying the connection's queue pair.

This is the ordering documented by rdma_destroy_qp(). Also add a useful
KASSERT to sdp_pcbfree().

Sponsored by: EMC / Isilon Storage Division

MFC r303646:
ipoib: Bound the number of egress mbufs buffered during pathrec lookups.

In pathological situations where the master subnet manager becomes
unresponsive for an extended period, we may otherwise end up queuing all
of the system's mbufs while waiting for a response to a path record lookup.

This addresses the same issue as commit 1e85b806f9 in Linux.

Reviewed by: cem, ngie
Sponsored by: EMC / Isilon Storage Division

MFC r329222:
Import the mthca kernel side infiniband driver from Linux 4.9 and fix
compilation under FreeBSD. The mthca driver was temporarily removed as
part of the Linux 4.9 RoCE/infinband upgrade.

Top commit in Linux source tree:
69973b830859bc6529a7a0468ba0d80ee5117826

Sponsored by: Mellanox Technologies

MFC r320418. Note that the socket lock _is_ the same as so_rcv's lock
in 11 and this is a no-op in this branch.

Sponsored by: Chelsio Communications

MFC r323082:
cxgbe/iw_cxgbe: Set TCP_NODELAY before initiating connection so that
t4_tom picks it up right away. This is less work than waiting for
the connection to be established before applying the setting.

Sponsored by: Chelsio Communications

MFC r330537:
Add mapping for several ethernet types used by Linux to FreeBSD
ethernet types.

Reviewed by: hselasky
Differential Revision: https://reviews.freebsd.org/D14594

MFC r326666:
mlx4: Remove redundant declarations to fix GCC build

These were made redundant in r325841.

Reviewed by: hselasky
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D13401

MFC r306486-r325841:
Update the mlx4, mlx4ib(4) and mlx4en(4) drivers to match FreeBSD-head
prior to the Linux 4.9 RoCE/infiniband upgrade.

Sponsored by: Mellanox Technologies

MFC r306486:
Move the ConnectX-3 and ConnectX-2 driver from sys/ofed into sys/dev/mlx4
like other PCI network drivers. The sys/ofed directory is now mainly
reserved for generic infiniband code, with exception of the mthca driver.

- Add new manual page, mlx4en(4), describing how to configure and load
mlx4en.

- All relevant driver C-files are now prefixed mlx4, mlx4_en and
mlx4_ib respectivly to avoid object filename collisions when compiling
the kernel. This also fixes an issue with proper dependency file
generation for the C-files in question.

- Device mlxen is now device mlx4en and depends on device mlx4, see
mlx4en(4). Only the network device name remains unchanged.

- The mlx4 and mlx4en modules are now built by default on i386 and
amd64 targets. Only building the mlx4ib module depends on
WITH_OFED=YES .

Sponsored by: Mellanox Technologies

MFC r310425:
mlx: avoid use of __DATE__ to make build reproducible

Reviewed by: hselasky
Differential Revision: https://reviews.freebsd.org/D8886

MFC r310488:
mlx(4): remove date from log message

Further to r310425, go one step further and just remove the date.

Reviewed by: hselasky
Differential Revision: https://reviews.freebsd.org/D8888

MFC r324200:
Implement SIOCGIFRSS{KEY,HASH} for the mlx4en(4) driver.

Differential Revision: https://reviews.freebsd.org/D12176
Sponsored by: Mellanox Technologies

MFC r324201:
Setup mbuf hash type properly when receiving IP packets in the mlx4en(4) driver.

Submitted by: sephe@
Differential Revision: https://reviews.freebsd.org/D12229
Sponsored by: Mellanox Technologies

MFC r324621:
mlx4: use enum constants instead of const vars for case exprs

Follow up from r324201 to fix compilation with gcc, which complains
about non-ICE case expressions.

Reviewed by: hselasky
Differential Revision: https://reviews.freebsd.org/D12675

MFC r325841:
Update the mlx4 core and mlx4en(4) modules towards Linux v4.9.

Background:
The coming ibcore update forces an update of mlx4ib(4) which in turn requires
an updated mlx4 core module. This also affects the mlx4en(4) module because
commonly used APIs are updated. This commit is a middle step updating the
mlx4 modules towards the new ibcore.

This change contains no major new features.

Changes in mlx4:
a) Improved error handling when mlx4 PCI devices are
detached inside VMs.
b) Major update of codebase towards Linux 4.9.

Changes in mlx4ib(4):
a) Minimal changes needed in order to compile using the
updated mlx4 core APIs.

Changes in mlx4en(4):
a) Update flow steering code in mlx4en to use new APIs for
registering MAC addresses and IP addresses.
b) Update all statistics counters to be 64-bit.
c) Minimal changes needed in order to compile using the
updated mlx4 core APIs.

Sponsored by: Mellanox Technologies

Move the ConnectX-3 and ConnectX-2 driver from sys/ofed into sys/dev/mlx4
like other PCI network drivers. The sys/ofed directory is now mainly
reserved for generic infiniband code, with exception of the mthca driver.

- Add new manual page, mlx4en(4), describing how to configure and load
mlx4en.

- All relevant driver C-files are now prefixed mlx4, mlx4_en and
mlx4_ib respectivly to avoid object filename collisions when compiling
the kernel. This also fixes an issue with proper dependency file
generation for the C-files in question.

- Device mlxen is now device mlx4en and depends on device mlx4, see
mlx4en(4). Only the network device name remains unchanged.

- The mlx4 and mlx4en modules are now built by default on i386 and
amd64 targets. Only building the mlx4ib module depends on
WITH_OFED=YES .

Sponsored by: Mellanox Technologies