mlx5(4): only detach IOV children if iov was successfully initialized

Reported by: jwd
Sponsored by: NVidia networking
MFC after: 1 week

mlx5: add fs_counters

Signed-off-by: Raed Salem <raeds@nvidia.com>
Sponsored by: NVidia networking
MFC after: 1 week

sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

mlx5: Remove unused debugfs node pointers.

No functional change intended.

MFC after: 1 week
Sponsored by: NVIDIA Networking

mlx5: Implement diagostic counters as sysctl(8) nodes.

MFC after: 1 week
Sponsored by: NVIDIA Networking

mlx5: Make MLX5_COMP_EQ_SIZE tunable.

When using hardware pacing, this value can be increased, because more SQ's
means more EQ events aswell. Make it tunable, hw.mlx5.comp_eq_size .

MFC after: 1 week
Sponsored by: NVIDIA Networking

mlx5/mlx5en: Add SQ remap support

Add support to map an SQ to a specific schedule queue using a
special WQE as performance enhancement.

SQ remap operation is handled by a privileged internal queue, IQ,
and the mapping is enabled from one rate to another.

The transition from paced to non-paced should however always go
through FW.

MFC after: 1 week
Sponsored by: NVIDIA Networking

mlx5: Remove support for FreeBSD 10 and older.

MFC after: 1 week
Sponsored by: NVIDIA Networking

ibcore: Kernel space update based on Linux 5.7-rc1.

Overview:

This is the first stage of a RDMA stack upgrade introducing kernel
changes only based on Linux 5.7-rc1.

This patch is based on about four main areas of work:
- Update of the IB uobjects system:
- The memory holding so-called AH, CQ, PD, SRQ and UCONTEXT objects
is now managed by ibcore. This also require some changes in the
kernel verbs API. The updated verbs changes are typically about
initialize and deinitialize objects, and remove allocation and
free of memory.

- Update of the uverbs IOCTL framework:
- The parsing and handling of user-space commands has been
completely refactored to integrate with the updated IB uobjects
system.

- Various changes and updates to the generic uverbs interfaces in
device drivers including the new uAPI surface.

- The mlx5_ib_devx.c in mlx5ib and related mlx5 core changes.

Dependencies:

- The mlx4ib driver code has been updated with the minimum changes
needed.

- The mlx5ib driver code has been updated with the minimum changes
needed including DV support.

Compatibility:

- All user-space facing APIs are backwards compatible after this
change.

- All kernel-space facing RDMA APIs are backwards compatible after
this change, with exception of ib_create_ah() and ib_destroy_ah()
which takes a new flag.

- The "ib_device_ops" structure exist, but only contains the driver ID
and some structure sizes.

Differences from Linux:

- Infiniband drivers must use the INIT_IB_DEVICE_OPS() macro to set
the sizes needed for allocating various IB objects, when adding
IB device instances.

Security:

- PRIV_NET_RAW is needed to use raw ethernet transmit features.
- PRIV_DRIVER is needed to use other privileged operations.

Based on upstream Linux, Torvalds (5.7-rc1):
8632e9b5645bbc2331d21d892b0d6961c1a08429

MFC after: 1 week
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D31149
Sponsored by: NVIDIA Networking

mlx5: Numa domain improvements.

Properly allocate all mlx5en(4) structures from correct numa domain.

While at it cleanup unused numa domain integers deriving from the
Linux version of mlx5en(4).

MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking

mlx5: Fix for uninitialized "uid" field.

Make sure the "uid" field gets properly set when destroying DCT and QP
objects by making a copy of the field when creating such objects.

MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking

mlx5: Set default timestamp format for mlx5en(4) and mlx5ib.

MFC after: 1 week
Reviewed by: kib
Sponsored by: Mellanox Technologies // NVIDIA Networking

Update user access region, UAR, APIs in the core in mlx5core.

This change include several changes as listed below all related to UAR.
UAR is a special PCI memory area where the so-called doorbell register and
blue flame register live. Blue flame is a feature for sending small packets
more efficiently via a PCI memory page, instead of using PCI DMA.

- All structures and functions named xxx_uuars were renamed into xxx_bfreg.
- Remove partially implemented Blueflame support from mlx5en(4) and mlx5ib.
- Implement blue flame register allocator.
- Use blue flame register allocator in mlx5ib.
- A common UAR page is now allocated by the core to support doorbell register
writes for all of mlx5en and mlx5ib, instead of allocating one UAR per
sendqueue.
- Add support for DEVX query UAR.
- Add support for 4K UAR for libmlx5.

Linux commits:
7c043e908a74ae0a935037cdd984d0cb89b2b970
2f5ff26478adaff5ed9b7ad4079d6a710b5f27e7
0b80c14f009758cefeed0edff4f9141957964211
30aa60b3bd12bd79b5324b7b595bd3446ab24b52
5fe9dec0d045437e48f112b8fa705197bd7bc3c0
0118717583cda6f4f36092853ad0345e8150b286
a6d51b68611e98f05042ada662aed5dbe3279c1e

MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

Make mlx5_cmd_exec_cb() a safe API in mlx5core.

APIs that have deferred callbacks should have some kind of cleanup
function that callers can use to fence the callbacks. Otherwise things
like module unloading can lead to dangling function pointers, or worse.

The IB MR code is the only place that calls this function and had a
really poor attempt at creating this fence. Provide a good version in
the core code as future patches will add more places that need this
fence.

Linux commit:
e355477ed9e4f401e3931043df97325d38552d54

MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

Report EQE data upon CQ completion in mlx5core.

Report EQE data upon CQ completion to let upper layers use this data.

Linux commit:
4e0e2ea1886afe8c001971ff767f6670312a9b04

MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking

mlx5_core: Import PDDR register definitions

PDDR (Port Diagnostics Database Register) is used to read the physical
layer debug database, which contains helpful troubleshooting information
regarding the state of the link.

PDDR register can only be queried when PCAM register reports it as
supported in its register mask. A new helper macro was added to
the MLX5_CAP_* infrastructure in order to access this mask.

Sponsored by: Mellanox Technologies - Nvidia
MFC after: 1 week

mlx5: Restore eswitch management code from attic.

Reviewed by: hselasky
Sponsored by: Mellanox Technologies
MFC after: 2 weeks

Add support for disabling and polling MSIX interrupts in mlx5core.

MFC after: 1 week
Sponsored by: Mellanox Technologies

Widen EPOCH(9) usage in mlx5en(4).

Make completion event path mostly lockless using EPOCH(9).

Implement a mechanism using EPOCH(9) which allows us to make
the callback path for completion events mostly lockless.

Simplify draining callback events using epoch_wait().

While at it make sure all receive completion callbacks are
covered by the network EPOCH(9), because this is required
when calling if_input() and ether_input() after r357012.

Sponsored by: Mellanox Technologies

mlx5: Do not poke hardware for statistic after teardown is started.

Sponsored by: Mellanox Technologies
MFC after: 1 week

Add sysctl(8) to get and set forward error correction, FEC, configuration
in mlx5en(4).

MFC after: 3 days
Sponsored by: Mellanox Technologies

Add definition for the Port Buffer Status Register in mlx5core.

Submitted by: kib@
MFC after: 3 days
Sponsored by: Mellanox Technologies

Sort the ports registers definitions numerically in mlx5core.

Submitted by: kib@
MFC after: 3 days
Sponsored by: Mellanox Technologies

Read rege map from crdump scan space in mlx5core.

Submitted by: kib@
MFC after: 3 days
Sponsored by: Mellanox Technologies

Make the mlx5_vsc_wait_on_flag(9) function global.

Submitted by: kib@
MFC after: 3 days
Sponsored by: Mellanox Technologies

Add port module event software counters in mlx5core.
While at it, fixup PME based on latest PRM defines.

Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies

Cleanup naming of IRQ vectors in mlx5en.
Remove unused IRQ naming functions and arrays.

MFC after: 3 days
Sponsored by: Mellanox Technologies

Add support for Multi-Physical Function Switch, MPFS, in mlx5en.

MPFS is a logical switch in the Mellanox device which forward packets
based on a hardware driven L2 address table, to one or more physical-
or virtual- functions. The physical- or virtual- function is required
to tell the MPFS by using the MPFS firmware commands, which unicast
MAC addresses it is requesting from the physical port's traffic.
Broadcast and multicast traffic however, is copied to all listening
physical- and virtual- functions and does not need a rule in the MPFS
switching table.

Linux commit: eeb66cdb682678bfd1f02a4547e3649b38ffea7e
MFC after: 3 days
Sponsored by: Mellanox Technologies

Make command workqueue persistant in mlx5core.

There is no reason to re-create the command workqueue during healthcare.
This also fixes an issue where a previous work struct may refer to a
destroyed workqueue.

MFC after: 3 days
Sponsored by: Mellanox Technologies

Fix race between driver unload and dumping firmware in mlx5core.

Present code uses lock-less accesses to the dump data to prevent top
level ioctls from blocking bottom-level call to dump. Unfortunately, this
depends on the type stability of the dump data structure, which makes it
non-functional during driver teardown.

Switch to the mutex locking scheme where top levels use the mutex in the
bound regions, while copyouts and drain for completion utilize condvars.
The mutex lifetime is guaranteed to be strictly larger than the time
interval where driver can initiate dump, and most of the control fields
of the old struct mlx5_dump_data are directly embedded into struct
mlx5_core_dev.

Submitted by: kib@
MFC after: 3 days
Sponsored by: Mellanox Technologies

Flush command workqueue when command completion is triggered in mlx5core.

Avoid race for command completion when triggering a command completions event.
Serialize operation by queueing all commands on the same work queue.
This can happen when healthcare triggers.

MFC after: 3 days
Sponsored by: Mellanox Technologies

Make command timeout way shorter in mlx5core.

The command timeout is terribly long, whole two hours. Make it 60s so if
things do go wrong, the user gets feedback in relatively short time, so
they can take corrective actions and/or investigate using tools and such.

Linux commit:
6b6c07bdcdc97ccac2596063bfc32a5faddfe884

MFC after: 3 days
Sponsored by: Mellanox Technologies

Add Firmware Reset Level, MFRL, register accessors in mlx5core.

Submitted by: kib@
MFC after: 3 days
Sponsored by: Mellanox Technologies

Implement reading PCI power status in mlx5core.

Implement a watchdog as part of the healtcare subsystem which
reads the PCI power status during startup and upon the PCI
power status change event and store it into the core device
structure. This value is then exported to user-space via a
read-only SYSCTL. A dmesg print has been added to inform
the admin about the PCI power status.

MFC after: 3 days
Sponsored by: Mellanox Technologies

Move workqueue from mlx5en(4) to mlx5core.

This avoids creating more workqueues in mlx5core to do
simple firmware command polling tasks.

MFC after: 3 days
Sponsored by: Mellanox Technologies

Remove unused speed enums in mlx5core.

Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies

Add MCC (Management Component Control) register definitions in mlx5core.

MCC (Management Component Control) allows to control a firmware
component update.

MCDA (Management Component Data Access) allows to read and write
a firmware component.

MCQI (Management Component Query Information) allows to query
information about firmware components.

Linux commit:
4717628938423fcba0aa8fa889e9fed4eb6a655f

Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies

Query and cache PCAM, MCAM registers on initialization in mlx5core.

On load_one, we now cache our capabilities registers internally, similar
to QUERY_HCA_CAP. Capabilities can later be queried using macros
introduced in this patch.

Linux commit:
71862561f3a62015a11de16d1c306481e8415c08

Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies

Expose PCAM, MCAM registers infrastructure in mlx5core.

PCAM: Ports capabilities mask register.
MCAM: Management capabilities mask register.

PCAM and MCAM registers will provide information regarding firmware
support for different features, in order to avoid cases where new driver
combined with old firmware results in syndromes (for ex. PCIe counters
before this patchset).

Linux commit:
cfdcbceaeffc669b70d904d80a2df9c86c232566

Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies

Protect from infinite sw-reset loop in mlx5core.

Avoid an infinite software firmware reset loop that may be caused by a
hardware bug by limiting the maximum number of resets.
The counter between resets is reset by request for reset, and not by a
successful reset.
The interval between two resets can be configured via sysctl:
hw.mlx5.sw_reset_timeout
which is global to all mlx5 devices in the system.

Submitted by: slavash@
MFC after: 3 days
Sponsored by: Mellanox Technologies

Disable all MSIX interrupts before shutdown in mlx5.

Make sure the interrupt handlers don't race with the fast unload one
code in the shutdown handler.

MFC after: 3 days
Sponsored by: Mellanox Technologies

Correctly define the interface state bits in mlx5en(4).

While at it remove unused interface state bits. This also fixes and issue
during shutdown:

There is an issue where the firmware fails during mlx5_load_one,
the health_care timer detects the issue and schedules a health_care call.
Then the mlx5_load_one detects the issue, cleans up and quits. Then
the health_care starts and calls mlx5_unload_one to clean up the resources
that no longer exist and causes kernel panic.

The root cause is that the bit MLX5_INTERFACE_STATE_DOWN is not set
after mlx5_load_one fails. The solution is removing the bit
MLX5_INTERFACE_STATE_DOWN and quit mlx5_unload_one if the
bit MLX5_INTERFACE_STATE_UP is not set. The bit MLX5_INTERFACE_STATE_DOWN
is redundant and we can use MLX5_INTERFACE_STATE_UP instead.

Linux commit:
10a8d00707082955b177164d4b4e758ffcbd4017
b3cb5388499c5e219324bfe7da2e46cbad82bfcf

MFC after: 3 days
Sponsored by: Mellanox Technologies

mlx5fpga: IOCTL for FPGA temperature measurement

Submitted by: kib@
Approved by: hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies

mlx5fpga: Initial code import.

Submitted by: kib@
Approved by: hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies

mlx5: Fixes to allow command polling mode to exist alongside event mode.

A command is either polling or event driven and the mode cannot change
during execution of a command. Make sure the event handler only handle
commands which are not polled. This is done by checking the command mode
in the command handler before completing commands.

Submitted by: hselasky@
Approved by: hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies

mlx5: Extend vector argument to u64.

Else the MLX5_TRIGGERED_CMD_COMP flag will be masked away.

Submitted by: hselasky@
Approved by: hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies

mlx5: Fix use-after-free in self-healing flow

When the mlx5 health mechanism detects a problem while the driver
is in the middle of init_one or remove_one, the driver needs to prevent
the health mechanism from scheduling future work; if future work
is scheduled, there is a problem with use-after-free: the system WQ
tries to run the work item (which has been freed) at the scheduled
future time.

Prevent this by disabling work item scheduling in the health mechanism
when the driver is in the middle of init_one() or remove_one().

Approved by: hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies

mlx5: Add SRQ fixes from Linux

Combine multiple fixes from Linux to SRQ.
Linux commits:
c73b791 IB/mlx5: Assign SRQ type earlier
0fd27a8 IB/mlx5: Fix out-of-bound access
c2b37f7 IB/mlx5: Fix integer overflows in mlx5_ib_create_srq
d63c467 RDMA/mlx5: Fix memory leak in mlx5_ib_create_srq() error path

Approved by: hselasky (mentor)
MFC after: 1 week
Sponsored by: Mellanox Technologies

Make the MSIX module parameter limit per device, in mlx5en(4).

MFC after: 3 days
Approved by: re (marius)
Sponsored by: Mellanox Technologies

Refactor access to CR-space into using VSC APIs in mlx5core.

Remove no longer used files and APIs.

MFC after: 1 week
Sponsored by: Mellanox Technologies

Implement support for Differentiated Service Code Point, DSCP, in mlx5en(4).

The DSCP feature is controlled using a set of sysctl(8) fields under
the qos sysctl directory entry for mlx5en(4).

For Routable RoCE QPs, the DSCP should be set in the QP's address path.
The DSCP's value is derived from the traffic class.

Linux commit:
ed88451e1f2d400fd6a743d0a481631cf9f97550

MFC after: 1 week
Sponsored by: Mellanox Technologies

Add support for hardware rate limiting to mlx5en(4).

The hardware rate limiting feature is enabled by the RATELIMIT kernel
option. Please refer to ifconfig(8) and the txrtlmt option and the
SO_MAX_PACING_RATE set socket option for more information. This
feature is compatible with hardware transmit send offload, TSO.

A set of sysctl(8) knobs under dev.mce.<N>.rate_limit are provided to
setup the ratelimit table and also to fine tune various rate limit
related parameters.

Sponsored by: Mellanox Technologies

Remove unused structure field in mlx5core.

MFC after: 3 days
Sponsored by: Mellanox Technologies

Remove redundant prototype to fix compilation with GCC.

Reported by: jeff@
MFC after: 1 week
Sponsored by: Mellanox Technologies

Add mutual exclusion mechanism for software reset of firmware in mlx5core.

Since the FW can be shared between PCI functions it is common that
more than one health poll will detected a failure, this can lead to
multiple resets.

The solution is to use a FW locking mechanism using semaphore space to
provide a way to synchronize between functions. The FW semaphore is
acquired via config cycle access. First the VSEC gateway must be
acquired, then the semaphore can be locked by writing a value to it
and confirmed it's locked by reading the same value back. The process
in the same to free the semaphore, except the value written should be
zero.

Submitted by: slavash@
MFC after: 1 week
Sponsored by: Mellanox Technologies

Handle software reset of firmware in error flow in mlx5core.

Some mlx5 adapter firmware allows the driver to reset the firmware in
the event of an error. When a software reset is issued on any physical
function all PFs enter reset state. This is a recoverable condition.
The existing recovery flow was designed to allow the recovery of a
VF after a PF driver reload. This patch expands the scope of that
flow to recover PFs or VFs after a SW reset has been issued.
When a software reset is issued the following occurs:

1. The NIC interface mode is set to SW_RESET (7) while the reset is in
progress.
2. Once the reset completes the NIC interface mode is set to NIC
disabled (1).

After the reset has been issued (added in a subsequent patch) the
health poll for other functions will detect that the NIC interface
state has been set to disabled. This will cause it to enter the
existing recovery flow. If the PCI is still working (meaning it
doesn't return 0xff on all reads) it means recovery can proceed
immediately instead of waiting 60 seconds.

The error detetion has also been refactored to avoid incorrect or
misleading log messages.

Submitted by: slavash@
MFC after: 1 week
Sponsored by: Mellanox Technologies

Cancel delayed recovery work when unloading the mlx5core driver.

linux commit 2a0165a034ac024b60cca49c61e46f4afa2e4d98

Submitted by: Matthew Finlay <matt@mellanox.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies

Add support for fast unload in shutdown flow in mlx5core.

This patch accumulates the following Linux commits:

- 8812c24d28f4972c4f2b9998bf30b1f2a1b62adf
net/mlx5: Add fast unload support in shutdown flow
- 59211bd3b6329c3e5f4a90ac3d7f87ffa7867073
net/mlx5: Split the load/unload flow into hardware and software flows
- 4525abeaae54560254a1bb8970b3d4c225d32ef4
net/mlx5: Expose command polling interface

Submitted by: Matthew Finlay <matt@mellanox.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies

Improve support for health recovery in mlx5core.

This patch accumulates the following Linux commits:

- 04c0c1ab38e95105d950db5b84e727637e149ce7
net/mlx5: PCI error recovery health care simulation
- 0179720d6be2096b8d0a4d143254ff9e77747daa
net/mlx5: Introduce trigger_health_work function
- 3fece5d676939f42f434c63dfe1bd42d7d94e6f0
net/mlx5: Continue health polling until it is explicitly stopped

Submitted by: Matthew Finlay <matt@mellanox.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies

mlx5(4): Remove redundant declaration of mlx5_enter_error_state

Broken in r330644.

Sponsored by: Dell EMC Isilon

Add kernel and userspace code to dump the firmware state of supported
ConnectX-4/5 devices in mlx5core.

The dump is obtained by reading a predefined register map from the
non-destructive crspace, accessible by the vendor-specific PCIe
capability (VSC). The dump is stored in preallocated kernel memory and
managed by the mlx5tool(8), which communicates with the driver using a
character device node.

The utility allows to store the dump in format
<address> <value>
into a file, to reset the dump content, and to manually initiate the
dump.

A call to mlx5_fwdump() should be added at the places where a dump
must be fetched automatically. The most likely place is right before a
firmware reset request.

Submitted by: kib@
MFC after: 1 week
Sponsored by: Mellanox Technologies

Add vendor specific capability interface support in mlx5core.

Add the ability to access the vendor specific space gateway in order
to support reading and writing data into the different configuration
domains.

Submitted by: Matthew Finlay <matt@mellanox.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies

Use the autogenerated interface file for all commands in mlx5core.

This patch accumulates the following Linux commits:
- 90b3e38d048f09b22fb50bcd460cea65fd00b2d7
mlx5_core: Modify CQ moderation parameters
- 09a7d9eca1a6cf5eb4f9abfdf8914db9dbd96f08
mlx5_core: QP/XRCD commands via mlx5 ifc
- 1a412fb1caa2c1b77719ccb5ed8b0c3c2bc65da7
mlx5_core: Modify QP commands via mlx5 ifc
- ec22eb53106be1472ba6573dc900943f52f8fd1e
mlx5_core: MKey/PSV commands via mlx5 ifc
- 73b626c182dff06867ceba996a819e8372c9b2ce
mlx5_core: EQ commands via mlx5 ifc
- 20ed51c643b6296789a48adc3bc2cc875a1612cf
mlx5_core: Access register and MAD IFC commands via mlx5 ifc
- a533ed5e179cd15512d40282617909d3482a771c
mlx5_core: Pages management commands via mlx5 ifc
- b8a4ddb2e8f44f872fb93bbda2d541b27079fd2b
mlx5_core: Add MLX5_ARRAY_SET64 to fix BUILD_BUG_ON
- af1ba291c5e498973cc325c501dd8da80b234571
mlx5_core: Refactor internal SRQ API
- b06e7de8a9d8d1d540ec122bbdf2face2a211634
mlx5_core: Refactor device capability function
- c4f287c4a6ac489c18afc4acc4353141a8c53070
mlx5_core: Unify and improve command interface

Submitted by: Matthew Finlay <matt@mellanox.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies

Fix race between PCI error handlers and health work in mlx5core.

linux commit 05ac2c0b7438ea08c5d54b48797acf9b22cb2f6f

Submitted by: Matthew Finlay <matt@mellanox.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies

Updates for PCI and health monitor recovery in mlx5core.
This patch accumulates the following Linux commits:

mlx5_health.c
- 78ccb25861d76a8fc5c678d762180e6918834200
mlx5_core: Fix wrong name in struct
- 171bb2c560f45c0427ca3776a4c8f4e26e559400
mlx5_core: Update health syndromes
- 0144a95e2ad53a40c62148f44fb0c1f9d2a0d1e9
mlx5_core: Use accessor functions to read from device memory
- ac6ea6e81a80172612e0c9ef93720f371b198918
mlx5_core: Use private health thread for each device
- fd76ee4da55abb21babfc69310d321b9cb9a32e0
mlx5_core: Fix internal error detection conditions
- 2241007b3d783cbdbaa78c30bdb1994278b6f9b9
mlx5: Clear health sick bit when starting health poll
- 712bfef60912d91033cb25739f7444d5b8d8c59f
mlx5: Fix version printout in case of health issue
- 89d44f0a6c732db23b219be708e2fe1e03ee4842
mlx5_core: Add pci error handlers to mlx5_core driver

mlx5_cmd.c
- be87544de8df2b1eb34bcb5e32691287d96f9ec4
mlx5_core: Fix async commands return code
- a31208b1e11df334d443ec8cace7636150bb8ce2
mlx5_core: New init and exit flow for mlx5_core
- 020446e01eebc9dbe7eda038e570ab9c7ab13586
mlx5_core: Prepare cmd interface to system errors handling
- 89d44f0a6c732db23b219be708e2fe1e03ee4842
mlx5_core: Add pci error handlers to mlx5_core driver
- 0d834442cc247c7b3f3bd6019512ae03e96dd99a
mlx5: Fix teardown errors that happen in pci error handler

mlx5_main.c
- 5fc7197d3a256d9c5de3134870304b24892a4908
mlx5: Add pci shutdown callback

Submitted by: Matthew Finlay <matt@mellanox.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies

Implement missing query for current port rate in mlx5ib(4).

- Factor out port speed definitions into new port.h header file,
similarly as done in Linux upstream.
- Correct two existing port speed definitions in mlx5en according to
Linux upstream.

MFC after: 1 week
Sponsored by: Mellanox Technologies

Add timeout handle to commands with callback in mlx5core.

The current implementation does not handle timeout in case of command
with callback request, and this can lead to deadlock if the command
doesn't get firmware response. Add delayed callback timeout work
before posting the command to firmware. In case of real firmware
command completion we will cancel the delayed work. In case of
firmware command timeout the callback timeout handler will be called
and it will simulate firmware completion with timeout error.

linux commit 65ee67084589c1783a74b4a4a5db38d7264ec8b5

MFC after: 1 week
Sponsored by: Mellanox Technologies

Add more and update existing mlx5 core firmware structure definitions and bits.
This change is part of coming ibcore and mlx5ib updates.

Sponsored by: Mellanox Technologies
MFC after: 1 week

Make physical address of init segment available in the priv of mlx5 core.
This change is needed by mlx5ib(4).

Sponsored by: Mellanox Technologies
MFC after: 1 week

Add API functions to set and query dropless port mode in mlx5 core.

Sponsored by: Mellanox Technologies
MFC after: 1 week

Add const keyword to input-only argument in mlx5 core.

Sponsored by: Mellanox Technologies
MFC after: 1 week

Refactor the flowsteering APIs used by mlx5en(4). This change is needed by
the coming ibcore and mlx5ib updates in order to support traffic redirection
to so-called raw ethernet QPs.

Remove unused E-switch related routines and files while at it.

Sponsored by: Mellanox Technologies
MFC after: 1 week

Use the busdma API to allocate all DMA-able memory.

The MLX5 driver has four different types of DMA allocations which are
now allocated using busdma:

1) The 4K firmware DMA-able blocks. One busdma object per 4K allocation.
2) Data for firmware commands use the 4K firmware blocks split into four 1K blocks.
3) The 4K firmware blocks are also used for doorbell pages.
4) The RQ-, SQ- and CQ- DMA rings. One busdma object per allocation.

After this patch the mlx5en driver can be used with DMAR enabled in
the FreeBSD kernel.

MFC after: 1 week
Sponsored by: Mellanox Technologies

Add support for device surprise removal and other PCI errors.

- When device disappears from PCI indicate error device state and:
1) Trigger command completion for all pending commands
2) Prevent new commands from executing and return:
- success for modify and remove/cleanup commands
- failure for create/query commands
3) When reclaiming pages for a device in error state don't ask FW to
return all given pages, just release the allocated memory

MFC after: 1 week
Sponsored by: Mellanox Technologies

Wait for all VFs pages to be reclaimed before closing EQ pages.

MFC after: 1 week
Sponsored by: Mellanox Technologies

Use ffs() to scan for first bit instead of using a for() loop.
Minor code refactor while at it.

MFC after: 1 week
Sponsored by: Mellanox Technologies

Make fw_pages statistics counter 64-bit to avoid overflow.

MFC after: 1 week
Sponsored by: Mellanox Technologies

Add more firmware related structures and update existing ones in the
MLX5 core module. Update the set and query diagnostics counter API.

Sponsored by: Mellanox Technologies
MFC after: 1 week

Update the MLX5 core module:
- Add new firmware commands and update existing ones.
- Add more firmware related structures and update existing ones.
- Some minor fixes, like adding missing \n to some prints.

Sponsored by: Mellanox Technologies
MFC after: 1 week

Add SR-IOV guest support to the mlx5en driver.

This patch adds the missing pieces needed for device setup using the
mlx5en driver inside a virtual machine which is providing hardware
access through SR-IOV.

Sponsored by: Mellanox Technologies
MFC after: 1 week

Add function to detect the presence of a port module and use this
function to error out early when no port module is present and doing
eeprom access. This also prevents error codes from filling up in
dmesg.

Sponsored by: Mellanox Technologies
Tested by: Netflix
MFC after: 1 week

Update the mlx5 shared driver code to the latest version, which
include the following list of changes:

- Added eswitch ACL table management
Introduce API for managing ACL table.
This API include the following features:
1) vlan filter - for VST/VGT+ support.
2) spoofcheck.
3) robust functionality to allow/drop general untagged/tagged traffic.
4) support for both ingress and egress ACL types.

- Added loopback filter to the vacl table.

- Added multicast list set in the vPort context

- Added promiscuous mode set in the vPort context

- Set the vlan list in vPort context
1) Check caps if VLAN list is not longer than FW supports
2) Set MODIFY_NIC_VPORT_CONTEXT command

- Changed MLX5_EEPROM_MAX_BYTES from 48 to 32 so that a single EEPROM
reading cannot cross the 128-byte boundary. Previously reading the
MCIA register was done in batches of 48 bytes. The third reading
would then by-pass the 127th byte, which means that part of the low
page and part of the high page would be read at the same time, which
created a bug:
1st: 0-47 bytes
2nd: 48-95 bytes
3rd: 96-143 bytes

MFC after: 1 week
Sponsored by: Mellanox Technologies
Differential Revision: https://reviews.freebsd.org/D4411

Add mlx5 and mlx5en driver(s) for ConnectX-4 and ConnectX-4LX cards
from Mellanox Technologies. The current driver supports ethernet
speeds up to and including 100 GBit/s. Infiniband support will be
done later.

The code added is not compiled by default, which will be done by a
separate commit.

Sponsored by: Mellanox Technologies
MFC after: 2 weeks