Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r258879:
cxgbe(4): T4_SET_SCHED_CLASS and T4_SET_SCHED_QUEUE ioctls to program
scheduling classes in the chip and to bind tx queue(s) to a scheduling
class respectively. These can be used for various kinds of tx traffic
throttling (to force selected tx queues to drain at a fixed Kbps rate,
or a % of the port's total bandwidth, or at a fixed pps rate, etc.).

r253688:
Reserve room for ioctls that aren't in this copy of the driver yet.

MFC all cxgbe(4) changes missing from stable/9:
r248925, r249368, r249370, r249376, r249382, r249383, r249385, r249391,
r249392, r249393, r249627, r249629, r250090, r250092, r250093, r250117,
r250218, r250221, r250614, r251213, r251317, r251358, r251434, r251518,
r251638, r252312, r252469, r252470, r250697(kib).

r248925:
Support for Chelsio's 40G Terminator 5 (aka T5) ASIC.
...

r249368:
Set and display the IP fragment bit correctly when dealing with
the filter mode.

r249370:
cxgbe(4): Ensure that the MOD_LOAD handler runs before either t4nex or
t5nex attach to their devices.

r249376:
- Explain clearly why a different firmware is being installed (if/when
it is being installed). Improve other error messages while here.

- Select special FPGA specific configuration profile when appropriate.

r249382:
There is no need for elaborate queries and error checking when trying to
set FW4MSG_ENCAP.

r249383:
Get rid of a couple of stray \n's.

r249385:
cxgbe/tom: Slight simplification of code that calculates options2.

r249391:
Auto-reduce the holdoff timers that are greater than the maximum value
allowed by the hardware.

r249392:
Cosmetic change (s/wrwc/wcwr/;s/WRWC/WCWR/).

r249393:
Add pciids of the T5 based cards. The ones that I haven't tested with
cxgbe(4) are disabled for now. This will change.

r249627:
cxgbe/tom: Update the CLIP table on the chip when there are changes
to the list of IPv6 addresses on the system. The table is used for
TOE+IPv6 only.

r249629:
cxgbe(4): Refuse to install T5 firmwares on a T4 card (and vice versa).

r250090:
cxgbe(4): Some updates to shared code.

r250092:
- Provide accurate ifmedia information so that 40G ports/transceivers are
displayed properly in ifconfig, etc.

- Use the same number of tx and rx queues for a 40G port as for a 10G port.

r250093:
Attach to the T580 (2 x 40G) card.

r250117:
Fix DDP breakage introduced in r248925. Bitwise OR has higher
precedence than ternary conditional.

r250218:
cxgbe/tom: Do not use M_PROTO1 to mark rx zero-copy mbufs as special.
All the M_PROTOn flags are clobbered when an mbuf is appended to the
socket buffer.

r250221:
cxgbe: Switch to a better way to install firmware.

r250614:
Deal correctly with 40G ports that don't have any transceiver plugged
in. Do not claim that they have unknown tranceivers.

r251213:
cxgbe(4): Some more debug sysctls. These work on both T4 and T5 based
cards.

r251317:
cxgbe(4): t4fw_cfg must be explicitly loaded if the driver is being
loaded via loader.conf.

r251358:
cxgbe(4): Provide accurate hit count for filters on T5 cards. The
location within the TCB and the size have both changed.

r251434:
cxgbe(4): Never install a firmware if hw.cxgbe.fw_install is 0.

r251518:
cxgbe/tom: Fix bad signed/unsigned mixup in the stid allocator. This
fixes a panic when allocating a mixture of IPv6 and IPv4 stids.

r251638:
cxgbe/tom: Allow caller to select the queue (control or data) used to
send the CPL_SET_TCB_FIELD request in t4_set_tcb_field().

r252312:
Update T5 register ranges. This is so that regdump skips over registers
with read side-effects.

r252469:
Add a sysctl to get the number of filters available.

sysctl dev.t4nex.<N>.nfilters
sysctl dev.t5nex.<N>.nfilters

r252470:
Count the number of hits for a filter by default.

r250697:
Add dependencies on the firmware, which allows the loading of the cxgb
and cxgbe modules.

MFC r241397-241399, r241409, r241493-24194.

r241397:
Remove unused item. cxgbe's rx queue's lock was removed a long time ago.

r241398:
There is no need to report the same error twice.

r241399:
Add a driver ioctl to read a byte from any device on a port's i2c bus.
This lets userspace read arbitrary information from the SFP+ modules
etc. on this bus.

Reading multiple bytes in the same transaction isn't possible right now.
I'll update the driver once the chip's firmware supports this.

r241409:
Add a driver ioctl to clear a port's MAC statistics.

r241493:
Use global knob in the TP_PARA_REG3 register to disable congestion
drops if the user has chosen this behaviour.

r241494:
Temporary fix for kern/172364.

MFC r228491, r228561, r228594.

r228491:
Do not clobber the ingress queue's congestion setting.

r228561:
Many updates to cxgbe(4)

- Device configuration via plain text config file. Also able to operate
when not attached to the chip as the master driver.

- Generic "work request" queue that serves as the base for both ctrl and
ofld tx queues.

- Generic interrupt handler routine that can process any event on any
kind of ingress queue (via a dispatch table).

- A couple of new driver ioctls. cxgbetool can now install a firmware
to the card ("loadfw" command) and can read the card's memory
("memdump" and "tcb" commands).

- Lots of assorted information within dev.t4nex.X.misc.* This is
primarily for debugging and won't show up in sysctl -a.

- Code to manage the L2 tables on the chip.

- Updates to cxgbe(4) man page to go with the tunables that have changed.

- Updates to the shared code in common/

- Updates to the driver-firmware interface (now at fw 1.4.16.0)

r228594:
Catch up with new driver ioctls in cxgbe.

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

- driver ioctl to get SGE context for any given queue.
- sysctls to display the context id, cidx, and pidx of all kinds of queues.

MFC after: 3 days

L2 table code. This is enough to get the T4's switch + L2 rewrite
filters working. (All other filters - switch without L2 info rewrite,
steer, and drop - were already fully-functional).

Some contrived examples of "switch" filters with L2 rewriting:

# cxgbetool t4nex0 iport 0 dport 80 action switch vlan +9 eport 3
Intercept all packets received on physical port 0 with TCP port 80 as
destination, insert a vlan tag with VID 9, and send them out of port 3.

# cxgbetool t4nex0 sip 192.168.1.1/32 ivlan 5 action switch \
vlan =9 smac aa:bb:cc:dd:ee:ff eport 0
Intercept all packets (received on any port) with source IP address
192.168.1.1 and VLAN id 5, rewrite the VLAN id to 9, rewrite source mac
to aa:bb:cc:dd:ee:ff, and send it out of port 0.

MFC after: 1 week

T4 packet filtering/steering.

- Enable 5-tuple and every-packet lookup.

- Setup the default filter mode to allow filtering/steering based on IP
protocol, ingress port, inner VLAN ID, IP frag, FCoE, and MPS match
type; all combined together. You can also filter based on MAC index,
Ethernet type, IP TOS/IPv6 Traffic Class, and outer VLAN ID but you'll
have to modify the default filter mode and exclude some of the
match-fields in it.

IPv4 and IPv6 SIP/DIP/SPORT/DPORT are always available in all filter
rules.

- Add driver ioctls to get/set the global filter mode.

- Add driver ioctls to program and delete hardware filters. A couple of
the "switch" actions that rewrite Ethernet and VLAN information and
switch the packet out of another port may not work as the L2 code is not
yet in place. Everything else, including all "drop" and "pass" rules
with RSS or absolute qid, should work.

Obtained from: Chelsio Communications

Modify read/write ioctls to work with 64 bit registers too.

MFC after: 3 days

cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters.

MFC after: 3 weeks