History log of /freebsd-current/sys/dev/cxgbe/t4_main.c
Revision Date Author Comments
# a1b84451 04-Feb-2024 Mark Johnston <markj@FreeBSD.org>

cxgbe: Use device_set_descf()

No functional change intended.

MFC after: 1 week


# 7f10048f 30-Apr-2024 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Query TPCHMAP once and not once per port.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 480ff89c 30-Apr-2024 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Rename rx_c_chan to rx_chan.

It is the equivalent of tx_chan but for receive so rx_chan is a better
name. Initialize both using helper functions and make sure both are
displayed in the sysctl MIB.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 21aba396 30-Apr-2024 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Minor tweaks to comments.

No functional change intended.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 9de0036b 30-Apr-2024 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Initialize mps_bg_map to an invalid value if it's not known.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 857d74b6 30-Apr-2024 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Allocate a taskqueue per port instead of per channel.

All the channels are not used on all boards and there's no point
allocating taskqueues that will never be used.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 43f6f084 01-Apr-2023 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Reword the comment explaining the atid/cookie split.

Avoid a magic constant while here. No functional change intended.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 1c7f9c8b 25-Apr-2024 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Retire t4_intr_clear.

The firmware clears the interrupts already and it has a better idea of
exactly what to clear for which generation of the ASIC. There is no
need for the driver to get involved.

MFC after: 1 week
Sponsored by: Chelsio Communications


# b59c5d97 24-Apr-2024 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add a helper function to locate MPS/MAC registers.

These register blocks are at different locations in different chips.

MFC after: 1 week
Sponsored by: Chelsio Communications


# eba13bbc 20-Mar-2024 John Baldwin <jhb@FreeBSD.org>

cxgbe: Support TCP_USE_DDP on offloaded TOE connections

When this socket option is enabled, relatively large contiguous
buffers are allocated and used to receive data from the remote
connection. When data is received a wrapper M_EXT mbuf is queued to
the socket's receive buffer. This reduces the length of the linked
list of received mbufs and allows consumers to consume receive data in
larger chunks.

To minimize reprogramming the page pods in the adapter, receive
buffers for a given connection are recycled. When a buffer has been
fully consumed by the receiver and freed, the buffer is placed on a
per-connection free buffers list.

The size of the receive buffers defaults to 256k and can be set via
the hw.cxgbe.toe.ddp_rcvbuf_len sysctl. The
hw.cxgbe.toe.ddp_rcvbuf_cache sysctl (defaults to 4) determines the
maximum number of free buffers cached per connection. Note that this
limit does not apply to "in-flight" receive buffers that are
associated with mbufs in the socket's receive buffer.

Co-authored-by: Navdeep Parhar <np@FreeBSD.org>
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D44001


# afaefb77 30-Jan-2024 John Baldwin <jhb@FreeBSD.org>

cxgbe tom: Limit TOE connections to 2 reassembly islands always

Previously this was only limited on T6 cards to support switching from
ULP_MODE_NONE to ULP_MODE_TLS. To support switching to
ULP_MODE_TCPDDP, enable this for all adapters.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D43669


# c3d4aea6 30-Jan-2024 John Baldwin <jhb@FreeBSD.org>

cxgbe: Add counters for POSIX async I/O requests handled by the driver

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D43668


# 0201eb29 02-Jan-2024 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix virtual interface reattach.

Replace the DOOMED flag with a transient DETACHING flag that is cleared
when VI is detached. This fixes VI reattach when only the VI and not
the parent nexus is detached. The old flag was never cleared and
prevented subsequent synch op's related to the VI.

PR: 275260
Reviewed by: jhb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D43287
Sponsored by: Chelsio Communications


# 8b144c01 27-Dec-2023 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Destroy the tick mutex during VI detach.

This avoids a mutex reinitialization when the VI is detached and
reattached.

Fixes: 516fe911a6b7 cxgbe(4): Always use the per-VI callout to read interface stats.
MFC after: 1 week
Sponsored by: Chelsio Communications


# 3814249f 05-Sep-2023 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Avoid hang on kldunload on netlink enabled kernels.

netlink(4) calls back into the driver during detach and it attempts to
start an internal synchronized op recursively, causing an interruptible
hang. Fix it by failing the ioctl if the VI has been marked as DOOMED
by cxgbe_detach.

Here's the stack for the hang for reference.
#6 begin_synchronized_op
#7 cxgbe_media_status
#8 ifmedia_ioctl
#9 cxgbe_ioctl
#10 if_ioctl
#11 get_operstate_ether
#12 get_operstate
#13 dump_iface
#14 rtnl_handle_ifevent
#15 rtnl_handle_ifnet_event
#16 rt_ifmsg
#17 if_unroute
#18 if_down
#19 if_detach_internal
#20 if_detach
#21 ether_ifdetach
#22 cxgbe_vi_detach
#23 cxgbe_detach
#24 DEVICE_DETACH

MFC after: 3 days
Sponsored by: Chelsio Communications


# 685dc743 16-Aug-2023 Warner Losh <imp@FreeBSD.org>

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/


# 884eaacd 05-Jul-2023 John Baldwin <jhb@FreeBSD.org>

ddb: Rework macros to make it easier to add new command tables.

- Add new DB_DEFINE_TABLE and DB_DECLARE_TABLE macros to define new
command tables. DB_DECLARE_TABLE is intended for use in headers
similar to MALLOC_DECLARE and SYSCTL_DECL.

DB_DEFINE_TABLE takes three arguments, the name of the parent table,
the command name, and the name of the table itself, e.g.
DB_DEFINE_TABLE(show, foo, show_foo) defines a new "show foo" table.

- DB_TABLE_COMMAND, DB_TABLE_COMMAND_FLAGS, DB_TABLE_ALIAS, and
DB_ALIAS_FLAGS allow new commands and aliases to be defined. These
are similar to the existing DB_COMMAND, etc. except that they take
an initial argument giving the name of the parent table, e.g.:

DB_TABLE_COMMAND(show_foo, bar, db_show_foo_bar)

defines a new "show foo bar" command.

This provides a cleaner interface than the ad-hoc use of internal
macros like _DB_SET that was required previously (e.g. in cxgbe(4)).

This retires DB_FUNC macro as well as the internal _DB_FUNC macro.

Reviewed by: melifaro, kib, markj
Differential Revision: https://reviews.freebsd.org/D40819


# 4d846d26 10-May-2023 Warner Losh <imp@FreeBSD.org>

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix


# 27913351 20-Apr-2023 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Dump the firmware log before falling back to a minimal config.

It might have errors that explain why the attempted configuration
failed.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 954712e8 30-May-2022 Justin Hibbits <jhibbits@FreeBSD.org>

Mechanically convert cxgb(4) and cxgbe(4) to IfAPI

Reviewed by: np
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D38597


# 8afd23de 17-Feb-2023 John Baldwin <jhb@FreeBSD.org>

cxgbe: Allow parse_pkt to internally queue a packet.

If parse_pkt returns EINPROGRESS, return from cxgbe_transmit
without queueing the packet in a txq. Use this to move the call
to ethofld_transmit for packet pacing into parse_pkt.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D38577


# 47131a32 10-Feb-2023 Jung-uk Kim <jkim@FreeBSD.org>

cxgbe: fix module loading

After c03414326909, if_cxgbe.ko fails to load if crypto is not compiled
in kernel, e.g., MINIMAL.

link_elf_obj: symbol hmac_init_ipad undefined
linker_load_file: /boot/kernel/if_cxgbe.ko - unsupported file type
kldload: an error occurred while loading module if_cxgbe. Please check dmesg(8) for more details.

Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D38482


# 2ff447ee 15-Nov-2022 John Baldwin <jhb@FreeBSD.org>

cxgbe: Enable TOE TLS RX when an RX key is provided via setsockopt().

Rather than requiring a socket to be created as a TLS socket from the
get go, switch a TOE socket from "plain" TOE to TLS mode when a
receive key is added to the socket.

The firmware is only able to switch a "plain" TOE connection to TLS
mode if the head of the pending socket data is the start of a TLS
record, so the connection is migrated to TLS mode as a multi-step
process.

When TOE TLS RX is enabled, the associated connection's receive side
is frozen via a flag in the TCB. The state of the socket buffer is
then examined to determine if the pending data in the socket buffer
ends on a TLS record boundary. If so, the connection is migrated to
TLS mode and unfrozen. Otherwise, the connection is unfrozen
temporarily until more data arrives. Once more data arrives, the
receive queue is frozen again and rechecked. This continues until the
connection is paused at a record boundary. Any records received
before TLS mode is enabled are decrypted as software records.

Note that this removes the 'rx_tls_ports' sysctl. TOE TLS offload for
receive is now enabled automatically on existing TOE connections when
using a KTLS-aware SSL library just as it was previously enabled
automatically for TLS transmit. This also enables TLS offload for TOE
connections which enable TLS after passing initial data in the clear
(e.g. STARTTLS with SMTP).

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37351


# 21186bdb 15-Nov-2022 John Baldwin <jhb@FreeBSD.org>

cxgbe: Various whitespace fixes.

Mostly trailing whitespace and spaces before tabs.

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37350


# cee4fc7c 26-Sep-2022 John Baldwin <jhb@FreeBSD.org>

cxgbe: Use secq(9) to manage the timestamp generations.

This is mostly cosmetic, but it also doesn't leave a gap of time where
no structures are valid. Instead, we permit the ISR to continue to
use the previous structure if the write to update cal_current isn't
yet visible.

Reviewed by: gallatin
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D36669


# 2c74c9da 26-Sep-2022 John Baldwin <jhb@FreeBSD.org>

cxgbe: Compute timestamps via sbintime_t.

This uses fixed-point math already used elsewhere in the kernel for
sub-second time values. To avoid overflows this does require updating
the calibration once a second rather than once every 30 seconds. Note
that the cxgbe driver already queries multiple registers once a second
for the statistics timers. This version also uses fewer instructions
with no branches (for the math portion) in the per-packet fast path.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D36663


# 8d420190 20-Sep-2022 Konstantin Belousov <kib@FreeBSD.org>

cxgbe: fix kernel build without TCP_OFFLOAD or RATELIMIT

Reviewed by: np
Fixes: e398922eaf6
Sponsored by: Nvidia networking
Differential revision: https://reviews.freebsd.org/D36648


# d0b235c7 20-Sep-2022 Mateusz Guzik <mjg@FreeBSD.org>

cxgbe: fix the build after e398922eaf66978b5e556f6b4b095693c865f329


# e398922e 20-Sep-2022 Randall Stewart <rrs@FreeBSD.org>

Enable M_TSTMP in Chelsio cxgbe driver by creating a mechanism that can sync the time.

Chelsio has always been recording a timestamp in the mbuf (rcv_tstmp) but
not setting the M_TSTMP bit in the mbuf flags. This is because the timestamp
was just the free running 60bit clock. This change fixes that so that
we keep a synchronization by periodically (every 30 seconds after startup)
getting the timestamp and the current nanosecond time. We always keep
several sets around and the current one we always keep the current pair
and the previous pair of timestamps. This allows us to setup a ratio
between the two so we can correctly translate the time. Note that
we use special care to split the timestamp into seconds (per the clock tick)
and nanoseconds otherwise 64bit math would overflow.

Reviewed by: np
Sponsored by: Netflix Inc
Differential Revision: https://reviews.freebsd.org/D36315


# a8dc6738 16-Aug-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add a knob to request that clocks be gated on suspend.

MFC after: 3 months
Sponsored by: Chelsio Communications


# bbb2f537 08-Aug-2022 John Baldwin <jhb@FreeBSD.org>

cxgbe: Rename t4_kern_tls.c to t6_kern_tls.c.

This implementation of NIC TLS is specific to T6 adapters.

Sponsored by: Chelsio Communications


# 4ef7db5a 14-Jun-2022 Mitchell Horne <mhorne@FreeBSD.org>

ddb: namespacing of struct command

'command' is too generic for something specific to the kernel debugger;
change this so it is less likely to collide with local variable names.
Also rename struct command_table to struct db_command_table.

Reviewed by: markj
MFC after: 1 week
Sponsored by: Juniper Networks, Inc.
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D35367


# 12b37f8f 17-May-2022 John Baldwin <jhb@FreeBSD.org>

cxgbe: Deactivate upper layer drivers (like TOE) during detach.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D35237


# e8d1145d 19-Apr-2022 John Baldwin <jhb@FreeBSD.org>

cxgbe: Remove unused devclass arguments to *DRIVER_MODULE().

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D34964


# 169e94c4 14-Apr-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add support for PPOD_EDRAM feature.

The driver queries the firmware to find out if it supports this feature
and enables it if it does. The firmware moves the iSCSI page pod region
to a lower address so that some of it is located in the faster on-chip
memory instead of external DDR.

Reviewed by: jhb@
MFC after: 3 weeks
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D34895


# 239170f2 14-Apr-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix control flow issues reported by Coverity.

CID 1487932: Control flow issues (NESTING_INDENT_MISMATCH).
The macro on this line expands into multiple statements, only the first
of which is nested within the preceding parent while the rest are not.
9828 ulp_region(RX_TLS_KEY);

Reported by: Coverity (CID 1487932)
Fixes: f88b31885c4 cxgbe(4): meminfo should get the TLS region's limits from the hardware.
MFC after: 3 days
Sponsored by: Chelsio Communications


# c5a7885c 05-Apr-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Display the number of free PM pages and p-structs in meminfo.

MFC after: 1 week
Sponsored by: Chelsio Communications


# f88b3188 05-Apr-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): meminfo should get the TLS region's limits from the hardware.

meminfo is meant for debugging and it should read hardware configuration
directly when possible instead of relying on values queried from the
firmware. Rename the region to "TLSKey region" to match other drivers
while here.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 9983e5cc 05-Apr-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix the sorting order of the regions listed in meminfo.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 231f2112 25-Mar-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Handle FORCE_FEC in pcaps correctly.

The firmware doesn't report FORCE_FEC in pcaps if the transceiver
plugged in at that time does not support a speed that may use FEC. It
is incorrect for the driver to assume that the FORCE_FEC value it read
during attach (in init_link_config) is permanent. Instead, it should
check pcaps just before issuing the L1CFG command.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 41c4e1c7 22-Mar-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Allow dump_cimla and dump_devlog to sleep.

This has been safe since e9e7bc82505, which moved parts of error
handling from the ithread to a taskqueue.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 9282f04f 07-Mar-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): dump_devlog should never fail silently.

Do the same thing as dump_cimla and log a warning on failure.

MFC after: 3 days
Sponsored by: Chelsio Communications


# 17f564a7 02-Mar-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Dump some more debug registers in cim_dump_regs.

MFC after: 3 days
Sponsored by: Chelsio Communications


# e9e7bc82 04-Feb-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Changes to the fatal error handler.

* New error_flags that can be used from the error ithread and elsewhere
without a synch_op.
* Stop the adapter immediately in t4_fatal_err but defer most of the
rest of the handling to a task. The task is allowed to sleep, unlike
the ithread. Remove async_event_task as it is no longer needed.
* Dump the devlog, CIMLA, and PCIE_FW exactly once on any fatal error
involving the firmware or the CIM block. While here, dump some
additional info (see dump_cim_regs) for these errors.
* If both reset_on_fatal_err and panic_on_fatal_err are set then attempt
a reset first and do not panic the system if it is successful.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 08c7dc7f 11-Feb-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix illegal hardware access in cxgbe_refresh_stats.

cxgbe_refresh_stats takes into account VI_SKIP_STATS but not
VI_INIT_DONE when deciding whether to read the hardware stats. But
before this change VI_SKIP_STATS was set only for VIs with VI_INIT_DONE.
That meant that cxgbe_refresh_stats always accessed the hardware for
uninitialized VIs, and this is a problem if the adapter is suspended or
in the middle of a reset.

Fix this by setting VI_SKIP_STATS on all VIs during suspend. While
here, ignore VI_INIT_DONE in vi_refresh_stats too to be consistent with
cxgbe_refresh_stats.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 39a36707 11-Feb-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Avoid unsafe hardware access in the ifmedia ioctls.

The hardware is unavailable when the device is suspended or in the
middle of a reset.

MFC after: 1 week
Sponsored by: Chelsio Communications


# a727d953 13-Jan-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix bad races between sysctl and driver detach.

The default sysctl context setup by newbus for a device is eventually
freed by device_sysctl_fini, which runs after the device driver's detach
routine. sysctl nodes associated with this context must not use any
resources (like driver locks, hardware access, counters, etc.) that are
released by driver detach.

There are a lot of sysctl nodes like this in cxgbe(4) and the fix is to
hang them off a context that is explicitly freed by the driver before it
releases any resource that might be used by a sysctl.

This fixes panics when running "sysctl dev.t6nex dev.cc" in a tight loop
and loading/unloading the driver in parallel.

Reported by: Suhas Lokesha
MFC after: 1 week
Sponsored by: Chelsio Communications


# cdd7fe04 10-Jan-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Do not ignore the return value of ifmedia_ioctl.

This ensures that the driver reports an error instead of failing
silently when an invalid media is requested.

Reported by: Suhas Lokesha @ Chelsio
MFC after: 1 week
Sponsored by: Chelsio Communications


# 39d5cbdc 05-Jan-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix "set but not used [-Wunused-but-set-variable]" warnings.

MFC after: 1 week
Sponsored by: Chelsio Communications


# bbab9ab5 03-Jan-2022 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix stats collection for ports with port_id != tx_chan

This fixes a driver panic during stats collection when a port's id does
not match its tx channel. The bug affected only the T580 card running
with a non-default VPD.

Reported by: Suhas Lokesha @ Chelsio
MFC after: 1 week
Sponsored by: Chelsio Communications


# a8eacf93 15-Nov-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Change the way t4_shutdown_adapter brings the link(s) down.

Modify the GPIO pins only on the Base-T cards and even there drive all
of them low instead of putting them in hi-z state. For the rest (this
is the common case), directly power off the PLLs of the high speed
serdes. This is the simplest method that does not involve or conflict
with the firmware but still works with all T4-T6 cards regardless of
what's plugged into the port.

This fixes a problem where the peer wouldn't always see a link down if
it is connected to the device using a -CR4 copper cable.

MFC after: 3 weeks
Sponsored by: Chelsio Communications


# 448bcd01 10-Nov-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): internal knob for flexible control over FEC selection.

Recent firmwares have support for autonomous FEC selection and a "force"
knob to let the driver control this behavior (or not) in a fine grained
manner. This change adds a driver knob so that all the different ways of
configuring the link FEC can be exercised. Note that this controls the
internal driver/firmware interaction for link configuration and is not
meant for general use.

MFC after: 1 week
Sponsored by: Chelsio Communications


# f6a2e110 10-Nov-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): separate sysctls for user-requested and in-use FEC.

Recent firmwares have more leeway in FEC selection and there is a need
to track the FECs requested by the driver separately from the FEC in use
on the link. The existing dev.<port>.<inst>.fec sysctl can read both but
its behavior depends on the link state and it is sometimes hard to find
out what was requested when the link is up.

Split the fec sysctl into two (requested_fec and link_fec) to get access
to both pieces of information regardless of the link state.

MFC after: 1 week
Sponsored by: Chelsio Communications


# d99b1d83 03-Nov-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): sysctl to track the last L1_CFG32 requested by the driver.

dev.<port>.<inst>.rcaps

# sysctl dev.cc | grep rcaps
dev.cc.1.rcaps: 581107776
dev.cc.0.rcaps: 582156414

MFC after: 1 week
Sponsored by: Chelsio Communications


# ef3f98ae 14-Oct-2021 John Baldwin <jhb@FreeBSD.org>

cxgbe: Only run ktls_tick when NIC TLS is enabled.

Previously the body of ktls_tick was a nop when NIC TLS was disabled,
but the callout was still scheduled consuming power on otherwise-idle
systems with Chelsio T6 adapters. Now the callout only runs while NIC
TLS is enabled on at least one interface of an adapter.

Reported by: mav
Reviewed by: np, mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D32491


# c782ea8b 14-Sep-2021 John Baldwin <jhb@FreeBSD.org>

Add a switch structure for send tags.

Move the type and function pointers for operations on existing send
tags (modify, query, next, free) out of 'struct ifnet' and into a new
'struct if_snd_tag_sw'. A pointer to this structure is added to the
generic part of send tags and is initialized by m_snd_tag_init()
(which now accepts a switch structure as a new argument in place of
the type).

Previously, device driver ifnet methods switched on the type to call
type-specific functions. Now, those type-specific functions are saved
in the switch structure and invoked directly. In addition, this more
gracefully permits multiple implementations of the same tag within a
driver. In particular, NIC TLS for future Chelsio adapters will use a
different implementation than the existing NIC TLS support for T6
adapters.

Reviewed by: gallatin, hselasky, kib (older version)
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D31572


# 92de7379 07-Sep-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix the decode and display of the DBVFIFO region in meminfo.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 83a611e0 07-Sep-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Display HMA information in meminfo.

This should have been added with initial T6 support many years ago.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 5b27e4b2 06-Aug-2021 John Baldwin <jhb@FreeBSD.org>

cxgbei: Support for ISO (iSCSI segmentation offload).

ISO can be disabled before establishing a connection by setting
dev.tNnex.N.toe.iso to 0.

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D31223


# ec8004dd 24-Jun-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Do not configure traffic classes automatically on attach.

The driver used to configure all available classes with some default
parameters on attach and the rest of t4_sched.c was written with the
assumption that all traffic classes are always valid in the hardware.
But this resulted in a lot of informational messages being logged in the
firmware's circular log, crowding out other more useful messages.

This change leaves the tx scheduler alone during attach to reduce the
spam in the devlog. The state of every class is now tracked separately
from its flags and there is support for an 'uninitialized' state.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# 6beb67c7 21-Jun-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Get the number of usable traffic classes from the firmware.

Recent firmwares are able to utilize the traffic classes of tx channels
that were previously unused. This effectively doubles the number of
traffic classes available per port for 2 port cards. Stop using the raw
per-channel value in the driver and ask the firmware for the number of
usable traffic classes instead.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# ddfc9c4c 22-Jun-2021 Warner Losh <imp@FreeBSD.org>

newbus: Move from bus_child_{pnpinfo,location}_src to bus_child_{pnpinfo,location} with sbuf

Now that the upper layers all go through a layer to tie into these
information functions that translates an sbuf into char * and len. The
current interface suffers issues of what to do in cases of truncation,
etc. Instead, migrate all these functions to using struct sbuf and these
issues go away. The caller is also in charge of any memory allocation
and/or expansion that's needed during this process.

Create a bus_generic_child_{pnpinfo,location} and make it default. It
just returns success. This is for those busses that have no information
for these items. Migrate the now-empty routines to using this as
appropriate.

Document these new interfaces with man pages, and oversight from before.

Reviewed by: jhb, bcr
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D29937


# db15dbf8 01-Jun-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Check if the firmware supports 512 SGL per FR MR.

Firmwares >= 1.25.6.0 support 512 SGL entries in a single memory
registration request.

Obtained from: Chelsio Communications
MFC after: 1 week
Sponsored by: Chelsio Communications


# 24b98f28 23-May-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Overhaul CLIP (Compressed Local IPv6) table management.

- Process the list of local IPs once instead of once per adapter. Add
addresses from all VNETs to the driver's list but leave hardware
updates for later when the global VNET/IFADDR list locks have been
released.

- Add address to the hardware table synchronously when a CLIP entry is
requested for an address that's not already in there.

- Provide ioctls that allow userspace tools to manage addresses in the
CLIP table.

- Add a knob (hw.cxgbe.clip_db_auto) that controls whether local IPs are
automatically added to the CLIP table or not.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# 4b6ed075 14-May-2021 John Baldwin <jhb@FreeBSD.org>

cxgbe: Make the TOE ISCSI RX stats per-queue instead of per adapter.

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D29903


# 83b5cda1 27-Apr-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add support for NIC suspend/resume and live reset.

Add suspend/resume callbacks to the driver and a live reset built around
them. This commit covers the basic NIC and future commits will expand
this functionality to other stateful parts of the chip. Suspend and
resume operate on the chip (the t?nex nexus device) and affect all its
ports. It is not possible to suspend/resume or reset individual ports.
All these operations can be performed on a running NIC. A reset will
look like a link bounce to the networking stack.

Here are some ways to exercise this functionality:

/* Manual suspend and resume. */
# devctl suspend t6nex0
# devctl resume t6nex0

/* Manual reset. */
# devctl reset t6nex0

/* Manual reset with driver sysctl. */
# sysctl dev.t6nex.0.reset=1

/* Automatic adapter reset on any fatal error. */
# hw.cxgbe.reset_on_fatal_err=1

Suspend disables the adapter (DMA, interrupts, and the port PHYs) and
marks the hardware as unavailable to the driver. All ifnets associated
with the adapter are still visible to the kernel but operations that
require hardware interaction will fail with ENXIO. All ifnets report
link-down while the adapter is suspended.

Resume will reattach to the card, reconfigure it as before, and recreate
the queues servicing the existing ifnets. The ifnets are able to send
and receive traffic as soon as the link comes back up.

Reset is roughly the same as a suspend and a resume with at least one of
these events in between: D0->D3Hot->D0, FLR, PCIe link retrain.

MFC after: 1 month
Relnotes: yes
Sponsored by: Chelsio Communications


# 43bbae19 26-Apr-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Separate the sw- and hw-specific parts of resource allocations

The driver uses both software resources (locks, callouts, memory for
descriptors and for bookkeeping, sysctls, etc.) and hardware resources
(VIs, DMA queues, TCAM entries, etc.) to operate the NIC. This commit
splits the single *_ALLOCATED flag used to track all these resources
into separate *_SW_ALLOCATED and *_HW_ALLOCATED flags.

This is the simplified pseudocode that now applies to most queues (foo
can be ctrlq/txq/rxq/ofld_txq/ofld_rxq):

/* Idempotent */
alloc_foo
{
if (!SW_ALLOCATED)
init_iq/init_eq/init_fl no-fail sw init
alloc_iq_fl/alloc_eq/alloc_wrq may-fail sw alloc
add_foo_sysctls, etc. no-fail post-alloc items
if (!HW_ALLOCATED)
alloc_iq_fl_hwq/alloc_eq_hwq hw resource allocation
}

/* Idempotent */
free_foo
{
if (!HW_ALLOCATED)
free_iq_fl_hwq/free_eq_hwq release hw resources
if (!SW_ALLOCATED)
free_iq_fl/free_eq/free_wrq release sw resources
}

The routines that take the driver to FULL_INIT_DONE and VI_INIT_DONE and
back are now all idempotent. The quiesce routines pay attention to the
HW_ALLOCATED flag and will not wait on the hardware for pidx/cidx
updates and other completions if this flag is not set.

MFC after: 1 month
Sponsored by: Chelsio Communications


# 50f5d13e 23-Apr-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): hw.cxgbe.panic_on_fatal_err can be changed any time.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# 5f00292f 23-Apr-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Move the hw-specific parts of VXLAN setup to a separate function.

It can be called to (re)apply the settings in the driver softc to the
hardware.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# b47b28e5 22-Apr-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add flag to reliably stop the driver from accessing hw stats.

There are two kinds of routines in the driver that read statistics from
the hardware: the cxgbe_* variants read the per-port MPS/MAC registers
and the vi_* variants read the per-VI registers. They can be called
from the 1Hz callout or if_get_counter. All stats collection now takes
place under the callout lock and there is a new flag to indicate that
these routines should not access any hardware register.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# dc77e792 22-Apr-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix minor nit in the display of MPS TCAM entries.

MFC after: 3 days


# 557c4521 13-Apr-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe/t4_tom: Implement tod_pmtu_update.

tod_pmtu_update was added to the kernel in 01d74fe1ffc.

Sponsored by: Chelsio Communications


# 568e69e4 12-Apr-2021 John Baldwin <jhb@FreeBSD.org>

cxgbe: Add counters for iSCSI PDUs transmitted via TOE.

Reviewed by: np
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D29297


# 516fe911 01-Apr-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Always use the per-VI callout to read interface stats.

There is no change in the source of the stats (t4_get_port_stats or
t4_get_vi_stats) but the per-port callout is gone.

Sponsored by: Chelsio Communications
Reviewed by: jhb@
Differential Revision: https://reviews.freebsd.org/D29527


# fe496dc0 26-Mar-2021 John Baldwin <jhb@FreeBSD.org>

cxgbe: Make the TOE TLS stats per-queue instead of per-port.

This avoids some atomics by using counter_u64 for TX and relying on
existing single-threading (single ithread per rxq) for RX.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D29383


# 077ba6a8 26-Mar-2021 John Baldwin <jhb@FreeBSD.org>

cxgbe: Add a struct sge_ofld_txq type.

This type mirrors struct sge_ofld_rxq and holds state for TCP offload
transmit queues. Currently it only holds a work queue but will
include additional state in future changes.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D29382


# 15f33555 23-Mar-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Allow a T6 adapter to switch between TOE and NIC TLS mode.

The hw.cxgbe.kern_tls tunable was used for this in the past and if it
was set then all T6 adapters would be configured for NIC TLS operation
and could not be reconfigured for TOE without a reload. With this
change ifconfig can be used to manipulate toe and txtls caps like any
other caps. hw.cxgbe.kern_tls continues to work as usual but its
effects are not permanent any more.

* Enable nic_ktls_ofld in the default configuration file and use the
firmware instead of direct register manipulation to apply/rollback
NIC TLS configuration. This allows the driver to switch the hardware
between TOE and NIC TLS mode in a safe manner. Note that the
configuration is adapter-wide and not per-port.

* Remove the kern_tls config file as it works with 100G T6 cards only
and leads to firmware crashes with 25G cards. The configurations
included with the driver (with the exception of the FPGA configs) are
supposed to work with all adapters.

Reported by: Veeresh U.K. at Chelsio
MFC after: 2 weeks
Sponsored by: Chelsio Communications
Reviewed by: jhb@
Differential Revision: https://reviews.freebsd.org/D29291


# 3cc6f777 19-Mar-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): create a separate helper routine to write the global RSS key.

While here, make sure only the PF driver attempts to program the global
RSS key (with options RSS). The VF driver doesn't have access to those
device registers.

MFC after: 1 week
Sponsored by: Chelsio Communications


# a1d803c1 19-Mar-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): make it safe to call setup_memwin repeatedly.

A repeat call will recreate the memory windows in the hardware and move
them to their last-known positions without repeating any of the software
initialization.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 0b373f26 15-Mar-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): catch up with the latest cryptocaps.

There are two crypto capabilities that the driver didn't know about.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 4a4e9c51 05-Mar-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix an assertion that is not valid during attach.

Firmware access from t4_attach takes place without any synchronization.
The driver should not panic (debug kernels) if something goes wrong in
early communication with the firmware. It should still load so that
it's possible to poke around with cxgbetool.

MFC after: 1 week
Sponsored by: Chelsio Communications


# dfff1de7 25-Feb-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Read the rx 'c' channel for a port and make it available.

MFC after: 1 week
Sponsored by: Chelsio Communications


# c91dda5a 19-Feb-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add a driver ioctl to set the filter mask.

Allow the filter mask (aka the hashfilter mode when hashfilters are
in use) to be set any time it is safe to do so. The requested mask
must be a subset of the filter mode already. The driver will not change
the mode or ingress config just to support a new mask.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# fae028dd 18-Feb-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Break up t4_read_chip_settings.

Read the PF-only hardware settings directly in get_params__post_init.
Split the rest into two routines used by both the PF and VF drivers: one
that reads the SGE rx buffer configuration and another that verifies
miscellaneous hardware configuration.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 3447df8b 01-Feb-2021 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fixes to tx coalescing.

- The behavior implemented in r362905 resulted in delayed transmission
of packets in some cases, causing performance issues. Use a different
heuristic to predict tx requests.

- Add a tunable/sysctl (hw.cxgbe.tx_coalesce) to disable tx coalescing
entirely. It can be changed at any time. There is no change in
default behavior.


# 3f43ada9 28-Jan-2021 Gleb Smirnoff <glebius@FreeBSD.org>

Catch up with 6edfd179c86: mechanically rename IFCAP_NOMAP to IFCAP_MEXTPG.

Originally IFCAP_NOMAP meant that the mbuf has external storage pointer
that points to unmapped address. Then, this was extended to array of
such pointers. Then, such mbufs were augmented with header/trailer.
Basically, extended mbufs are extended, and set of features is subject
to change. The new name should be generic enough to avoid further
renaming.


# 0082e479 03-Dec-2020 John Baldwin <jhb@FreeBSD.org>

Clear TLS offload mode if a TLS socket hangs without receiving data.

By default, if a TOE TLS socket stops receiving data for more than 5
seconds, revert the connection back to plain TOE mode. This provides
a fallback if the userland SSL library does not support KTLS. In
addition, for client TLS 1.3 sockets using connect(), the TOE socket
blocks before the handshake has completed since the socket option is
only invoked for the final handshake.

The timeout defaults to 5 seconds, but can be changed at boot via the
hw.cxgbe.toe.tls_rx_timeout tunable or for an individual interface via
the dev.<nexus>.toe.tls_rx_timeout sysctl.

Reviewed by: np
MFC after: 2 weeks
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D27470


# 180c2dca 03-Dec-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix vertical alignment in sysctl_cpl_stats.

MFC after: 3 days
Sponsored by: Chelsio Communications


# dbc5c85c 03-Dec-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): two new debug sysctls.

dev.<nexus>.<instance>.misc.tid_stats
dev.<nexus>.<instance>.misc.tnl_stats

MFC after: 3 days
Sponsored by: Chelsio Communications


# 05d56755 03-Dec-2020 John Baldwin <jhb@FreeBSD.org>

Fix downgrading of TOE TLS sockets to plain TOE.

If a TOE TLS socket ends up using an unsupported TLS version or
ciphersuite, it must be downgraded to a "plain" TOE socket with TLS
encryption/decryption performed on the host. The previous
implementation of this fallback was incomplete and resulted in hung
connections.

Reviewed by: np
MFC after: 2 weeks
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D27467


# f14d7c95 11-Nov-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbev(4): Make sure that the iq/eq map sizes are correct for VFs.

This should have been part of r366929.

MFC after: 3 days
Sponsored by: Chelsio Communications


# b20b25e7 22-Oct-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): fix the size of the iq/eq maps.

The firmware can allocate ingress and egress context ids anywhere from
its configured range. Size the iq/eq maps to match the entire range
instead of assuming that the firmware always allocates the first
available context id.

Reported by: Baptiste Wicht @ Verisign
MFC after: 1 week
Sponsored by: Chelsio Communications


# 37d41133 21-Oct-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): display correct tid range for T6 based -SO cards.

Reported by: Chelsio QA
MFC after: 1 week
Sponsored by: Chelsio Communications


# ae5da4e1 19-Oct-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Updates to the drop features from r366532.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 31deb3cc 09-Oct-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): More fixes for the T6 FCS error counter.

r365732 was the first attempt to get an accurate count but it was
writing to some read-only registers to clear them and that obviously
didn't work. Instead, note the counter's value when it is supposed to
be cleared and subtract it from future readings.

dev.<port>.stats.rx_fcs_error should not be serviced from the MPS
register for T6.

The stats.* sysctls should all use T5_PORT_REG for T5 and above. This
must have been missed in the initial T5 support years ago. Fix it while
here.

MFC after: 3 days
Sponsored by: Chelsio Communications


# 77af2b2c 08-Oct-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): knobs to drop various kinds of undesirable frames on ingress.

These kind of drops come for free in the sense that they do not use the
filter TCAM or any other resource that wouldn't normally be used during
rx. Frames dropped by the hardware get counted in the MAC's rx stats
but are not delivered to the driver.

hw.cxgbe.attack_filter
Set to 1 to enable the "attack filter". Default is 0. The attack
filter will drop an incoming frame if any of these conditions is true:
src ip/ip6 == dst ip/ip6; tcp and src/dst ip is not unicast; src/dst ip
is loopback (127.x.y.z); src ip6 is not unicast; src/dst ip6 is loopback
(::1/128) or unspecified (::/128); tcp and src/dst ip6 is mcast
(ff00::/8).

hw.cxgbe.drop_ip_fragments
Set to 1 to drop all incoming IP fragments. Default is 0. Note that
this drops valid frames.

hw.cxgbe.drop_pkts_with_l2_errors
Set to 1 to drop incoming frames with Layer 2 length or checksum errors.
Default is 1.

hw.cxgbe.drop_pkts_with_l3_errors
Set to 1 to drop incoming frames with IP version, length, or checksum
errors. Default is 0.

hw.cxgbe.drop_pkts_with_l4_errors
Set to 1 to drop incoming frames with Layer 4 length, checksum, or other
errors. Default is 0.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# 56fb710f 06-Oct-2020 John Baldwin <jhb@FreeBSD.org>

Store the send tag type in the common send tag header.

Both cxgbe(4) and mlx5(4) wrapped the existing send tag header with
their own identical headers that stored the type that the
type-specific tag structures inherited from, so in practice it seems
drivers need this in the tag anyway. This permits removing these
extra header indirections (struct cxgbe_snd_tag and struct
mlx5e_snd_tag).

In addition, this permits driver-independent code to query the type of
a tag, e.g. to know what type of tag is being queried via
if_snd_query.

Reviewed by: gallatin, hselasky, np, kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D26689


# 8741306b 05-Oct-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4) sysctls do not need Giant.

Sponsored by: Chelsio Communications


# 30e3f2b4 21-Sep-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): let the PF driver use VM work requests for transmit.

This allows the PF interfaces to communicate with the VF interfaces over
the internal switch in the ASIC. Fix the GL limits for VM work requests
while here.

MFC after: 3 days
Sponsored by: Chelsio Communications


# a4a4ad2d 17-Sep-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): add support for stateless offloads for VXLAN traffic.

Hardware assistance includes checksumming (tx and rx), TSO, and RSS on
the inner traffic in a VXLAN tunnel.

Relnotes: Yes
Sponsored by: Chelsio Communications


# 6a59b994 22-Aug-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Use large clusters for TOE rx queues when TOE+TLS is enabled.

Rx is more efficient within the chip when the receive buffer size
matches the TLS PDU size.

MFC after: 3 days
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D26127


# 5822a14c 27-Jul-2020 Mark Johnston <markj@FreeBSD.org>

cxgbe(4): Stop checking for failures from malloc(M_WAITOK).

PR: 240545
Submitted by: Andrew Reiter <arr@watson.org>
Reviewed by: np
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25767


# d735920d 02-Jul-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): changes in the Tx path to help increase tx coalescing.

- Ask the firmware for the number of frames that can be stuffed in one
work request.

- Modify mp_ring to increase the likelihood of tx coalescing when there
are just one or two threads that are doing most of the tx. Add teeth
to the abdication mechanism by pushing the consumer lock into mp_ring.
This reduces the likelihood that a consumer will get stuck with all
the work even though it is above its budget.

- Add support for coalesced tx WR to the VF driver. This, with the
changes above, results in a 7x improvement in the tx pps of the VF
driver for some common cases. The firmware vets the L2 headers
submitted by the VF driver and it's a big win if the checks are
performed for a batch of packets and not each one individually.

Reviewed by: jhb@
MFC after: 2 weeks
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D25454


# 7c228be3 25-Jun-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add a pointer to the adapter softc in vi_info.

There were quite a few places where port_info was being accessed only to
get to the adapter.

Reviewed by: jhb@
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D25432


# cbb9ccf7 15-Jun-2020 Ryan Moeller <freqlabs@FreeBSD.org>

Avoid trying to toggle TSO twice

Remove TSO from the toggle mask when automatically disabled by TXCKSUM* in
various NIC drivers.

Reviewed by: hselasky, np, gallatin, jpaetzel
Approved by: mav (mentor)
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25120


# b0dede77 19-May-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe/iw_cxgbe: Add an async callback to notify iw_cxgbe in case of a
fatal error.

Submitted by: Krishnamraju Eraparaju @ Chelsio
MFC after: 2 weeks
Sponsored by: Chelsio Communications


# 6d44e8e6 27-Feb-2020 John Baldwin <jhb@FreeBSD.org>

Rename TOE TLS stats from [rt]x_tls_* to [rt]x_toe_tls_*.

This more clearly differentiates TLS records encrypted and decrypted
in TOE connections from those encrypted via NIC TLS.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 7029da5c 26-Feb-2020 Pawel Biernacki <kaktus@FreeBSD.org>

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718


# 02cd7739 18-Feb-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Congestion drops are maintained per E-channel and not per
buffer group.

This fixes a bug where congestion drops on port 1 of a T6 card would
incorrectly be counted as drops on port 0.

MFC after: 1 week
Sponsored by: Chelsio Communications


# ca3b3c57 13-Feb-2020 John Baldwin <jhb@FreeBSD.org>

Remove the per-TXQ tls_wrs stat.

It duplicated the kern_tls_records stat and was not conditional on NIC
TLS being enabled.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D23670


# 21935a41 05-Feb-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add native netmap support to the main interface.

This means that extra virtual interfaces (VIs) created with
hw.cxgbe.num_vis are no longer required to use netmap. Use this
tunable to enable native netmap support on the main interface:

hw.cxgbe.native_netmap="3"

There is no change in default behavior.

Suggested by: jch@
MFC after: 2 weeks
Sponsored by: Chelsio Communications


# 87bbb333 03-Feb-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add pfil(9) hooks to the driver's rx.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 46e1e307 03-Feb-2020 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Retire the allow_mbufs_in_cluster optimization.

This simplifies the driver's rx fast path as well as the bookkeeping
code that tracks various rx buffer sizes and layouts.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 93065a5a 18-Dec-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): check if the firmware supports FW_RI_FR_NSMR_TPTE_WR work
request.

This is used by iw_cxgbe to figure out how best to register memory.

MFC after: 1 month
Sponsored by: Chelsio Communications


# c08c2d42 10-Dec-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Simplify the firmware version checks a bit.

No functional change.

MFC after: 1 week


# e3338dee 25-Nov-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Allow the driver to specify multiple FECs that the firmware
should try in order to link up with the peer.

Various FEC variables within the driver can now have multiple bits set
instead of being powers of 2. 0 and -1 in the user knobs still mean no
FEC and auto (driver decides) respectively for backward compatibility,
but no-FEC and auto now have their own bits in the internal
representation. There is a new bit that can be set to request the FEC
recommended by the cable/transceiver module.

Add sysctls to display link related capabilities of the local side as
well as the link partner.

Note that all this needs a new firmware and the documentation for the
driver FEC knobs will be updated after that firmware is added to the
driver.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 515a40d5 24-Nov-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): sysctl to reset the temperature/voltage sensor.

# sysctl dev.<nexus>.<inst>.reset_sensor=1
# sysctl dev.t6nex.0.reset_sensor=1

MFC after: 1 week
Sponsored by: Chelsio Communications


# bddf7343 21-Nov-2019 John Baldwin <jhb@FreeBSD.org>

NIC KTLS for Chelsio T6 adapters.

This adds support for ifnet (NIC) KTLS using Chelsio T6 adapters.
Unlike the TOE-based KTLS in r353328, NIC TLS works with non-TOE
connections.

NIC KTLS on T6 is not able to use the normal TSO (LSO) path to segment
the encrypted TLS frames output by the crypto engine. Instead, the
TOE is placed into a special setup to permit "dummy" connections to be
associated with regular sockets using KTLS. This permits using the
TOE to segment the encrypted TLS records. However, this approach does
have some limitations:

1) Regular TOE sockets cannot be used when the TOE is in this special
mode. One can use either TOE and TOE-based KTLS or NIC KTLS, but
not both at the same time.

2) In NIC KTLS mode, the TOE is only able to accept a per-connection
timestamp offset that varies in the upper 4 bits. Put another way,
only connections whose timestamp offset has the 28 lower bits
cleared can use NIC KTLS and generate correct timestamps. The
driver will refuse to enable NIC KTLS on connections with a
timestamp offset with any of the lower 28 bits set. To use NIC
KTLS, users can either disable TCP timestamps by setting the
net.inet.tcp.rfc1323 sysctl to 0, or apply a local patch to the
tcp_new_ts_offset() function to clear the lower 28 bits of the
generated offset.

3) Because the TCP segmentation relies on fields mirrored in a TCB in
the TOE, not all fields in a TCP packet can be sent in the TCP
segments generated from a TLS record. Specifically, for packets
containing TCP options other than timestamps, the driver will
inject an "empty" TCP packet holding the requested options (e.g. a
SACK scoreboard) along with the segments from the TLS record.
These empty TCP packets are counted by the
dev.cc.N.txq.M.kern_tls_options sysctls.

Unlike TOE TLS which is able to buffer encrypted TLS records in
on-card memory to handle retransmits, NIC KTLS must re-encrypt TLS
records for retransmit requests as well as non-retransmit requests
that do not include the start of a TLS record but do include the
trailer. The T6 NIC KTLS code tries to optimize some of the cases for
requests to transmit partial TLS records. In particular it attempts
to minimize sending "waste" bytes that have to be given as input to
the crypto engine but are not needed on the wire to satisfy mbufs sent
from the TCP stack down to the driver.

TCP packets for TLS requests are broken down into the following
classes (with associated counters):

- Mbufs that send an entire TLS record in full do not have any waste
bytes (dev.cc.N.txq.M.kern_tls_full).

- Mbufs that send a short TLS record that ends before the end of the
trailer (dev.cc.N.txq.M.kern_tls_short). For sockets using AES-CBC,
the encryption must always start at the beginning, so if the mbuf
starts at an offset into the TLS record, the offset bytes will be
"waste" bytes. For sockets using AES-GCM, the encryption can start
at the 16 byte block before the starting offset capping the waste at
15 bytes.

- Mbufs that send a partial TLS record that has a non-zero starting
offset but ends at the end of the trailer
(dev.cc.N.txq.M.kern_tls_partial). In order to compute the
authentication hash stored in the trailer, the entire TLS record
must be sent as input to the crypto engine, so the bytes before the
offset are always "waste" bytes.

In addition, other per-txq sysctls are provided:

- dev.cc.N.txq.M.kern_tls_cbc: Count of sockets sent via this txq
using AES-CBC.

- dev.cc.N.txq.M.kern_tls_gcm: Count of sockets sent via this txq
using AES-GCM.

- dev.cc.N.txq.M.kern_tls_fin: Count of empty FIN-only packets sent to
compensate for the TOE engine not being able to set FIN on the last
segment of a TLS record if the TLS record mbuf had FIN set.

- dev.cc.N.txq.M.kern_tls_records: Count of TLS records sent via this
txq including full, short, and partial records.

- dev.cc.N.txq.M.kern_tls_octets: Count of non-waste bytes (TLS header
and payload) sent for TLS record requests.

- dev.cc.N.txq.M.kern_tls_waste: Count of waste bytes sent for TLS
record requests.

To enable NIC KTLS with T6, set the following tunables prior to
loading the cxgbe(4) driver:

hw.cxgbe.config_file=kern_tls
hw.cxgbe.kern_tls=1

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D21962


# 782b97cb 15-Nov-2019 Gleb Smirnoff <glebius@FreeBSD.org>

Fix regression from r353841: ctx.rc needs to be initialized,
otherwise driver might silently fail to initialize.

Pointy hat to: glebius


# a1b2b6e1 12-Nov-2019 John Baldwin <jhb@FreeBSD.org>

Create a file to hold shared routines for dealing with T6 key contexts.

ccr(4) and TLS support in cxgbe(4) construct key contexts used by the
crypto engine in the T6. This consolidates some duplicated code for
helper functions used to build key contexts.

Reviewed by: np
MFC after: 1 month
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D22156


# 43b57124 07-Nov-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Query Vdd from the firmware if its last known value is 0.

TVSENSE may not be ready by the time t4_fw_initialize returns and the
firmware returns 0 if the driver asks for the Vdd before the sensor is
ready.

MFC after: 1 week
Sponsored by: Chelsio Communications


# e38a50e8 22-Oct-2019 John Baldwin <jhb@FreeBSD.org>

Split Chelsio send tags into a generic base tag and a ratelimit tag.

NIC KTLS will add a new TLS send tag type in cxgbe(4) that is a
distinct tag from a ratelimit tag. To support this, refactor
cxgbe_snd_tag to be a simple send tag with a type and convert the
existing ratelimit tag to a new cxgbe_rate_tag structure.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D22072


# 866a7f28 22-Oct-2019 John Baldwin <jhb@FreeBSD.org>

Always allocate the atid table during attach.

Previously the table was allocated on first use by TOE and the
ratelimit code. The forthcoming NIC KTLS code also uses this table.
Allocate it unconditionally during attach to simplify consumers.

Reviewed by: np
Differential Revision: https://reviews.freebsd.org/D22028


# 02cc07d1 21-Oct-2019 Gleb Smirnoff <glebius@FreeBSD.org>

Convert to if_foreach_llmaddr() KPI.


# b60229e2 09-Oct-2019 John Baldwin <jhb@FreeBSD.org>

Remove adapters from t4_list earlier during detach.

This ensures the clip task won't race with t4_destroy_clip_table.

While here, make some mutex destroys unconditional since attach always
initializes them.

Reviewed by: np
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D21952


# c537e887 26-Aug-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe/t4_tom: Initialize all TOE connection parameters in one place.
Remove now-redundant items from toepcb and synq_entry and the code to
support them.

Let the driver calculate tx_align, rx_coalesce, and sndbuf by default.

Reviewed by: jhb@
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D21387


# 20abea66 01-Aug-2019 Randall Stewart <rrs@FreeBSD.org>

This adds the third step in getting BBR into the tree. BBR and
an updated rack depend on having access to the new
ratelimit api in this commit.

Sponsored by: Netflix Inc.
Differential Revision: https://reviews.freebsd.org/D20953


# f8f1b967 09-Jul-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Clear the freelist statistics in the clearstats ioctl.

Move all clearstats code into its own function while here.

MFC after: 1 week
Sponsored by: Chelsio Communications


# a920680d 09-Jul-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Use the simplest configuration possible when falling back from
the default configuration.

MFC after: 1 week
Sponsored by: Chelsio Communications


# d76bbe17 28-Jun-2019 John Baldwin <jhb@FreeBSD.org>

Add support for IFCAP_NOMAP to cxgbe(4).

Since cxgbe(4) uses sglist instead of bus_dma, this required updates
to the code that generates scatter/gather lists for packets. Also,
unmapped mbufs are always sent via DMA and never as immediate data in
the payload of a work request.

Submitted by: gallatin (earlier version)
Reviewed by: gallatin, hselasky, rrs
Discussed with: np
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20616


# 27c3a85d 06-Jun-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Rename the DDP sysctl to rx_zcopy to match the tx_zcopy sysctl
and update its description. The old name continues to work for now.

Sponsored by: Chelsio Communications


# 0a16ee75 25-May-2019 Alexey Dokuchaev <danfe@FreeBSD.org>

Fix two errors reported by PVS Studio: V646 Consider inspecting the
application's logic. It's possible that 'else' keyword is missing.

Reviewed by: gallatin, np, pfg
Approved by: pfg
Differential Revision: https://reviews.freebsd.org/D20396


# fb3bc596 24-May-2019 John Baldwin <jhb@FreeBSD.org>

Restructure mbuf send tags to provide stronger guarantees.

- Perform ifp mismatch checks (to determine if a send tag is allocated
for a different ifp than the one the packet is being output on), in
ip_output() and ip6_output(). This avoids sending packets with send
tags to ifnet drivers that don't support send tags.

Since we are now checking for ifp mismatches before invoking
if_output, we can now try to allocate a new tag before invoking
if_output sending the original packet on the new tag if allocation
succeeds.

To avoid code duplication for the fragment and unfragmented cases,
add ip_output_send() and ip6_output_send() as wrappers around
if_output and nd6_output_ifp, respectively. All of the logic for
setting send tags and dealing with send tag-related errors is done
in these wrapper functions.

For pseudo interfaces that wrap other network interfaces (vlan and
lagg), wrapper send tags are now allocated so that ip*_output see
the wrapper ifp as the ifp in the send tag. The if_transmit
routines rewrite the send tags after performing an ifp mismatch
check. If an ifp mismatch is detected, the transmit routines fail
with EAGAIN.

- To provide clearer life cycle management of send tags, especially
in the presence of vlan and lagg wrapper tags, add a reference count
to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
Provide a helper function (m_snd_tag_init()) for use by drivers
supporting send tags. m_snd_tag_init() takes care of the if_ref
on the ifp meaning that code alloating send tags via if_snd_tag_alloc
no longer has to manage that manually. Similarly, m_snd_tag_rele
drops the refcount on the ifp after invoking if_snd_tag_free when
the last reference to a send tag is dropped.

This also closes use after free races if there are pending packets in
driver tx rings after the socket is closed (e.g. from tcpdrop).

In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
Drivers now also check this flag instead of checking snd_tag against
NULL. This avoids false positive matches when a forwarded packet
has a non-NULL rcvif that was treated as a send tag.

- cxgbe was relying on snd_tag_free being called when the inp was
detached so that it could kick the firmware to flush any pending
work on the flow. This is because the driver doesn't require ACK
messages from the firmware for every request, but instead does a
kind of manual interrupt coalescing by only setting a flag to
request a completion on a subset of requests. If all of the
in-flight requests don't have the flag when the tag is detached from
the inp, the flow might never return the credits. The current
snd_tag_free command issues a flush command to force the credits to
return. However, the credit return is what also frees the mbufs,
and since those mbufs now hold references on the tag, this meant
that snd_tag_free would never be called.

To fix, explicitly drop the mbuf's reference on the snd tag when the
mbuf is queued in the firmware work queue. This means that once the
inp's reference on the tag goes away and all in-flight mbufs have
been queued to the firmware, tag's refcount will drop to zero and
snd_tag_free will kick in and send the flush request. Note that we
need to avoid doing this in the middle of ethofld_tx(), so the
driver grabs a temporary reference on the tag around that loop to
defer the free to the end of the function in case it sends the last
mbuf to the queue after the inp has dropped its reference on the
tag.

- mlx5 preallocates send tags and was using the ifp pointer even when
the send tag wasn't in use. Explicitly use the ifp from other data
structures instead.

- Sprinkle some assertions in various places to assert that received
packets don't have a send tag, and that other places that overwrite
rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.

Reviewed by: gallatin, hselasky, rgrimes, ae
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D20117


# 7687707d 22-Apr-2019 Andrew Gallatin <gallatin@FreeBSD.org>

Track device's NUMA domain in ifnet & alloc ifnet from NUMA local memory

This commit adds new if_alloc_domain() and if_alloc_dev() methods to
allocate ifnets. When called with a domain on a NUMA machine,
ifalloc_domain() will record the NUMA domain in the ifnet, and it will
allocate the ifnet struct from memory which is local to that NUMA
node. Similarly, if_alloc_dev() is a wrapper for if_alloc_domain
which uses a driver supplied device_t to call ifalloc_domain() with
the appropriate domain.

Note that the new if_numa_domain field fits in an alignment pad in
struct ifnet, and so does not alter the size of the structure.

Reviewed by: glebius, kib, markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D19930


# be7eaf97 22-Apr-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Make sure bundled_fw is always initialized before use.

This fixes a bug that prevented the driver from auto-flashing the
firmware when it didn't see one on the card. This feature was
introduced in r321390 and this bug was introduced in r343269.

Reported by: gallatin@
MFC after: 1 week
Sponsored by: Chelsio Communications


# be09e82a 29-Mar-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe/t4_tom: Catch up with r344433, which removed tcb_autorcvbuf_inc.

The declaration in tcp_var.h is still around so t4_tom continued to
compile but wouldn't load. A separate commit will fix tcp_var.h

Reported By: Dustin Marquess (dmarquess at gmail)

Sponsored by: Chelsio Communications


# dd3b96ec 28-Mar-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Count and clear interrupts generated at the software's request.

An interrupt can be requested by setting the F_SWINT bit in PL_PF_CTL.

MFC after: 1 week
Sponsored by: Chelsio Communications


# edb518f4 20-Mar-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Treat the viid as an opaque identifier.

Recent firmwares prefer to use a different format for viid internally
and this change allows them to do so.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 41dda0d9 28-Feb-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Don't forget to report link state to the kernel if the link is
already up at attach.

Reported by: Fabrice Bruel @ Orange Business Service
MFC after: 1 week
Sponsored by: Chelsio Communications


# d18e5419 28-Feb-2019 John Baldwin <jhb@FreeBSD.org>

Don't assume all children of a nexus are ports.

Specifically, ccr(4) devices are also children of cxgbe nexus devices.
Rather than making assumptions about the child device's softc, walk
the list of ports from the nexus' softc to determine if a child is a
port in t4_child_location_str(). This fixes a panic when detaching a
ccr device.

Reviewed by: np
MFC after: 1 week
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D19399


# a0989590 27-Feb-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Request high priority filter support explicitly, as required
by recent firmwares.

MFC after: 1 week
Sponsored by: Chelsio Communications


# a71c41cc 08-Feb-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Delay the panic due to a fatal error by 30s.

This lets information logged by the interrupt handler reach the system
log before the system goes down.


# 644b22ae 06-Feb-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Auto-dump the CIM block's logic analyzer on a TIMER0 interrupt.

Sponsored by: Chelsio Communications


# 286fd42b 06-Feb-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Auto-dump the device log on a mailbox timeout or when the
firmware reports an error in pcie_fw.

Sponsored by: Chelsio Communications


# cb7c3f12 01-Feb-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Improved error reporting and diagnostics.

"slow" interrupt handler:
- Expand the list of INT_CAUSE registers known to the driver.
- Add decode information for many more bits but decouple it from the
rest of intr_info so that it is entirely optional.
- Call t4_fatal_err exactly once, and from the top level PL intr handler.

t4_fatal_err:
- Use t4_shutdown_adapter from the common code to stop the adapter.
- Stop servicing slow interrupts after the first fatal one.

Driver/firmware interaction:
- CH_DUMP_MBOX: note whether the mailbox being dumped is a command or a
reply or something else.
- Log the raw value of pcie_fw for some errors.
- Use correct log levels (debug vs. error).

Sponsored by: Chelsio Communications


# cecf8beb 25-Jan-2019 John Baldwin <jhb@FreeBSD.org>

Fix a few more places to handle ofld tx queues for RATELIMIT.

- Drain offload transmit queues when RATELIMIT is enabled but
TCP_OFFLOAD is not.
- Expose the per-VI nofldtxq and first_ofld_txq sysctls when
RATELIMIT is enabled but TCP_OFFLOAD is not.
- Clear offload transmit queue stats as part of a 'cxgbetool clearstats'
request when RATELIMIT is enabled but TCP_OFFLOAD is not.

Reviewed by: np
MFC after: 2 weeks
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D18966


# 2a857082 21-Jan-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Allow negative values in hw.cxgbe.fw_install and take them to
mean that the driver should taste the firmware in the KLD and use that
firmware's version for all its fw_install checks.

The driver gets firmware version information from compiled-in values by
default and this change allows custom (or older/newer) firmware modules
to be used with the stock driver.

There is no change in default behavior.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 8d510676 21-Jan-2019 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Use a truncated firmware header for version checks. All the
version numbers are towards the begining of the header.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 450ffb7c 29-Dec-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Attach to two T540 variants.

MFC after: 1 week


# 121684b7 19-Dec-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe/iw_cxgbe: Use DSGLs to write to card's memory when appropriate.

Submitted by: Krishnamraju Eraparaju @ Chelsio
Sponsored by: Chelsio Communications


# f02cc9b2 05-Dec-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fall back to a basic configuration in case of any error during
card initialization. This is an expanded version of r333682.

Break up prep_firmware into simpler routines while here. Load the
firmware/config KLD only if needed.

MFC after: 1 month
Sponsored by: Chelsio Communications


# 31562c44 29-Nov-2018 John Baldwin <jhb@FreeBSD.org>

Make most of the CLIP code conditional on #ifdef INET6.

This fixes builds of kernels without INET6 such as LINT-NOINET6.

Reported by: arybchik
Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D18384


# 78afed13 28-Nov-2018 John Baldwin <jhb@FreeBSD.org>

Move CLIP table handling out of TOM and into the base driver.

- Store the clip table in 'struct adapter' instead of in the TOM softc.
- Init the clip table during attach and teardown during detach.
- While here, add a dev.<nexus>.<unit>.misc.clip sysctl to dump the
CLIP table.

This does mean that we update the clip table even if TOE is not enabled,
but non-TOE things need the CLIP table anyway.

Reviewed by: np, Krishnamraju Eraparaju @ Chelsio
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D18010


# 43cf589c 28-Nov-2018 Vincenzo Maffione <vmaffione@FreeBSD.org>

cxgbe: revert r309725

After the fix contained in r341144, cxgbe does not need anymore
to set the IFCAP_NETMAP flag manually.

Reviewed by: np
Approved by: gnn (mentor)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D17987


# 2d714dbc 27-Nov-2018 John Baldwin <jhb@FreeBSD.org>

Add read-only sysctls for all tunables in the cxgbe(4) driver.

Reviewed by: np
MFC after: 1 month
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D18360


# 2939ecd3 15-Nov-2018 John Baldwin <jhb@FreeBSD.org>

Change the quantum for TLS key addresses to 32 bytes.

The addresses passed when reading and writing keys are always shifted
right by 5 as the memory locations are addressed in 32-byte chunks, so
the quantum needs to be 32, not 8.

MFC after: 1 month
Sponsored by: Chelsio Communications


# bc13c69b 15-Nov-2018 John Baldwin <jhb@FreeBSD.org>

Move the TLS key map into the adapter softc so non-TOE code can use it.

Sponsored by: Chelsio Communications


# 2f3736eb 06-Nov-2018 John Baldwin <jhb@FreeBSD.org>

Treat the memory lengths for CHELSIO_T4_GET_MEM as unsigned.

Previously attempts to read the MC region were failing since the
length was greater than 2^31.

Reviewed by: np
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D17857


# 5cdaef71 05-Nov-2018 John Baldwin <jhb@FreeBSD.org>

Add a facility for transmitting "raw" work requests on regular NIC queues.

- Use PH_loc.eight[1] as a general 'cflags' (Chelsio flags) field to
describe properties of a queued packet. The MC_RAW_WR flag
indicates an mbuf holding a raw work request. mbuf_cflags() returns
the current flags.
- Raw work request mbufs are allocated via alloc_wr_mbuf() which will
allocate a single contiguous range to hold the mbuf data. The
consumer can use mtod() to obtain the start of the work request and
write the required work request in the buffer. The mbuf can then be
enqueued directly to the txq via mp_ring_enqueue().
- Since raw work requests might potentially send arbitrary work
requests, only set the EQUIQ and EQUEQ bits on work requests that
support them such as the normal tunneled Ethernet packet work
requests.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D17811


# 1d8d91db 01-Nov-2018 John Baldwin <jhb@FreeBSD.org>

Add support for port unit wiring to cxgbe(4).

- Add a bus_child_location_str method to the nexus drivers that prints
out 'port=N' as the location string exported via devinfo and the
'%location' sysctl node.

- We can't use a bus_hint_device_unit to wire the unit numbers of
devices with a fixed devclass as the device gets assigned a unit in
make_device() before the device creator can set softc, etc.
Instead, when adding a child device, use a helper function much like
a bus_hint_device_unit method to look for wiring hints or to return
-1 to let the system choose a unit number. This function requires
an "at" hint for the port pointing to the nexus device and a "port"
hint listing the port number. For example:

hint.cxl.4.at="t5nex0"
hint.cxl.4.port="0"

wires cxl4 to the first port on the t5nex0 adapter.

Requested by: gallatin
MFC after: 2 months


# 1272051e 31-Oct-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Report a reasonable non-zero if_hw_tsomaxsegsize to the
kernel.

This reverts an accidental change that snuck in with r339628.

Sponsored by: Chelsio Communications


# d54dafc6 25-Oct-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Allow "pass" filters to distribute matching traffic using a
subset of a VI's RSS indirection table.

This makes it possible to make groups out of rx queues and steer
different kinds of traffic to different groups. For example, an
interface with 8 rx queues could have all non-TCP traffic delivered to
queues 0-3 and all TCP traffic to queues 4-7.

Note that it is already possible for filters to steer traffic to a
particular queue or to distribute it using the full indirection table
(much like normal rx does).

Sponsored by: Chelsio Communications


# b77aaff9 25-Oct-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Update the VI's default queue when netmap is enabled/disabled.

Sponsored by: Chelsio Communications


# b5da13f7 24-Oct-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): new sysctl to display the start of the RSS region for a VI.

dev.<ifname>.<inst>.rss_base

For example:
dev.cc.0.rss_base: 0
dev.cc.1.rss_base: 128
dev.vcc.0.rss_base: 256
dev.vcc.1.rss_base: 384

Sponsored by: Chelsio Communications


# 17e81b78 22-Oct-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): improve the accuracy of various TSO limits reported to the kernel.

Sponsored by: Chelsio Communications


# ac4031d8 26-Sep-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Enable support for per-connection rate limiting in the default
firmware configuration files.

Approved by: re@ (gjb@)
Sponsored by: Chelsio Communications


# cb8dedc7 25-Sep-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Treat base/end of firmware parameters as signed integers when
figuring out whether the range is valid or not.

Approved by: re@ (rgrimes@)
MFC after: 1 week
Sponsored by: Chelsio Communications


# ea710848 24-Sep-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Link related changes.

- Switch to using 32b port/link capabilities in the driver. The 32b
format is used internally by firmwares > 1.16.45.0 and the driver will
now interact with the firmware in its native format, whether it's 16b
or 32b. Note that the 16b format doesn't have room for 50G, 200G, or
400G speeds.

- Add a bit in the pause_settings knobs to allow negotiated PAUSE
settings to override manual settings.

- Ensure that manual link settings persist across an administrative
down/up as well as transceiver unplug/replug.

- Remove unused is_*G_port() functions.

Approved by: re@ (gjb@)
MFC after: 1 month
Sponsored by: Chelsio Communications


# 5f65b4ca 18-Sep-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Enable TXRTLMT by default when the feature is available in the
kernel (options RATELIMIT) and provisioned in the driver's configuration
file (nethofld > 0).

Submitted by: gallatin@
Approved by: re@ (kib@)


# 4bb64e96 13-Sep-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Use the correct number of parameters when querying the tid
range for hashfilters.

Approved by: re@ (gjb@)


# b8bfcb71 22-Aug-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbev(4): Updates to the VF driver to cope with recent ifmedia and
ctrlq changes in the base driver.

MFC after: 1 week
Sponsored by: Chelsio Communications


# e758d775 21-Aug-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Do not leak memory in case of errors during VI initialization.

Reported by: Coverity (CID 1392026)
MFC after: 1 week
Sponsored by: Chelsio Communications


# 036ff794 21-Aug-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Check the RO bit properly before disabling relaxed ordering.

Reported by: Coverity (CID 1384286)
MFC after: 1 week
Sponsored by: Chelsio Communications


# e7e08444 17-Aug-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Replace T4_PKT_TIMESTAMP with something slightly less hackish.


# a56e2056 17-Aug-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Adjust ntids to account for nhptids in the TOE case too.
This should have been part of r337538.


# 9f784349 15-Aug-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Use VLAN_TRUNKDEV instead of private cookie to figure out the
parent of a VLAN ifnet.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 51347c3f 14-Aug-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Use two hashes instead of a table to keep track of
hashfilters. Two because the driver needs to look up a hashfilter by
its 4-tuple or tid.

A couple of fixes while here:
- Reject attempts to add duplicate hashfilters.
- Do not assume that any part of the 4-tuple that isn't specified is 0.
This makes it consistent with all other mandatory parameters that
already require explicit user input.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# 37310a98 11-Aug-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Move all control queues to the adapter.

There used to be one control queue per adapter (the mgmtq) that was
initialized during adapter init and one per port that was initialized
later during port init. This change moves all the control queues (one
per port/channel) to the adapter so that they are initialized during
adapter init and are available before any port is up. This allows the
driver to issue ctrlq work requests over any channel without having to
bring up any port.

MFH: 2 weeks
Sponsored by: Chelsio Communications


# 3098bcfc 10-Aug-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Create two variants of service_iq, one for queues with
freelists and one for those without.

MFH: 3 weeks
Sponsored by: Chelsio Communications


# 5fc0f72f 09-Aug-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add support for high priority filters on T6+. They have their
own region in the TCAM starting with T6, unlike previous chips where
they were in the same region as normal filters.

These filters "hit" before anything else in the LE's lookup. The exact
order is:
a) High priority filters
b) TOE's active region (TCAM and/or hash)
c) Servers (TOE hw listeners)
d) Normal filters

MFC after: 1 week
Sponsored by: Chelsio Communications


# 09a7189f 07-Aug-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Allow the driver to specify a burst size when configuring a
traffic class for rate limiting.

Add experimental knobs that allow the user to specify a default pktsize
and burstsize for traffic classes associated with a port:

dev.<ifname>.<instance>.tc.pktsize
dev.<ifname>.<instance>.tc.burstsize

Sponsored by: Chelsio Communications


# 1979b511 06-Aug-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Allow user-configured and driver-configured traffic classes to
be used simultaneously. Move sysctl_tc and sysctl_tc_params to
t4_sched.c while here.

MFC after: 3 weeks
Sponsored by: Chelsio Communications


# 7b8f5a20 06-Aug-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Break up sysctl_bitfield into 8 bit and 16 bit variants. Have
them display the current value of the bitfield rather than the fixed
value that was provided when the sysctl node was created.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 0c71c9cc 02-Aug-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Improvements in TID management.

- Ignore any type of TID where the start/end values are not in the
correct order. There are situations where the firmware isn't able to
reserve room for the number requested in the config file but doesn't
report a failure during configuration and instead sets end <= start.

- Track start/end in tid_tab and remove some redundant copies from
adapter->params.

- Move all the start/end and other read-only parameters to a quiet part
of tid_tab, away from the tid locks.

MFC after: 1 week
Sponsored by: Chelsio Communications


# aa8c29e5 26-Jul-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Consider rateunit before ratemode when displaying information
about a traffic class. This matches the order in which the firmware
evaluates unit and mode internally.

Sponsored by: Chelsio Communications


# 069262a7 10-Jul-2018 Navdeep Parhar <np@FreeBSD.org>

Fix vertical whitespace nit in cxgbe.


# 82df14c3 09-Jul-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add a sysctl to report the chip's microprocessor's load
averages. This works with debug or custom firmwares only.

sysctl dev.<nexus>.<instance>.loadavg
sysctl dev.t6nex.0.loadavg

MFC after: 1 month
Sponsored by: Chelsio Communications


# af8854fd 26-Jun-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Do not leak the filters in the hashfilter table on module
unload.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 6ddad9de 15-Jun-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): sysctls to display the local and intr CPUs for the adapter.

The driver assumes the list can change (even though it does't right now)
and queries it every time the sysctl runs.

sysctl dev.<nexus>.<inst>.local_cpus
sysctl dev.<nexus>.<inst>.intr_cpus

sysctl dev.t6nex.0.local_cpus
sysctl dev.t6nex.0.intr_cpus

Sponsored by: Chelsio Communications


# 2ea5b0f5 13-Jun-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Catch up with recent changes in the kernel -- it no longer
holds non-sleepable locks around any of the driver ioctls.

Sponsored by: Chelsio Communications


# 1bb577b4 12-Jun-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Remove homemade version of htobe32 from the driver.

It was needed only for ia64 where it was implemented as a call to
bswapXX, which was always a real function. htobeXX with a constant
argument is calculated at compile-time everywhere else.

MFC after: 1 week
Sponsored by: Chelsio Communications


# c27fcc70 01-Jun-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Include full duplex mediaopt in media that can be reported as
active. Always report full duplex in active media.

Sponsored by: Chelsio Communications


# b9330ed7 31-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Retire an old check.


# 2dae2a74 31-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add code to deal with the chip's source MAC table (aka SMT).

Submitted by: Krishnamraju Eraparaju @ Chelsio
Sponsored by: Chelsio Communications


# 1e3e6b63 30-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Use ifm for ifmedia just like the rest of the kernel.

No functional change.


# 7cff4fd2 30-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Implement ifm_change callback.

Sponsored by: Chelsio Communications


# 56226f56 30-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Consider all supported speeds when building the ifmedia list
for a port. Fix other related issues while here:
- Require port lock for access to link_config.
- Allow 100Mbps operation by tracking the speed in Mbps. Yes, really.
- New port flag to indicate that the media list is immutable. It will
be used in future refinements.

This also fixes a bug where the driver reports incorrect media with
recent firmwares.

MFC after: 2 days
Sponsored by: Chelsio Communications


# 475d42db 24-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Report IFCAP_TXRTLMT to kernels built with RATELIMIT if the
firmware has provisioned resources for this feature.

Sponsored by: Chelsio Communications


# 786099de 24-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Data path for rate-limited tx.

This is hardware support for the SO_MAX_PACING_RATE sockopt (see
setsockopt(2)), which is available in kernels built with "options
RATELIMIT".

Relnotes: Yes
Sponsored by: Chelsio Communications


# 1dd95f64 24-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix range checks in is_etid.


# d7c5a620 18-May-2018 Matt Macy <mmacy@FreeBSD.org>

ifnet: Replace if_addr_lock rwlock with epoch + mutex

Run on LLNW canaries and tested by pho@

gallatin:
Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5
based ConnectX 4-LX NIC, I see an almost 12% improvement in received
packet rate, and a larger improvement in bytes delivered all the way
to userspace.

When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1,
I see, using nstat -I mce0 1 before the patch:

InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree
4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32
4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32
4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32
4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32
4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32
4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32
4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32

After the patch

InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree
5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51
5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51
5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51
5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51
5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52
5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52

Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch

Reviewed by: gallatin
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D15366


# 67e07112 18-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Implement ifnet callbacks that deal with send tags.

An etid (ethoffload tid) is allocated for a send tag and it acquires a
reference on the traffic class that matches the send parameters
associated with the tag.

Sponsored by: Chelsio Communications


# d88a0442 17-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix s->neq miscalculation in r333698.


# eff62dba 16-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Allocate offload Tx queues when a card has resources
provisioned for NIC_ETHOFLD and the kernel has option RATELIMIT.

It is possible to use the chip's offload queues for normal NIC Tx and
not just TOE Tx. The difference is that these queues support out of
order processing of work requests and have a per-"flowid" mechanism for
tracking credits between the driver and hardware. This allows Tx for
any number of flows bound to different rate limits to be submitted to a
single Tx queue and the work requests for slow flows won't cause HOL
blocking for the rest.

Sponsored by: Chelsio Communications


# 93c0bfb8 16-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add NIC_ETHOFLD to the NIC capabilities allowed by the driver
by default.

This is the first of a series of commits that will add support for
RATELIMIT kernel option to the base if_cxgbe driver, for use with
ordinary NIC traffic "flows". RATELIMIT is already supported by t4_tom
for the fully-offloaded TCP connections that it handles.

Sponsored by: Chelsio Communications


# 47ae7a7e 16-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fall back to a failsafe configuration built into the firmware
if an error is reported while pre-processing the configuration file that
the driver attempted to use.

Also, allow the user to explicitly use the built-in configuration with
hw.cxgbe.config_file="built-in"

MFC after: 2 days
Sponsored by: Chelsio Communications


# 40f242e4 15-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Claim some more T5 and T6 boards.

MFC after: 2 days
Sponsored by: Chelsio Communications


# f7a203bc 10-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Disable write-combined doorbells by default.

This had been the default behavior but was changed accidentally as part
of the recent iw_cxgbe+OFED overhaul. Fix another bug in that change
while here: the global knob affects all the adapters in the system and
should be left alone by per-adapter code.

MFC after: 3 days
Sponsored by: Chelsio Communications


# 5174205d 09-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Determine whether the firmware supports the FILTER2 work
request, which can be used to configure hardware NAT and swapmac.

All firmwares released after Jan 2017 support this work request.

Sponsored by: Chelsio Communications


# 89f651e7 08-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add support for hash filters.

These filters reside in the card's memory instead of its TCAM and can be
configured via a new "hashfilter" subcommand in cxgbetool. Hash and
normal TCAM filters can be used together. The hardware does an
exact-match of packet fields for hash filters, unlike the masked match
performed for TCAM filters. Any T5/T6 card with memory can support at
least half a million hash filters. The sample config file with the
driver configures 512K of these, it is possible to double this to 1
million+ in some cases.

The chip does an exact-match of fields of incoming datagrams with hash
filters and performs the action configured for the filter if it matches.
The fields to match are specified in a "filter mask" in the firmware
config file. The filter mask always includes the 5-tuple (sip, dip,
sport, dport, ipproto). It can, optionally, also include any subset of
the filter mode (see filterMode and filterMask in the firmware config
file).

For example:
filterMode = fragmentation, mpshittype, protocol, vlan, port, fcoe
filterMask = protocol, port, vlan

Exact values of the 5-tuple, the physical port, and VLAN tag would have
to be provided while setting up a hash filter with the chip
configuration above.

Hash filters support all actions supported by TCAM filters. A packet
that hits a hash filter can be dropped, let through (with optional
steering to a specific queue or RSS region), switched out of another
port (with optional L2 rewrite of DMAC, SMAC, VLAN tag), or get NAT'ed.
(Support for some of these will show up in the driver in a follow-up
commit very shortly).

Sponsored by: Chelsio Communications


# b6f2c452 05-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Update all firmwares to 1.19.1.0.

These firmwares and the following list of changes are from the public
ChelsioUwire-3.7.1.0 release.

T6 Firmware
================================================================================
Version : 1.19.1.0
Date : 04/23/2018
================================================================================

Fixes
-----

BASE:
- Fixed traffic stall when rate-limit is modified while running traffic.
- Fixes a firmware crash in FW_ETH_TX_EO_WR handling.
- Fixes host DCB support when FW_PORT_CMD is used.

ETH:
- Exit Auto-Negotiation if we don't receive base page from peer within 10s.
This fixes some cases where in we keep on restarting auto negotiation without
ever exiting, resulting in link failure.
- Fixes an issue where VF packets counter were not increasing if VF packets
coalesced WR is used by driver.

OFLD:
- Kernel and user mode NVMEoF performance enhancements.

FOiSCSI:
- Fixes fw crash when trying to connect to non-existence IPv6 iSNS target.

================================================================================
Version : 1.18.9.0
Date : 03/27/2018
================================================================================

Fixes
-----

BASE:
- For Ethernet frames less than 64B, pad them with zero bytes as per IEEE spec
(RFC 894).
- Added a new parameter iqtype to FW_IQ_CMD to identify the ingress NIC or offload
queues. This fixes an issue where driver was receiving interrupt with no new
messages in queue.
- FW_PARAMS_CMD processes all the valaid paramaters and returns value 0UL for
any unknown parameter.

OFLD:
- Fixes connection failure during SRQ reuse.
- Fixes incorrect cqe in case of WRITE with immediate operation.

FOiSCSI:
- Fixes a fw crash when wrong node-id is passed to FW_FOISCSI_CTRL_WR.

FOFCoE:
- Fixes a fw hang while creating NPIV.

Enhancements
------------

ETH:
- A new WR FW_ETH_TX_PKTS_VM_WR added to support VM packet coalescing.

================================================================================
Version : 1.18.4.0
Date : 02/28/2018
================================================================================

Fixes
-----

BASE:
- Fixed Rate limiting not working for 101Mbps<=rate limit<=163Mbps range.
- Fixed starting more than 32 VMs on PF4 causing firmware hang.

ETH:
- Fixed link failure due to FEC mismatch with optics.
- Fixed link failure with link toggle stress tests.
- Only BaseR FEC is supported for 50G.
- Fixed a bug in next page handling which sometimes causes link down.
- Fixed port down due to failre to read eeprom contents of some modules.
- Fixed a bug causing adapter to fail with spider configuration.

FOiSCSI:
- Fixed a bug causing login failure when connecting to multiple targets.

Enhancements
------------

BASE:
- Added a new firmware API to retrieve the maximum temperaturethreshold for
the chip (FW_PARAM_DEV_DIAG_MAXTMPTHRESH).

ETH:
- Added support for user to contol pause negotiation during auto negotiation.

FOiSCSI:
- Added a new facility to redirect few fw events to offload rx queue
(based on driver's configration)
- Driver can ignore providing ipv6 prefix len during ipv6 address configuration.

================================================================================
Version : 1.17.14.0
Date : 12/27/2017
================================================================================

FIXES
-----

BASE:
- Fixed an FLR failure during simulteneous power up of VM.
- Fixed an issue in vlan acl which was limiting vlan range to 1024.

ETH:
- Enabled RS-FEC for 25G active copper cable and 25GBASE-SR.
- When auto negotiation is enabled, final pause settings are resolved
based on local and peer pause settings.
- Handle NACK for an I2C access.

OFLD
- Fixed rdma connection cleanup in SO adpater.
- Fixed rdma connections during read invalidate.
- Fixed the crash when invalid BW rate is passed to fw.
- Fixed the traffic hang when BW allocation is changed from switch during traffic.

FOFCoE:
- Fixed an issue where initiator remains logged-in even after LLDP is disabled
on switch.

ENHANCEMENTS
------------

BASE:
- Added support for 248 VFs.
- Added fw driver periodic calibration for MC.

ETH:
- Added XLAUI port type support.
- Added raw mac entry deletion support (FW_VI_MAC_ID_BASED_FREE).

OFLD:
- Inline IPSec support added (flag F_FW_ULPTX_WR_DATA indicates the inline
IPSec WR).
- New work request FW_RI_RDMA_WRITE_CMPL_WR (write with completion) added to

T5 Firmware
================================================================================
Version : 1.19.1.0
Date : 04/23/2018
================================================================================

Fixes
-----

BASE:
- Fixes a firmware crash in FW_ETH_TX_EO_WR handling.
- Fixes host DCB support when FW_PORT_CMD is used.

ETH:
- Fixes an issue where VF packets counter were not increasing if VF packets
coalesced WR is used by driver.

OFLD:
- Fixes an issue where fw hangs if max traffic rate passed is 0.

FOiSCSI:
- Fixes fw crash when trying to connect to non-existence IPv6 iSNS target.

================================================================================
Version : 1.18.9.0
Date : 03/27/2018
================================================================================

Fixes
-----

BASE:
- For Ethernet frames less than 64B, pad them with zero bytes as per IEEE spec
(RFC 894).
- Added a new parameter iqtype to FW_IQ_CMD to identify the ingress NIC or offload
queues. This fixes an issue where driver was receiving interrupt with no new
messages in queue.

ETH:
- Pad the Ethernet packets of size less than 64B with zeros. This fixes the
incorrect checksum generation of packets less then 64B.

FOiSCSI:
- Fixes a fw crash when wrong node-id is passed to FW_FOISCSI_CTRL_WR.

FOFCoE:
- Fixes a fw hang while creating NPIV.

Enhancements
------------

ETH:
- A new WR FW_ETH_TX_PKTS_VM_WR added to support VM packet coalescing.

================================================================================
Version : 1.18.4.0
Date : 02/28/2018
================================================================================

Fixes
-----

BASE:
- Fixed starting more than 32 VMs on PF4 causing firmware hang.

FOiSCSI:
- Fixed a bug causing login failure when connecting to multiple targets.

Enhancements
------------

BASE:
- Added a new firmware API to retrieve the maximum temperaturethreshold for
the chip (FW_PARAM_DEV_DIAG_MAXTMPTHRESH).

ETH:
- Added support for user to contol pause negotiation during auto negotiation.

FOiSCSI:
- Added a new facility to redirect few fw events to offload rx queue
(based on driver's configration)
- Driver can ignore providing ipv6 prefix len during ipv6 address configuration.

================================================================================
Version : 1.17.14.0
Date : 12/27/2017
================================================================================

FIXES
-----

BASE:
- Fixed an issue in vlan acl which was limiting vlan range to 1024.

ETH:
- Corrected lane inversion logic.
- Fixed improper LED behavior in T580 cards.
- When auto negotiation is enabled, final pause settings are resolved
based on local and peer pause settings.
- Handle NACK for an I2C access.

OFLD
- Fixed rdma connections during read invalidate.

FOiSCSI:
- Fixed a connections hang when link is toggled frequently.

FOFCoE:
- Fixed an issue where initiator remains logged-in even after LLDP is disabled
on switch.

ENHANCEMENTS
------------

BASE:
- Added support for 124 VFs.

ETH:
- Added XLAUI port type support.
- Added raw mac entry deletion support (FW_VI_MAC_ID_BASED_FREE).

OFLD:
- New work request FW_RI_RDMA_WRITE_CMPL_WR (write with completion) added to
optimize NVMEoF write.

T4 Firmware
================================================================================
Version : 1.19.1.0
Date : 04/23/2018
================================================================================

Fixes
-----

BASE:
- Fixes a firmware crash in FW_ETH_TX_EO_WR handling.
- Fixes host DCB support when FW_PORT_CMD is used.

FOiSCSI:
- Fixes fw crash when trying to connect to non-existence IPv6 iSNS target.

================================================================================
Version : 1.18.9.0
Date : 03/27/2018
================================================================================

Fixes
-----

BASE:
- Added a new paramter iqtype to FW_IQ_CMD to identify the ingress NIC or
offload queues. This fixes an issue where driver was receiving interrupt with
no new messages in queue.

FOFCoE:
- Fixes a fw hang while creating NPIV.

Enhancements
------------

ETH:
- A new WR FW_ETH_TX_PKTS_VM_WR added to support VM packet coalescing.

================================================================================
Version : 1.18.4.0
Date : 02/28/2018
================================================================================

Enhancements
------------

BASE:
- Added a new firmware API to retrieve the maximum temperaturethreshold for
the chip (FW_PARAM_DEV_DIAG_MAXTMPTHRESH).

================================================================================
Version : 1.17.14.0
Date : 12/27/2017
================================================================================

FIXES
-----

BASE:
- Fixed an issue in vlan acl which was limiting vlan range to 1024.

MFC after: 3 days
Sponsored by: Chelsio Communications


# e1320420 01-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Move all TCAM filter code into a separate file.

Sponsored by: Chelsio Communications


# a378e594 01-May-2018 Andrew Gallatin <gallatin@FreeBSD.org>

Optionally panic when cxgbe encounters a fatal error

Sometimes it is better to panic than to leave a machine
unreachable.

Reviewed by: np
Sponsored by: Netflix


# faf6d96b 01-May-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Destroy the cdev before disabling interrupts in driver detach.

Filter work requests are submitted in the nexus cdev's ioctl which then
blocks waiting for a reply. If driver detach runs in this state and
disables interrupts the ioctl will never complete and detach will hang
in destroy_cdev.

Sponsored by: Chelsio Communications


# 111638bf 30-Apr-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Convert ACT_OPEN_RPL to a shared CPL.

Reserve 3b in the 14b atid to identify the owner and use it to dispatch
the CPL. This allows all CPLs that use an atid to be used as shared
CPLs, although ACT_OPEN_RPL is the only one being converted in this
revision.

Sponsored by: Chelsio Communications


# 4535e804 30-Apr-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Use opaque cookies or tid range-checks to determine the
intended recipient of a CPL when it can't be determined solely from the
opcode. Retire the per-queue handlers for such CPLs in favor of the new
scheme.

Sponsored by: Chelsio Communications


# 8896672a 26-Apr-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Move release_tid to the base NIC driver for future consumers.

Sponsored by: Chelsio Communications.


# 3747c1ff 26-Apr-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Break up alloc_tid_tabs and move the atid routines to the base
NIC driver. The atid services will be used by new features (hashfilters
and inline TLS) that do not involve TOE.

Sponsored by: Chelsio Communications


# 1131c927 14-Apr-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add support for Connection Offload Policy (aka COP).

COP allows fine-grained control on whether to offload a TCP connection
using t4_tom, and what settings to apply to a connection selected for
offload. t4_tom must still be loaded and IFCAP_TOE must still be
enabled for full TCP offload to take place on an interface. The
difference is that IFCAP_TOE used to be the only knob and would enable
TOE for all new connections on the inteface, but now the driver will
also consult the COP, if any, before offloading to the hardware TOE.

A policy is a plain text file with any number of rules, one per line.
Each rule has a "match" part consisting of a socket-type (L = listen,
A = active open, P = passive open, D = don't care) and a pcap-filter(7)
expression, and a "settings" part that specifies whether to offload the
connection or not and the parameters to use if so. The general format
of a rule is: [socket-type] expr => settings

Example. See cxgbetool(8) for more information.
[L] ip && port http => offload
[L] port 443 => !offload
[L] port ssh => offload
[P] src net 192.168/16 && dst port ssh => offload !nagle !timestamp cong newreno
[P] dst port ssh => offload !nagle ecn cong tahoe
[P] dst port http => offload
[A] dst port 443 => offload tls
[A] dst net 192.168/16 => offload !timestamp cong highspeed

The driver processes the rules for each new listen, active open, or
passive open and stops at the first match. There is an implicit rule at
the end of every policy that prohibits offload when no rule in the
policy matches:
[D] all => !offload

This is a reworked and expanded version of a patch submitted by
Krishnamraju Eraparaju @ Chelsio.

Sponsored by: Chelsio Communications


# de933532 04-Apr-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Always display an error message if SIOCSIFFLAGS will leave
IFF_UP and IFF_DRV_RUNNING out of sync. ifhwioctl in the kernel pays no
attention to the return code from the driver ioctl during SIOCSIFFLAGS
so these messages are the only indication that the ioctl was called but
failed.

MFC after: 1 week
Sponsored by: Chelsio Communications


# f8fea0d9 02-Apr-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe: Implement tcp_info handler for connections handled by t4_tom.

The TCB is read using a memory window right now. A better alternate to
get self-consistent, uncached information would be to use a GET_TCB
request but waiting for a reply from hw while holding non-sleepable
locks is quite inconvenient.

Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D14817


# 541d96aa 30-Mar-2018 Brooks Davis <brooks@FreeBSD.org>

Use an accessor function to access ifr_data.

This fixes 32-bit compat (no ioctl command defintions are required
as struct ifreq is the same size). This is believed to be sufficent to
fully support ifconfig on 32-bit systems.

Reviewed by: kib
Obtained from: CheriBSD
MFC after: 1 week
Relnotes: yes
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14900


# d57241d2 23-Mar-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Always initialize requested_speed to a valid value.

This fixes an avoidable EINVAL when the user tries to disable AN after
the port is initialized but l1cfg doesn't have a valid speed to use.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 5401e096 21-Mar-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Tunnel congestion drops on a port should be cleared when the
stats for that port are cleared.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 1e9538d2 13-Mar-2018 John Baldwin <jhb@FreeBSD.org>

Support for TLS offload of TOE connections on T6 adapters.

The TOE engine in Chelsio T6 adapters supports offloading of TLS
encryption and TCP segmentation for offloaded connections. Sockets
using TLS are required to use a set of custom socket options to upload
RX and TX keys to the NIC and to enable RX processing. Currently
these socket options are implemented as TCP options in the vendor
specific range. A patched OpenSSL library will be made available in a
port / package for use with the TLS TOE support.

TOE sockets can either offload both transmit and reception of TLS
records or just transmit. TLS offload (both RX and TX) is enabled by
setting the dev.t6nex.<x>.tls sysctl to 1 and requires TOE to be
enabled on the relevant interface. Transmit offload can be used on
any "normal" or TLS TOE socket by using the custom socket option to
program a transmit key. This permits most TOE sockets to
transparently offload TLS when applications use a patched SSL library
(e.g. using LD_LIBRARY_PATH to request use of a patched OpenSSL
library). Receive offload can only be used with TOE sockets using the
TLS mode. The dev.t6nex.0.toe.tls_rx_ports sysctl can be set to a
list of TCP port numbers. Any connection with either a local or
remote port number in that list will be created as a TLS socket rather
than a plain TOE socket. Note that although this sysctl accepts an
arbitrary list of port numbers, the sysctl(8) tool is only able to set
sysctl nodes to a single value. A TLS socket will hang without
receiving data if used by an application that is not using a patched
SSL library. Thus, the tls_rx_ports node should be used with care.
For a server mostly concerned with offloading TLS transmit, this node
is not needed as plain TOE sockets will fall back to software crypto
when using an unpatched SSL library.

New per-interface statistics nodes are added giving counts of TLS
packets and payload bytes (payload bytes do not include TLS headers or
authentication tags/MACs) offloaded via the TOE engine, e.g.:

dev.cc.0.stats.rx_tls_octets: 149
dev.cc.0.stats.rx_tls_records: 13
dev.cc.0.stats.tx_tls_octets: 26501823
dev.cc.0.stats.tx_tls_records: 1620

TLS transmit work requests are constructed by a new variant of
t4_push_frames() called t4_push_tls_records() in tom/t4_tls.c.

TLS transmit work requests require a buffer containing IVs. If the
IVs are too large to fit into the work request, a separate buffer is
allocated when constructing a work request. This buffer is associated
with the transmit descriptor and freed when the descriptor is ACKed by
the adapter.

Received TLS frames use two new CPL messages. The first message is a
CPL_TLS_DATA containing the decryped payload of a single TLS record.
The handler places the mbuf containing the received payload on an
mbufq in the TOE pcb. The second message is a CPL_RX_TLS_CMP message
which includes a copy of the TLS header and indicates if there were
any errors. The handler for this message places the TLS header into
the socket buffer followed by the saved mbuf with the payload data.
Both of these handlers are contained in tom/t4_tls.c.

A few routines were exposed from t4_cpl_io.c for use by t4_tls.c
including send_rx_credits(), a new send_rx_modulate(), and
t4_close_conn().

TLS keys for both transmit and receive are stored in onboard memory
in the NIC in the "TLS keys" memory region.

In some cases a TLS socket can hang with pending data available in the
NIC that is not delivered to the host. As a workaround, TLS sockets
are more aggressive about sending CPL_RX_DATA_ACK messages anytime that
any data is read from a TLS socket. In addition, a fallback timer will
periodically send CPL_RX_DATA_ACK messages to the NIC for connections
that are still in the handshake phase. Once the connection has
finished the handshake and programmed RX keys via the socket option,
the timer is stopped.

A new function select_ulp_mode() is used to determine what sub-mode a
given TOE socket should use (plain TOE, DDP, or TLS). The existing
set_tcpddp_ulp_mode() function has been renamed to set_ulp_mode() and
handles initialization of TLS-specific state when necessary in
addition to DDP-specific state.

Since TLS sockets do not receive individual TCP segments but always
receive full TLS records, they can receive more data than is available
in the current window (e.g. if a 16k TLS record is received but the
socket buffer is itself 16k). To cope with this, just drop the window
to 0 when this happens, but track the overage and "eat" the overage as
it is read from the socket buffer not opening the window (or adding
rx_credits) for the overage bytes.

Reviewed by: np (earlier version)
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D14529


# 6b554d26 27-Feb-2018 John Baldwin <jhb@FreeBSD.org>

Move #include for rijndael.h out of x86-specific region.

The #include was added inside of the conditional by accident and the lack
of it broke non-x86 builds.

Reported by: lwhsu (jenkins), andrew


# 52f8c526 26-Feb-2018 John Baldwin <jhb@FreeBSD.org>

Move ccr_aes_getdeckey() from ccr(4) to the cxgbe(4) driver.

This routine will also be used by the TOE module to manage TLS keys.

Sponsored by: Chelsio Communications


# 198729ea 26-Feb-2018 John Baldwin <jhb@FreeBSD.org>

Fetch TLS key parameters from the firmware.

The parameters describe how much of the adapter's memory is reserved for
storing TLS keys. The 'meminfo' sysctl now lists this region of adapter
memory as 'TLS keys' if present.

Sponsored by: Chelsio Communications


# ea3c5e5d 25-Jan-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Accept old names of a couple of tunables.


# 218c5b21 03-Jan-2018 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add a knob to enable/disable PCIe relaxed ordering. Disable it by
default when running on Intel CPUs.

This is a crude fix for the performance issues alluded to in these Linux commits:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=87e09cdec4dae08acdb4aa49beb793c19d73e73e
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a99b646afa8a02571ea298bedca6592d818229cd

MFC after: 1 week
Sponsored by: Chelsio Communications


# f549e352 22-Dec-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Do not forward interrupts to queues with freelists. This
leaves the firmware event queue (fwq) as the only queue that can take
interrupts for others.

This simplifies cfg_itype_and_nqueues and queue allocation in the driver
at the cost of a little (never?) used configuration. It also allows
service_iq to be split into two specialized variants in the future.

MFC after: 2 months
Sponsored by: Chelsio Communications


# 426b6bd5 21-Dec-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Read the MFG diags version from the VPD and make it available
in the sysctl MIB.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 718cf2cc 27-Nov-2017 Pedro F. Giffuni <pfg@FreeBSD.org>

sys/dev: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.


# 8e628d6d 20-Nov-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add a custom board to the device id list.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 879462e9 17-Nov-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add core Vdd to the sysctl MIB.

Sponsored by: Chelsio Communications


# d3d5e968 15-Nov-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Remove rsrv_noflowq from intrs_and_queues structure as it does
not influence or get affected by the number of interrupts or queues.

Sponsored by: Chelsio Communications


# d406c47d 15-Nov-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Sanitize t4_num_vis during MOD_LOAD like all other t4_*
tunables. Add num_vis to the intrs_and_queues structure as it affects
the number of interrupts requested and queues created. In future
cfg_itype_and_nqueues might lower it incrementally instead of going
straight to 1 when enough interrupts aren't available.

Sponsored by: Chelsio Communications


# 8c61c6bb 15-Nov-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Combine all _10g and _1g tunables and drop the suffix from
their names. The finer-grained knobs weren't practically useful.

Sponsored by: Chelsio Communications


# 5c2bacde 07-Nov-2017 Navdeep Parhar <np@FreeBSD.org>

Update the iw_cxgbe bits in the projects branch.

Submitted by: Krishnamraju Eraparaju @ Chelsio
Sponsored by: Chelsio Communications


# 5bcae8dd 23-Oct-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Read the MPS buffer group map from the firmware as it could be
different from hardware defaults. The congestion channel map, which is
still fixed, needs to be tracked separately now. Change the congestion
setting for TOE rx queues to match the drivers on other OSes while here.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# 08cd1f11 05-Oct-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Provide knobs to set the holdoff parameters of TOE rx queues
separately from NIC rx queues instead of using the same parameters for
both types of queues.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# 4fedff3c 18-Sep-2017 John Baldwin <jhb@FreeBSD.org>

Enable support for lookaside crypto operations by default.

This permits ccr(4) to be used with the default firmware configuration
file.

Discussed with: np
Sponsored by: Chelsio Communications


# efeb4688 13-Sep-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Ignore capabilities that depend on TOE when the firmware
reports TOE is not available.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 3ef74299 31-Aug-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe/t4_tom: Add a knob to select the congestion control algorigthm
used by the TOE hardware for fully offloaded connections. The knob
affects new connections only.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# 2f318252 30-Aug-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add two new debug flags -- one to allow manual firmware
install after full initialization, and another to disable the TCB
cache (T6+). The latter works as a tunable only.

Note that debug_flags are for debugging only and should not be set
normally.

MFC after: 1 week
Sponsored by: Chelsio Communications


# e3d00a3e 30-Aug-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Zero out the memory allocated for the debug dump.
cudbg_collect seems to expect it this way.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 1ba8c29c 28-Aug-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Do not access the mailbox without appropriate locks while
creating hardware VIs.

This fixes a bad race on systems with hw.cxgbe.num_vis > 1.

Reported by: olivier@
MFC after: 1 week
Sponsored by: Chelsio Communications


# 7023d9d4 28-Aug-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Maintain one ifmedia per physical port instead of one per
Virtual Interface (VI). All autonomous VIs that share a port share the
same media.

MFC after: 1 week
Sponsored by: Chelsio Communications


# 573443c5 28-Aug-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): vi_mac_funcs should include the base Ethernet function. It is
already used in the driver as if it does.

MFC after: 3 days
Sponsored by: Chelsio Communications


# f6d9d14b 27-Aug-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Verify that the driver accesses the firmware mailbox in a
thread-safe manner.

MFC after: 3 days


# 5d973bad 12-Aug-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Save the last reported link parameters and compare them with
the current state to determine whether to generate a link-state change
notification. This fixes a bug introduced in r321063 that caused the
driver to sometimes skip these notifications.

Reported by: Jason Eggleston @ LLNW
MFC after: 3 days
Sponsored by: Chelsio Communications


# 019c1a01 04-Aug-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Allow the TOE timer tunables to be set with microsecond
precision. These timers are already displayed in microseconds in the
sysctl MIB. Add variables to track these tunables while here.

MFC after: 3 days
Sponsored by: Chelsio Communications


# 6320b0f8 03-Aug-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Always use the first and not the last virtual interface
associated with a port in begin_synchronized_op.

MFC after: 3 days
Sponsored by: Chelsio Communications


# f856f099 03-Aug-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Initial import of the "collect" component of Chelsio unified
debug (cudbg) code, hooked up to the main driver via an ioctl.

The ioctl can be used to collect the chip's internal state in a
compressed dump file. These dumps can be decoded with the "view"
component of cudbg.

Obtained from: Chelsio Communications
MFC after: 2 months
Sponsored by: Chelsio Communications


# c45b1868 26-Jul-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Some updates to the common code.

- Updated register ranges.
- Helper routines for access to TP registers.
- Updated routine to read flash parameters.

Obtained from: Chelsio Communications
MFC after: 2 weeks
Sponsored by: Chelsio Communications


# dcbe7056 24-Jul-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Display some more TOE parameters related to retransmission
and keepalive in the sysctl MIB. Provide tunables to change some of
these parameters. These are supposed to be setup by the firmware so
these tunables are for experimentation only.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# 98d227f3 23-Jul-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Install the firmware bundled with the driver to the card if it
doesn't seem to have one. This lets the driver recover automatically
from incomplete firmware upgrades (panic, reboot, power loss, etc. in
the middle of an upgrade).

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# 1e5f9430 17-Jul-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): New ioctls to flash bootrom and boot config to the card.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# 01285747 16-Jul-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Various link/media related improvements.

- Deal with changes to port_type, and not just port_mod when a
transceiver is changed. This fixes hot swapping of transceivers of
different types (QSFP+ or QSA or QSFP28 in a QSFP28 port, SFP+ or
SFP28 in a SFP28 port, etc.).

- Always refresh media information for ifconfig if the port is down.
The firmware does not generate tranceiver-change interrupts unless at
least one VI is enabled on the physical port. Before this change
ifconfig diplayed potentially stale information for ports that were
administratively down.

- Always recalculate and reapply L1 config on a transceiver change.

- Display PAUSE settings in ifconfig. The driver sysctls for this
continue to work as well.

MFC after: 2 weeks
Sponsored by: Chelsio Communications


# 720e7bb6 16-May-2017 John Baldwin <jhb@FreeBSD.org>

Add support for child devices that aren't ports.

Invoke any identify routines of child drivers during attach before attaching
children, and delete any remaining devices after deleting ports.

MFC after: 1 month
Sponsored by: Chelsio Communications


# 1404daa7 09-May-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Do not assume that if_qflush is always followed by inteface-down.

MFC after: 3 days
Sponsored by: Chelsio Communications


# e006d2a6 09-May-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fixes related to the knob that controls link autonegotiation.

- Do not leak the adapter lock in sysctl_autoneg.
- Accept only 0 or 1 as valid settings for autonegotiation.
- A fixed speed must be requested by the driver when autonegotiation is
disabled otherwise the firmware will reject the l1cfg command. Use
the top speed supported by the port for now.

MFC after: 3 days
Sponsored by: Chelsio Communications


# 63febe64 04-May-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Update the list of PCIe devices claimed by the driver. At
this point any board with a T6 should just work.

Obtained from: Chelsio Communications
MFC after: 1 week
Sponsored by: Chelsio Communications


# 2204b427 02-May-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Support routines for Tx traffic scheduling.

- Create a new file, t4_sched.c, and move all of the code related to
traffic management from t4_main.c and t4_sge.c to this file.
- Track both Channel Rate Limiter (ch_rl) and Class Rate Limiter (cl_rl)
parameters in the PF driver.
- Initialize all the cl_rl limiters with somewhat arbitrary default
rates and provide routines to update them on the fly.
- Provide routines to reserve and release traffic classes.

MFC after: 1 month
Sponsored by: Chelsio Communications


# 0a600b63 13-Apr-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe: Query some more RDMA related parameters from the firmware.

MFC after: 3 days
Sponsored by: Chelsio Communications


# 3128b167 04-Apr-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Program the global RSS key once instead of once per ifnet.

MFC after: 3 days
Sponsored by: Chelsio Communications


# 73832aae 13-Mar-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix an always-true assertion (reported by PVS-Studio).

sys/dev/cxgbe/t4_main.c: PVS-Studio: Expression is Always True (CWE-571) (3)

PR: 217746
Submitted by: Svyatoslav (razmyslov at viva64 com)
MFC after: 1 week


# 987258d0 05-Feb-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Allow tunables that control the number of queues to be set to
'-n' to tell the driver to create _up to_ 'n' queues if enough cores are
available. For example, setting hw.cxgbe.nrxq10g="-32" will result in
16 queues if the system has 16 cores, 32 if it has 32.

There is no change in the default number of queues of any type.

MFC after: 1 week
Sponsored by: Chelsio Communications


# e8257dbe 09-Jan-2017 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Attach to the 2x25 debug card. This is for internal use only.

MFC after: 3 days


# 358bca3b 30-Dec-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Updates to link configuration.

- Update struct link_settings and associated shared code.

- Add tunables to control FEC and autonegotiation. All ports inherit
these values as their initial settings.
hw.cxgbe.fec
hw.cxgbe.autoneg

- Add per-port sysctls to control FEC and autonegotiation. These can be
modified at any time.
dev.<port>.<n>.fec
dev.<port>.<n>.autoneg

MFC after: 3 days
Sponsored by: Chelsio Communications


# ca276276 14-Dec-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix the tid range shown for T6 cards in misc.tids.

MFC after: 3 days


# b8c1ffef 08-Dec-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): netmap does not set IFCAP_NETMAP in an ifnet's if_capabilities
any more (since r307394). Do it in the driver instead.

MFC after: 1 week


# 3cbaf64f 05-Dec-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Update firmwares from version 1.16.12.0 to 1.16.22.0.

Obtained from: Chelsio Communications
MFC after: 3 days
Sponsored by: Chelsio Communications


# a10443e8 29-Nov-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Include firmware for T6 cards in the driver. Update all
firmwares to 1.16.12.0.

Obtained from: Chelsio Communications
MFC after: 3 days
Sponsored by: Chelsio Communications


# b806e571 17-Oct-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Adjust whitespace to line up the column titles in cim_qcfg
with the values displayed.


# 721f5406 13-Oct-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Allow the interface MTU to be set as high as the actual
hardware limit.

Submitted by: jpaetzel@
Differential Revision: https://reviews.freebsd.org/D8237


# 35b5ef91 07-Oct-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add an ioctl to copy a firmware config file to the card's flash.


# d4d953bf 06-Oct-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix whitespace in the pm_stats display.


# 4e6b9efc 29-Sep-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Claim the T6 -DBG card.


# 788f3c06 24-Sep-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Use the port's top speed to figure out whether it is "high
speed" or not (for the purpose of calculating the number of queues etc.)
This does the right thing for 25Gbps and 100Gbps ports.


# d44268d1 24-Sep-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Support SIOGIFXMEDIA so that ifconfig displays correct media
for 25Gbps and 100Gbps ports. This should have been part of r305713,
which is when the driver first started reporting extended media types.


# de580131 22-Sep-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix the output of the "tids" sysctl on T6.


# d4759c89 22-Sep-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Catch up with the different layout of WHOAMI in T6.

Note that the code moved below t4_prep_adapter() as part of this change
because now it needs a working chip_id().


# 6a0cae68 20-Sep-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Show wcwr_stats for T6 cards.


# b0c554c3 17-Sep-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe/t4_tom: The SMAC entry for a VI is at a different location in the T6.

Sponsored by: Chelsio Communications


# e6b81479 15-Sep-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Attach to cards with the Terminator 6 ASIC. T6 cards will
come up as 't6nex' nexus devices with 'cc' ports hanging off them.

The T6 firmware and configuration files will be added as soon as they
are released. For now the driver will try to work with whatever
firmware and configuration is on the card's flash.

Sponsored by: Chelsio Communications


# 4cf3aa13 11-Sep-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Catch up with the rename of tlscaps -> cryptocaps. TLS is one
of the capabilities of the crypto engine in T6.

Sponsored by: Chelsio Communications


# 9113e53d 11-Sep-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add support for additional port types and link speeds.

Sponsored by: Chelsio Communications.


# 769ef07a 11-Sep-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Rename the debug_flags driver tunable/sysctl to dflags.
Tunables that end with _flags are special.

Sponsored by: Chelsio Communications


# 82c1d6b7 11-Sep-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Deal with the slightly different SGE_STAT_CFG in T6.

Sponsored by: Chelsio Communications


# 3aea3293 09-Sep-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Avoid a NULL dereference in the clearstats ioctl handler.
Port softc's are not initialized when the adapter is in recovery mode.


# 6af45170 07-Sep-2016 John Baldwin <jhb@FreeBSD.org>

Chelsio T4/T5 VF driver.

The cxgbev/cxlv driver supports Virtual Function devices for Chelsio
T4 and T4 adapters. The VF devices share most of their code with the
existing PF4 driver (cxgbe/cxl) and as such the VF device driver
currently depends on the PF4 driver.

Similar to the cxgbe/cxl drivers, the VF driver includes a t4vf/t5vf
PCI device driver that attaches to the VF device. It then creates
child cxgbev/cxlv devices representing ports assigned to the VF.
By default, the PF driver assigns a single port to each VF.

t4vf_hw.c contains VF-specific routines from the shared code used to
fetch VF-specific parameters from the firmware.

t4_vf.c contains the VF-specific PCI device driver and includes its
own attach routine.

VF devices are required to use a different firmware request when
transmitting packets (which in turn requires a different CPL message
to encapsulate messages). This alternate firmware request does not
permit chaining multiple packets in a single message, so each packet
results in a firmware request. In addition, the different CPL message
requires more detailed information when enabling hardware checksums,
so parse_pkt() on VF devices must examine L2 and L3 headers for all
packets (not just TSO packets) for VF devices. Finally, L2 checksums
on non-UDP/non-TCP packets do not work reliably (the firmware trashes
the IPv4 fragment field), so IPv4 checksums for such packets are
calculated in software.

Most of the other changes in the non-VF-specific code are to expose
various variables and functions private to the PF driver so that they
can be used by the VF driver.

Note that a limited subset of cxgbetool functions are supported on VF
devices including register dumps, scheduler classes, and clearing of
statistics. In addition, TOE is not supported on VF devices, only for
the PF interfaces.

Reviewed by: np
MFC after: 2 months
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D7599


# 7cba15b1 01-Sep-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe/cxgbei: Retire all DDP related code from cxgbei and switch to
routines available in t4_tom to manage the iSCSI DDP page pod region.

This adds the ability to use multiple DDP page sizes to the iSCSI
driver, among other improvements.

Sponsored by: Chelsio Communications


# e25621e5 26-Aug-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Provide more details about the card in the sysctl MIB.

dev.t5nex.0.%desc: Chelsio T580-CR
dev.t5nex.0.hw_revision: 1
dev.t5nex.0.sn: PT13140042
dev.t5nex.0.pn: 110117150A0
dev.t5nex.0.ec: 0000000000000000
dev.t5nex.0.na: 0007432AF490
dev.t5nex.0.vpd_version: 3
dev.t5nex.0.scfg_version: 53255
dev.t5nex.0.bs_version: 1.1.0.0
dev.t5nex.0.er_version: 1.0.0.68
dev.t5nex.0.tp_version: 0.1.4.9
dev.t5nex.0.firmware_version: 1.16.2.0

Sponsored by: Chelsio Communications


# bd6ff080 19-Aug-2016 John Baldwin <jhb@FreeBSD.org>

Reorder sysctls so that nodes shared with the VF driver are added first.

This permits a single early return for VF devices in the routines that
add sysctl nodes.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D7512


# 8b2246b3 15-Aug-2016 John Baldwin <jhb@FreeBSD.org>

Add support for register dumps on VF devices.

- Add handling of VF register sets to t4_get_regs_len() and t4_get_regs().
- While here, use t4_get_regs_len() in the ioctl handler for regdump
instead of inlining it.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D7484


# 59c1e950 15-Aug-2016 John Baldwin <jhb@FreeBSD.org>

Make SGE parameter handling more VF-friendly.

Add fields to hold the SGE control register and free list buffer sizes to
the sge_params structure. Populate these new fields in
t4_init_sge_params() for PF devices and change t4_read_chip_settings() to
pull these values out of the params structure instead of reading
registers directly. This will permit t4_read_chip_settings() to be reused
for VF devices which cannot read SGE registers directly.

While here, move the call to t4_init_sge_params() to
get_params__post_init(). The VF driver will populate the SGE parameters
structure via a different method before calling t4_read_chip_settings().

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D7476


# f454e7eb 04-Aug-2016 John Baldwin <jhb@FreeBSD.org>

Add __printflike() to bus_describe_intr() to enable -Wformat checks.

Fix a few places that were passing a raw string as the format to use
a "%s" format string instead.

MFC after: 2 months


# 847bfa8e 03-Aug-2016 John Baldwin <jhb@FreeBSD.org>

Use the port device name for the iov device for Chelsio T4/T5 cards.

Chelsio T4/T5 adapters are multifunction cards. The main driver uses
physical function 4 (PF4). However, VF devices for SR-IOV are only
supported on physical functions 0 through 3, where PF0 creates VFs tied
to port 0, etc. The t4iov/t5iov driver was previously added to
create VF devices for ports that are present on each adapter. This
change uses the recently added pci_iov_attach_name() function to
name the character device in /dev/iov after the associated port on
the card (e.g. /dev/iov/cxl0 is used to create VFs that share the
cxl0 port). With this in place, mark the t4iov/t5iov devices quiet
to prevent them from cluttering dmesg.

Reviewed by: rstone
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D7402


# 315048f2 01-Aug-2016 John Baldwin <jhb@FreeBSD.org>

Store the offset of the KDOORBELL and GTS registers in the softc.

VF devices use a different register layout than PF devices. Storing
the offset in a value in the softc allows code to be shared between the
PF and VF drivers.

Reviewed by: np
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D7389


# 8dd07eab 29-Jul-2016 John Baldwin <jhb@FreeBSD.org>

Various fixes to the t4/5nex character device.

- Remove null open/close methods.
- Don't set d_flags to 0 explicitly.
- Remove t5_cdevsw as the .d_name member isn't really used and doesn't
warrant a separate cdevsw just for the name.
- Use ENOTTY as the error value for an unknown ioctl request.
- Use make_dev_s() to close race with setting si_drv1.

Sponsored by: Chelsio Communications


# 07159830 27-Jul-2016 John Baldwin <jhb@FreeBSD.org>

Add support for zero-copy aio_write() on TOE sockets.

AIO write requests for a TOE socket on a Chelsio T4+ adapter can now
DMA directly from the user-supplied buffer. This is implemented by
wiring the pages backing the user-supplied buffer and queueing special
mbufs backed by raw VM pages to the socket buffer. The TOE code
recognizes these special mbufs and builds a sglist from the VM page
array associated with the mbuf when queueing a work request to the TOE.

Because these mbufs do not have an associated virtual address, m_data
is not valid. Thus, the AIO handler does not invoke sosend() directly
for these mbufs but instead inlines portions of sosend_generic() and
tcp_usr_send().

An aiotx_buffer structure is used to describe the user buffer (e.g.
it holds the array of VM pages and a reference to the AIO job). The
special mbufs reference this structure via m_ext. Note that a single
job might be split across multiple mbufs (e.g. if it is larger than
the socket buffer size). The 'ext_arg2' member of each mbuf gives an
offset relative to the backing aiotx_buffer. The AIO job associated
with an aiotx_buffer structure is completed when the last reference to
the structure is released.

Zero-copy aio_write()'s for connections associated with a given
adapter can be enabled/disabled at runtime via the
'dev.t[45]nex.N.toe.tx_zcopy' sysctl.

MFC after: 1 month
Relnotes: yes
Sponsored by: Chelsio Communications


# 17146cd5 26-Jul-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Initialize the adapter queues (fwq and mgmtq) instead of
returning EAGAIN if they aren't available when the user tries to program
a filter. Do this after validating the filter so that the driver
doesn't bring up the queues if it doesn't have to.


# f91fca5b 22-Jul-2016 John Baldwin <jhb@FreeBSD.org>

Add a driver to create VF devices on Chelsio T4/T5 NICs.

Chelsio NICs are a bit unique compared to some other NICs in that they
expose different functionality on different physical functions. In
particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex').
However, PF4 is not able to create VF devices. Instead, VFs are only
supported by physical functions 0 through 3. This commit adds 't4iov'
and 't5iov' drivers that attach to PF0-3.

One extra wrinkle is that the iov devices cannot enable SR-IOV until the
firwmare has been initialized by the main PF4 driver. To handle this
case, a new t4_if kobj interface has been added to permit cross-calls
between the PF drivers. The PF4 driver notifies sibling drivers when it
is fully attached. It also requests sibling drivers to detach before it
detaches. Sibling drivers query the PF4 driver during their attach
routine to see if it is attached. If not, the sibling drivers defer
their attach actions until the PF4 driver informs them it is attached.

VF devices are associated with a single port on the NIC. VF devices
created from PF0 are associated with the first port on the NIC, VFs
from PF1 are associated with the second port, etc. VF devices can
only be created from a PF device that has an associated port. Thus,
on a 2-port card, VFs are only supported on PF0 and PF1.

Reviewed by: np (earlier versions)
MFC after: 1 month
Sponsored by: Chelsio Communications


# a33b0467 08-Jul-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add sysctl to display the RSS indirection table size for an
interface.

dev.cxl.<n>.rss_size
dev.vcxl.<n>.rss_size

MFC after: 3 days


# 671bf2b8 04-Jul-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Changes to the CPL-handler registration mechanism and code
related to "shared" CPLs.

a) Combine t4_set_tcb_field and t4_set_tcb_field_rpl into a single
function. Allow callers to direct the response to any iq. Tidy up
set_ulp_mode_iscsi while there to use names from t4_tcb.h instead of
magic constants.

b) Remove all CPL handler tables from struct adapter. This reduces its
size by around 2KB. All handlers are now registered at MOD_LOAD instead
of attach or some kind of initialization/activation. The registration
functions do not need an adapter parameter any more.

c) Add per-iq handlers to deal with CPLs whose destination cannot be
determined solely from the opcode. There are 2 such CPLs in use right
now: SET_TCB_RPL and L2T_WRITE_RPL. The base driver continues to send
filter and L2T_WRITEs over the mgmtq and solicits the reply on fwq.
t4_tom (including the DDP code) now uses the port's ctrlq to send
L2T_WRITEs and SET_TCB_FIELDs and solicits the reply on an ofld_rxq.
fwq and ofld_rxq have different handlers that know what kind of tid to
expect in the reply. Update t4_write_l2e and callers to to support any
wrq/iq combination.

Approved by: re@ (kib@)
Sponsored by: Chelsio Communications


# 5e03372b 29-Jun-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Do not bring up an interface when IFCAP_TOE is enabled on it.
The interface's queues are functional after VI_INIT_DONE (which is short
of interface-up) and that's all that's needed for t4_tom to communicate
with the chip.

Approved by: re@ (gjb@)
Sponsored by: Chelsio Communications


# 62291463 22-Jun-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Merge netmap support from the ncxgbe/ncxl interfaces to the
vcxgbe/vcxl interfaces and retire the 'n' interfaces. The main
cxgbe/cxl interfaces and tunables related to them are not affected by
any of this and will continue to operate as usual.

The driver used to create an additional 'n' interface for every
cxgbe/cxl interface if "device netmap" was in the kernel. The 'n'
interface shared the wire with the main interface but was otherwise
autonomous (with its own MAC address, etc.). It did not have normal
tx/rx but had a specialized netmap-only data path. r291665 added
another set of virtual interfaces (the 'v' interfaces) to the driver.
These had normal tx/rx but no netmap support.

This revision consolidates the features of both the interfaces into the
'v' interface which now has a normal data path, TOE support, and native
netmap support. The 'v' interfaces need to be created explicitly with
the hw.cxgbe.num_vis tunable. This means "device netmap" will not
result in the automatic creation of any virtual interfaces.

The following tunables can be used to override the default number of
queues allocated for each 'v' interface. nofld* = 0 will disable TOE on
the virtual interface and nnm* = 0 to will disable native netmap
support.

# number of normal NIC queues
hw.cxgbe.ntxq_vi
hw.cxgbe.nrxq_vi

# number of TOE queues
hw.cxgbe.nofldtxq_vi
hw.cxgbe.nofldrxq_vi

# number of netmap queues
hw.cxgbe.nnmtxq_vi
hw.cxgbe.nnmrxq_vi

hw.cxgbe.nnm{t,r}xq{10,1}g tunables have been removed.

--- tl;dr version ---
The workflow for netmap on cxgbe starting with FreeBSD 11 is:
1) "device netmap" in the kernel config.
2) "hw.cxgbe.num_vis=2" in loader.conf. num_vis > 2 is ok too, you'll
end up with multiple autonomous netmap-capable interfaces for every
port.
3) "dmesg | grep vcxl | grep netmap" to verify that the interface has
netmap queues.
4) Use any of the 'v' interfaces for netmap. pkt-gen -i vcxl<n>... .
One major improvement is that the netmap interface has a normal data
path as expected.
5) Just ignore the cxl interfaces if you want to use netmap only. No
need to bring them up. The vcxl interfaces are completely independent
and everything should just work.
---------------------

Approved by: re@ (gjb@)
Relnotes: Yes
Sponsored by: Chelsio Communications


# 468646f7 07-Jun-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): A couple of fixes to set_sched_queue.

- Validate the scheduling class against the actual limit (which is chip
specific) instead of a magic number.

- Return an error if an attempt is made to manipulate the tx queues of a
VI that hasn't been initialized.

Sponsored by: Chelsio Communications


# 0bc01d66 07-Jun-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Provide information about traffic classes in the sysctl mib.

Sponsored by: Chelsio Communications


# 46464b95 06-Jun-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Track the state of the hardware traffic schedulers in the
driver. This works as long as everyone uses set_sched_class_params
to program them.

Sponsored by: Chelsio Communications


# cfbe2911 06-Jun-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Break up set_sched_class. Validate the channel number and
min/max rates against their actual limits (which are chip and port
specific) instead of hardcoded constants.

Sponsored by: Chelsio Communications


# dc964385 06-May-2016 John Baldwin <jhb@FreeBSD.org>

Use DDP to implement zerocopy TCP receive with aio_read().

Chelsio's TCP offload engine supports direct DMA of received TCP payload
into wired user buffers. This feature is known as Direct-Data Placement.
However, to scale well the adapter needs to prepare buffers for DDP
before data arrives. aio_read() is more amenable to this requirement than
read() as applications often call read() only after data is available in
the socket buffer.

When DDP is enabled, TOE sockets use the recently added pru_aio_queue
protocol hook to claim aio_read(2) requests instead of letting them use
the default AIO socket logic. The DDP feature supports scheduling DMA
to two buffers at a time so that the second buffer is ready for use
after the first buffer is filled. The aio/DDP code optimizes the case
of an application ping-ponging between two buffers (similar to the
zero-copy bpf(4) code) by keeping the two most recently used AIO buffers
wired. If a buffer is reused, the aio/DDP code is able to reuse the
vm_page_t array as well as page pod mappings (a kind of MMU mapping the
Chelsio NIC uses to describe user buffers). The generation of the
vmspace of the calling process is used in conjunction with the user
buffer's address and length to determine if a user buffer matches a
previously used buffer. If an application queues a buffer for AIO that
does not match a previously used buffer then the least recently used
buffer is unwired before the new buffer is wired. This ensures that no
more than two user buffers per socket are ever wired.

Note that this feature is best suited to applications sending a steady
stream of data vs short bursts of traffic.

Discussed with: np
Relnotes: yes
Sponsored by: Chelsio Communications


# 453130d9 02-May-2016 Pedro F. Giffuni <pfg@FreeBSD.org>

sys/dev: minor spelling fixes.

Most affect comments, very few have user-visible effects.


# 74b8d63d 10-Apr-2016 Pedro F. Giffuni <pfg@FreeBSD.org>

Cleanup unnecessary semicolons from the kernel.

Found with devel/coccinelle.


# 307734b6 10-Apr-2016 John Baldwin <jhb@FreeBSD.org>

Add a 'show t4 devlog <nexus>' DDB command.

This command displays the adapter's firmware device log similar to the
dev.<nexus>.misc.devlog sysctl.

Sponsored by: Chelsio Communications


# 113f2316 09-Apr-2016 John Baldwin <jhb@FreeBSD.org>

Add a 'show t4 tcb <nexus> <tid>' command to dump a TCB from DDB.

This allows the contents of a TCB to be extracted from a T4/T5 card in
DDB after a panic.


# 784a631d 16-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Tidy up PAUSE frame accounting.

Figure out if the chip is counting PAUSE frames in the "normal" stats
and take them out if it is. This fixes a bug in the tx stats because
the default hardware behavior is different for Tx and Rx but the driver
was treating both the same way. The result was that OPACKETS, OBYTES,
and OMCASTS were under-reported (if tx_pause > 0) before this change.

Note that the mac_stats sysctl still gives you the raw value of these
statistics straight from the device registers.


# fc2aa1fc 16-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Enable additional capabilities in the default configuration
files. All features with FreeBSD drivers of some kind are now in the
default configuration.


# 78aa2c99 16-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Remove a couple of pointless assignments in sysctl_meminfo.
Do not display range if start = stop (this is a workaround for some
unused regions).


# 353cae49 12-Mar-2016 Dimitry Andric <dim@FreeBSD.org>

Fix the following gcc warnings on sparc64, when TCP_OFFLOAD is not
defined:

sys/dev/cxgbe/t4_main.c:7474: warning: 'sysctl_tp_tick' defined but not used
sys/dev/cxgbe/t4_main.c:7505: warning: 'sysctl_tp_dack_timer' defined but not used
sys/dev/cxgbe/t4_main.c:7519: warning: 'sysctl_tp_timer' defined but not used

This just adds a bunch of #ifdef TCP_OFFLOAD in the right places.

Reviewed by: np
Differential Revision: https://reviews.freebsd.org/D5620


# 537e5f9a 11-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix typo in previous commit.


# 0f2f53ef 11-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Catch up with the latest list of card capabilities as reported
by the firmware.


# 959824da 11-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): sysctls to display the TOE's TCP timers.

cask:~# sysctl -d dev.t5nex.0.toe
dev.t5nex.0.toe.finwait2_timer: FINWAIT2 timer (us)
dev.t5nex.0.toe.initial_srtt: Initial SRTT (us)
dev.t5nex.0.toe.keepalive_intvl: Keepidle interval (us)
dev.t5nex.0.toe.keepalive_idle: Keepidle idle timer (us)
dev.t5nex.0.toe.persist_max: Persist timer max (us)
dev.t5nex.0.toe.persist_min: Persist timer min (us)
dev.t5nex.0.toe.rexmt_max: Retransmit max (us)
dev.t5nex.0.toe.rexmt_min: Retransmit min (us)
dev.t5nex.0.toe.dack_timer: DACK timer (us)
dev.t5nex.0.toe.dack_tick: DACK tick (us)
dev.t5nex.0.toe.timestamp_tick: TCP timestamp tick (us)
dev.t5nex.0.toe.timer_tick: TP timer tick (us)
...

cask:~# sysctl dev.t5nex.0.toe
dev.t5nex.0.toe.finwait2_timer: 9765440
dev.t5nex.0.toe.initial_srtt: 244128
dev.t5nex.0.toe.keepalive_intvl: 73240800
dev.t5nex.0.toe.keepalive_idle: 7031116800
dev.t5nex.0.toe.persist_max: 9765440
dev.t5nex.0.toe.persist_min: 976544
dev.t5nex.0.toe.rexmt_max: 9765440
dev.t5nex.0.toe.rexmt_min: 244128
dev.t5nex.0.toe.dack_timer: 19520
dev.t5nex.0.toe.dack_tick: 32.768
dev.t5nex.0.toe.timestamp_tick: 1048.576
dev.t5nex.0.toe.timer_tick: 32.768
...


# 9945ceb8 10-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add sysctls to display the TP microcode version and the
expansion rom version (if there's one).

trantor:~# sysctl dev.t4nex dev.t5nex | grep _version
dev.t4nex.0.firmware_version: 1.15.28.0
dev.t4nex.0.tp_version: 0.1.9.4
dev.t5nex.0.firmware_version: 1.15.28.0
dev.t5nex.0.exprom_version: 1.0.0.68
dev.t5nex.0.tp_version: 0.1.4.9


# 5849e3bb 10-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add a sysctl for the event capture mask of the TP block's
logic analyzer.

dev.t5nex.<n>.misc.tp_la_mask
dev.t4nex.<n>.misc.tp_la_mask


# 98790d33 10-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Improvements to the code that deals with the firmware's log.

- Query the location of the log very early during attach. Refresh the
location later after establishing contact with the firmware.
- Save the log's location as a flat address in devlog_params.
- Use a memory window instead of backdoor access to the EDC/MC to read
the log.


# 30d816a7 10-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Fix bug in r296603. The memory window needs to be
repositioned if the start address isn't in the window already. One
of the bounds check used the end address instead.


# c9122890 09-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add general purpose routines that offer safe access to the
chip's memory windows. Convert existing users of these windows to the
new routines.


# f8083817 09-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Allow the addr/len pair that is being validated in
validate_mem_range to span multiple memory types. Update
validate_mt_off_len to use validate_mem_range.


# 4d131308 08-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Rename regwin_lock to reg_lock. It is used to protect access
to indirect registers only.


# f799998f 08-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Use t4_link_down_rc_str in shared code to decode the reason
the link is down, instead of doing it in OS specific code.


# 05e2c36c 08-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Remove __devinit and SPEED_<foo> as part of catch up with
internal shared code.

Obtained from: Chelsio Communications


# b3500921 08-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Updates to the shared routines that deal with the serial EEPROM,
flash, and VPD.

Obtained from: Chelsio Communications


# 207d9fea 07-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Update the interrupt handlers for hardware errors.

Obtained from: Chelsio Communications


# 700cfba7 07-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Overhaul the shared code that deals with the chip's TP block,
which is responsible for filtering and RSS.

Add the ability to use filters that match on PF/VF (aka "VNIC id") while
here. This is mutually exclusive with filtering on outer VLAN tag with
Q-in-Q.

Sponsored by: Chelsio Communications


# 90e7434a 07-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add a struct sge_params to store per-adapter SGE parameters.
Move the code that reads all the parameters to t4_init_sge_params in the
shared code. Use these per-adapter values instead of globals.

Sponsored by: Chelsio Communications


# 4f8a1fd8 07-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Updated register dumps.

- Get the list of registers to read during a regdump from the shared
code instead of the OS specific code. This follows a similar move
internally. The shared code includes the list for T6.

- Update cxgbetool to be able to decode T5 VF, T6, and T6 VF register
dumps (and catch up with some updates to T4 and T5 register decode).

Obtained from: Chelsio Communications
Sponsored by: Chelsio Communications


# d1205d09 04-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Very basic T6 awareness. This is part of ongoing work to
update to the latest internal shared code.

- Add a chip_params structure to keep track of hardware constants for
all generations of Terminators handled by cxgbe.
- Update t4_hw_pci_read_cfg4 to work with T6.
- Update the hardware debug sysctls (hidden within dev.<tNnex>.<n>.misc.*) to
work with T6. Most of the changes are in the decoders for the CIM
logic analyzer and the MPS TCAM.
- Acquire the regwin lock around indirect register accesses.

Obtained from: Chelsio Communications
Sponsored by: Chelsio Communications


# ecdff8fa 02-Mar-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): First of many changes to reduce diffs with internal shared
code:

- Rename some CamelCase variables.
- s/t4_link_start/t4_link_l1cfg/g
- Pull in t4_get_port_type_description.
- Move t4_wait_op_done to t4_hw.c.
- Flip the order of the RDMA stats.
- Remove unsused function t4_iq_start_stop.
- Move t4_wait_op_done and t4_wait_op_done_val to t4_hw.c

Obtained from: Chelsio Communications


# e8c6ba72 24-Feb-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add a sysctl to retrieve the maximum speed/bandwidth supported by a
port.

dev.cxgbe.<n>.max_speed
dev.cxl.<n>.max_speed

Sponsored by: Chelsio Communications


# 748d4408 12-Feb-2016 Navdeep Parhar <np@FreeBSD.org>

Remove duplicate definition (CPL_TRACE_PKT_T5).


# 5725f0e4 11-Jan-2016 Navdeep Parhar <np@FreeBSD.org>

cxgbe: bind the ithreads that handle NIC rx to the correct CPU if the kernel
is built with option RSS.


# 9eb533d3 25-Dec-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Updates to the base NIC driver and t4_tom to support the iSCSI
offload driver. These changes come from projects/cxl_iscsi.


# a89012b2 05-Dec-2015 Navdeep Parhar <np@FreeBSD.org>

Fix RSS build.

Reported by: arybchik@


# f28b929a 03-Dec-2015 Konstantin Belousov <kib@FreeBSD.org>

Fix build for !TCP_OFFLOAD case.


# fe2ebb76 02-Dec-2015 John Baldwin <jhb@FreeBSD.org>

Add support for configuring additional virtual interfaces (VIs) on a port.

Each virtual interface has its own MAC address, queues, and statistics.
The dedicated netmap interfaces (ncxgbeX / ncxlX) were already implemented
as additional VIs on each port. This change allows additional non-netmap
interfaces to be configured on each port. Additional virtual interfaces
use the naming scheme vcxgbeX or vcxlX.

Additional VIs are enabled by setting the hw.cxgbe.num_vis tunable to a
value greater than 1 before loading the cxgbe(4) or cxl(4) driver.
NB: The first VI on each port is the "main" interface (cxgbeX or cxlX).

T4/T5 NICs provide a limited number of MAC addresses for each physical port.
As a result, a maximum of six VIs can be configured on each port (including
the "main" interface and the netmap interface when netmap is enabled).

One user-visible result is that when netmap is enabled, packets received
or transmitted via the netmap interface are no longer counted in the stats
for the "main" interface, but are not accounted to the netmap interface.

The netmap interfaces now also have a new-bus device and export various
information sysctl nodes via dev.n(cxgbe|cxl).X.

The cxgbetool 'clearstats' command clears the stats for all VIs on the
specified port along with the port's stats. There is currently no way to
clear the stats of an individual VI.

Reviewed by: np
MFC after: 1 month
Sponsored by: Chelsio


# 02da5bb1 05-Nov-2015 John Baldwin <jhb@FreeBSD.org>

Chelsio T5 chips do not properly echo the No Snoop and Relaxed Ordering
attributes when replying to a TLP from a Root Port. As a workaround,
disable No Snoop and Relaxed Ordering in the Root Port of each T5 adapter
during attach so that CPU-initiated requests do not contain these flags.

Note that this affects CPU-initiated requests to all devices under this
root port.

Reviewed by: np
MFC after: 1 week
Sponsored by: Chelsio


# 07d684f7 15-Oct-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): support for the kernel RSS option.

You need PCBGROUP and RSS in the kernel config to use this.

Relnotes: Yes
Sponsored by: Chelsio Communications


# 8faf5701 19-Aug-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Save the flags for the last adapter-wide synchronized
operation that was initiated successfully. (The caller and thread are
already recorded).

MFC after: 1 week


# 3d3169c8 30-Jul-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): initialize debug_flags from the kernel environment.

MFC after: 3 days


# c7dbd802 14-Jul-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Update T4 and T5 firmwares to 1.14.2.0.

Obtained from: Chelsio Communications
MFC after: 3 days


# 847bf383 09-Jul-2015 Luigi Rizzo <luigi@FreeBSD.org>

Sync netmap sources with the version in our private tree.
This commit contains large contributions from Giuseppe Lettieri and
Stefano Garzarella, is partly supported by grants from Verisign and Cisco,
and brings in the following:

- fix zerocopy monitor ports and introduce copying monitor ports
(the latter are lower performance but give access to all traffic
in parallel with the application)

- exclusive open mode, useful to implement solutions that recover
from crashes of the main netmap client (suggested by Patrick Kelsey)

- revised memory allocator in preparation for the 'passthrough mode'
(ptnetmap) recently presented at bsdcan. ptnetmap is described in
S. Garzarella, G. Lettieri, L. Rizzo;
Virtual device passthrough for high speed VM networking,
ACM/IEEE ANCS 2015, Oakland (CA) May 2015
http://info.iet.unipi.it/~luigi/research.html

- fix rx CRC handing on ixl

- add module dependencies for netmap when building drivers as modules

- minor simplifications to device-specific routines (*txsync, *rxsync)

- general code cleanup (remove unused variables, introduce macros
to access rings and remove duplicate code,

Applications do not need to be recompiled, unless of course
they want to use the new features (monitors and exclusive open).

Those willing to try this code on stable/10 can just update the
sys/dev/netmap/*, sys/net/netmap* with the version in HEAD
and apply the small patches to individual device drivers.

MFC after: 1 month
Sponsored by: (partly) Verisign, Cisco


# 0e4cd4a2 15-Jun-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add the ability to dump mailbox commands and replies. It is
enabled/disabled via bit 0 of adapter->debug_flags (which is available
at dev.t5nex.<n>.debug_flags).

MFC after: 1 week


# a89409b8 31-May-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe: no need to display the per-lane GT/s rating of the pcie link.

MFC after: 1 week


# fd90e2ed 22-May-2015 Jung-uk Kim <jkim@FreeBSD.org>

CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten
years for head. However, it is continuously misused as the mpsafe argument
for callout_init(9). Deprecate the flag and clean up callout_init() calls
to make them more consistent.

Differential Revision: https://reviews.freebsd.org/D2613
Reviewed by: jhb
MFC after: 2 weeks


# a5e0fa40 26-Apr-2015 Gleb Smirnoff <glebius@FreeBSD.org>

Don't use ifm_data. It was used only for self checking debug.

Reviewed by: np


# bb481a8e 23-Mar-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Do not call sbuf_trim on an sbuf with a drain function.

MFC after: 1 week


# 1eafc078 14-Mar-2015 Ian Lepore <ian@FreeBSD.org>

Set the SBUF_INCLUDENUL flag in sbuf_new_for_sysctl() so that sysctl
strings returned to userland include the nulterm byte.

Some uses of sbuf_new_for_sysctl() write binary data rather than strings;
clear the SBUF_INCLUDENUL flag after calling sbuf_new_for_sysctl() in
those cases. (Note that the sbuf code still automatically adds a nulterm
byte in sbuf_finish(), but since it's not included in the length it won't
get copied to userland along with the binary data.)

Remove explicit adding of a nulterm byte in a couple places now that it
gets done automatically by the sbuf drain code.

PR: 195668


# b36424bd 14-Mar-2015 Ian Lepore <ian@FreeBSD.org>

Revert r279934, r279938; this is going to be fixed in sbuf instead.

PR: 195668


# 070d2627 13-Mar-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): fix if_media handling for T520-BT cards. 1Gbps and 100Mbps
are valid for this card.

MFC after: 1 week


# 9bc58e3d 12-Mar-2015 Ian Lepore <ian@FreeBSD.org>

Nullterminate strings returned via sysctl.

PR: 195668


# 5d1e01e9 11-Mar-2015 Navdeep Parhar <np@FreeBSD.org>

t4_iscsi_init sets up global chip registers. Make sure it doesn't
give the impression that it's doing something port or ifnet specific.


# d5d9fbba 09-Feb-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): allow the SET_FILTER_MODE ioctl to change the mode when it's
safe to do so.

MFC after: 1 month


# b3d44a68 08-Feb-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): tidy up some of the interaction between the Upper Layer
Drivers (ULDs) and the base if_cxgbe driver.

Track the per-adapter activation of ULDs in a new "active_ulds" field.
This was done pretty arbitrarily before this change -- via TOM_INIT_DONE
in adapter->flags for TOM, and the (1 << MAX_NPORTS) bit in
adapter->offload_map for iWARP.

iWARP and hw-accelerated iSCSI rely on the TOE (supported by the TOM
ULD). The rules are:
a) If the iWARP and/or iSCSI ULDs are available when TOE is enabled then
iWARP and/or iSCSI are enabled too.
b) When the iWARP and iSCSI modules are loaded they go looking for
adapters with TOE enabled and enable themselves on that adapter.
c) You cannot deactivate or unload the TOM module from underneath iWARP
or iSCSI. Any such attempt will fail with EBUSY.

MFC after: 2 weeks


# 319f2900 08-Feb-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): adapter_full_init is always a synchronized operation.

MFC after: 1 week


# d86a5ff9 08-Feb-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): a change to the synchronization rules within the the driver.
This is purely cosmetic because the new rules are already followed.

MFC after: 1 week


# cb6c101a 06-Feb-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): fix a test made while enabling TOE.

MFC after: 1 week


# 4f621933 26-Jan-2015 John Baldwin <jhb@FreeBSD.org>

Fix a couple of panics when detaching from a cxgbe/cxl interface that was
never brought up:
- Allow NULL to be passed to sglist_free().
- Don't try to stop an interface that was never fully initialized.

Reviewed by: np


# 88d7f6bd 15-Jan-2015 Navdeep Parhar <np@FreeBSD.org>

Allow cxgbe(4) to be built on i386. Driver attach will succeed only on a subset
of i386 systems.


# 008015d2 05-Jan-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): fix the description of a strange bunch of counters.

MFC after: 1 week


# 79b93bf6 02-Jan-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe/tom: do not engage the TOE's payload chopper for payload < 2 MSS
or for 10Gbps ports.

MFC after: 2 weeks


# 255155fc 01-Jan-2015 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): remove buf_ring specific restriction on the txq size.

MFC after: 2 months


# 7951040f 31-Dec-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): major tx rework.

a) Front load as much work as possible in if_transmit, before any driver
lock or software queue has to get involved.

b) Replace buf_ring with a brand new mp_ring (multiproducer ring). This
is specifically for the tx multiqueue model where one of the if_transmit
producer threads becomes the consumer and other producers carry on as
usual. mp_ring is implemented as standalone code and it should be
possible to use it in any driver with tx multiqueue. It also has:
- the ability to enqueue/dequeue multiple items. This might become
significant if packet batching is ever implemented.
- an abdication mechanism to allow a thread to give up writing tx
descriptors and have another if_transmit thread take over. A thread
that's writing tx descriptors can end up doing so for an unbounded
time period if a) there are other if_transmit threads continuously
feeding the sofware queue, and b) the chip keeps up with whatever the
thread is throwing at it.
- accurate statistics about interesting events even when the stats come
at the expense of additional branches/conditional code.

The NIC txq lock is uncontested on the fast path at this point. I've
left it there for synchronization with the control events (interface
up/down, modload/unload).

c) Add support for "type 1" coalescing work request in the normal NIC tx
path. This work request is optimized for frames with a single item in
the DMA gather list. These are very common when forwarding packets.
Note that netmap tx in cxgbe already uses these "type 1" work requests.

d) Do not request automatic cidx updates every 32 descriptors. Instead,
request updates via bits in individual work requests (still every 32
descriptors approximately). Also, request an automatic final update
when the queue idles after activity. This means NIC tx reclaim is still
performed lazily but it will catch up quickly as soon as the queue
idles. This seems to be the best middle ground and I'll probably do
something similar for netmap tx as well.

e) Implement a faster tx path for WRQs (used by TOE tx and control
queues, _not_ by the normal NIC tx). Allow work requests to be written
directly to the hardware descriptor ring if room is available. I will
convert t4_tom and iw_cxgbe modules to this faster style gradually.

MFC after: 2 months


# c2529042 01-Dec-2014 Hans Petter Selasky <hselasky@FreeBSD.org>

Start process of removing the use of the deprecated "M_FLOWID" flag
from the FreeBSD network code. The flag is still kept around in the
"sys/mbuf.h" header file, but does no longer have any users. Instead
the "m_pkthdr.rsstype" field in the mbuf structure is now used to
decide the meaning of the "m_pkthdr.flowid" field. To modify the
"m_pkthdr.rsstype" field please use the existing "M_HASHTYPE_XXX"
macros as defined in the "sys/mbuf.h" header file.

This patch introduces new behaviour in the transmit direction.
Previously network drivers checked if "M_FLOWID" was set in "m_flags"
before using the "m_pkthdr.flowid" field. This check has now now been
replaced by checking if "M_HASHTYPE_GET(m)" is different from
"M_HASHTYPE_NONE". In the future more hashtypes will be added, for
example hashtypes for hardware dedicated flows.

"M_HASHTYPE_OPAQUE" indicates that the "m_pkthdr.flowid" value is
valid and has no particular type. This change removes the need for an
"if" statement in TCP transmit code checking for the presence of a
valid flowid value. The "if" statement mentioned above is now a direct
variable assignment which is then later checked by the respective
network drivers like before.

Additional notes:
- The SCTP code changes will be committed as a separate patch.
- Removal of the "M_FLOWID" flag will also be done separately.
- The FreeBSD version has been bumped.

MFC after: 1 month
Sponsored by: Mellanox Technologies


# 729fee33 19-Nov-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): figure out the max payload size and save it for later.

MFC after: 1 week


# 05c4567d 12-Nov-2014 Navdeep Parhar <np@FreeBSD.org>

Fix some bad interaction between cxgbe(4) and lacp lagg(4) that could
leave a port permanently disabled when a copper cable is unplugged and
then plugged right back in.

lacp_linkstate goes looking for the current ifmedia on a link state
change and it could get stale information from cxgbe(4) on a module
unplug followed by replug. The fix is to process module events before
link-state events within the driver, and to always rebuild the ifmedia
list on a module change event (instead of rebuilding it lazily).

Thanks to asomers@ for the problem report and detailed analysis to go
with it.

MFC after: 1 week


# 7bfc9835 11-Nov-2014 John Baldwin <jhb@FreeBSD.org>

Add device ID for the T502-BT (dual-port 1G) adapter.

Reviewed by: np
MFC after: 1 week


# f0188618 21-Oct-2014 Hans Petter Selasky <hselasky@FreeBSD.org>

Fix multiple incorrect SYSCTL arguments in the kernel:

- Wrong integer type was specified.

- Wrong or missing "access" specifier. The "access" specifier
sometimes included the SYSCTL type, which it should not, except for
procedural SYSCTL nodes.

- Logical OR where binary OR was expected.

- Properly assert the "access" argument passed to all SYSCTL macros,
using the CTASSERT macro. This applies to both static- and dynamically
created SYSCTLs.

- Properly assert the the data type for both static and dynamic
SYSCTLs. In the case of static SYSCTLs we only assert that the data
pointed to by the SYSCTL data pointer has the correct size, hence
there is no easy way to assert types in the C language outside a
C-function.

- Rewrote some code which doesn't pass a constant "access" specifier
when creating dynamic SYSCTL nodes, which is now a requirement.

- Updated "EXAMPLES" section in SYSCTL manual page.

MFC after: 3 days
Sponsored by: Mellanox Technologies


# 2d891085 26-Sep-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): implement if_get_counter.


# acc45299 26-Sep-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): explicitly set various if_hw_tso* values.

MFC after: 3 days


# db25c97a 26-Sep-2014 Navdeep Parhar <np@FreeBSD.org>

Make sure the adapter's management queue and the event queue are
available before any uppper layer driver (TOE, iWARP, or iSCSI)
registers with the base cxgbe(4) driver.

Submitted by: Hariprasad at chelsio dot com
Reviewed by: np@


# 1dee8327 23-Sep-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Verify that the addresses in if_multiaddrs really are multicast
addresses. (The chip doesn't really care, it's just that it needs to be
told explicitly if unicast DMACs are checked for "hits" in the hash that
is used after the TCAM entries are all used up).


# 8374717d 12-Sep-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): add support for the SIOCGI2C ioctl.


# 3eb2c201 11-Sep-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): knobs to enable/disable PAUSE frame based flow control.

MFC after: 1 week


# bc22dc70 06-Aug-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Let caller specify whether it's ok to sleep in
t4_sched_config and t4_sched_params.

MFC after: 2 weeks


# 46a64694 04-Aug-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Do not run any sleepable code in the SIOCSIFFLAGS handler when
IFF_PROMISC or IFF_ALLMULTI is being flipped. bpf(4) holds its global
mutex around ifpromisc in at least the bpf_dtor path.

MFC after: 3 days


# 0fe98277 24-Jul-2014 Navdeep Parhar <np@FreeBSD.org>

Some hooks in cxgbe(4) for the offloaded iSCSI driver.

(I'm committing this on behalf of my colleagues in the Storage team
at Chelsio).

Submitted by: Sreenivasa Honnur <shonnur at chelsio dot com>
Sponsored by: Chelsio Communications.


# 82eff304 23-Jul-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Keep track of the clusters that have to be freed by the
custom free routine (rxb_free) in the driver. Fail MOD_UNLOAD with
EBUSY if any such cluster has been handed up to the kernel but hasn't
been freed yet. This prevents a panic later when the cluster finally
needs to be freed but rxb_free is gone from the kernel.

MFC after: 1 week


# bae4e5af 15-Jul-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Display CF facility correctly in the device log.

MFC after: 3 days


# 44eb8936 14-Jul-2014 Navdeep Parhar <np@FreeBSD.org>

Allow multi-byte reads in the private CHELSIO_T4_GET_I2C ioctl. The
firmware allows up to 48B to be read this way but the driver limits
itself to 8B at a time to remain compatible with old cxgbetool
binaries.

MFC after: 1 week


# 298d969c 27-May-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards.
Netmap gets its own hardware-assisted virtual interface and won't take
over or disrupt the "normal" interface in any way. You can use both
simultaneously.

For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface
(note the 'n' prefix) in the hardware to accompany each cxl<N>
interface. These two ifnet's per port share the same wire but really
are separate interfaces in the hardware and software. Each gets its own
L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You
should run netmap on the 'n' interfaces only, that's what they are for.

With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port
of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now.
Single port receive is at 33Mpps but this is very much a work in
progress. I expect it to be closer to 40Mpps once done. In any case
the current effort can already saturate multiple 10G ports of a T5 card
at the smallest legal packet size. T4 gear is totally untested.

trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef
881.952141 main [1621] interface is ncxl0
881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0
881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0
881.962540 main [1804] mapped 334980KB at 0x801dff000
Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus.
10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef)
881.962562 main [1882] Sending 512 packets every 0.000000000 s
881.962563 main [1884] Wait 2 secs for phy reset
884.088516 main [1886] Ready...
884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1
884.088607 sender_body [996] start
884.093246 sender_body [1064] drop copy
885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec)
886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec)
887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec)
888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec)
889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec)
890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec)
891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec)
892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec)
893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec)
894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec)
895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec)
896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec)
...

Relnotes: Yes
Sponsored by: Chelsio Communications.


# 080a4b9b 17-Apr-2014 Maksim Yevmenkin <emax@FreeBSD.org>

use correct (integer) type for the temperature sysctl

Reviewed by: np, scottl
Obtained from: Netflix
MFC after: 3 days


# 8b3f42d5 20-Mar-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Recognize the "spider" configuration where a T5 card's 40G
QSFP port is presented as 4 distinct 10G SFP+ ports to the driver.

MFC after: 2 weeks


# 65bd4d1c 19-Mar-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Use ifi_oqdrops in if_data to count drops in the tx path.


# 475992bd 19-Mar-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): if_iqdrops statistic should include tunnel congestion drops.

MFC after: 1 week


# 38035ed6 18-Mar-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): significant rx rework.

- More flexible cluster size selection, including the ability to fall
back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in
case an allocation of a larger size fails.
- A single get_fl_payload() function that assembles the payload into an
mbuf chain for any kind of freelist. This replaces two variants: one
for freelists with buffer packing enabled and another for those without.
- Buffer packing with any sized cluster. It was limited to 4K clusters
only before this change.
- Enable buffer packing for TOE rx queues as well.
- Statistics and tunables to go with all these changes. The driver's
man page will be updated separately.

MFC after: 5 weeks


# f7a74e06 06-Feb-2014 Scott Long <scottl@FreeBSD.org>

Add a new sysctl, dev.cxgbe.N.rsrv_noflow, and a companion tunable,
hw.cxgbe.rsrv_noflow. When set, queue 0 of the port is reserved for
TX packets without a flowid. The hash value of packets with a flowid
is bumped up by 1. The intent is to provide a private queue for
link-level packets like LACP that is unlikely to overflow or suffer
deep queue latency.

Reviewed by: np
Obtained from: Netflix
MFC after: 3 days


# 454813ff 05-Feb-2014 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Use the port's tx channel to identify it to t4_clr_port_stats.

MFC after: 3 days


# 3af0f449 02-Jan-2014 Adrian Chadd <adrian@FreeBSD.org>

Add an option to enable or disable the small RX packet copying that
is done to improve performance of small frames.

When doing RX packing, the RX copying isn't necessarily required.

Reviewed by: np


# 93e9cae3 13-Dec-2013 Navdeep Parhar <np@FreeBSD.org>

Read card capabilities after firmware initialization, instead of setting
them up as part of firmware initialization (which the driver gets to do
only if it's the master driver).

Read the range of tids available for the ETHOFLD functionality if it's
enabled.

New is_ftid() and is_etid() functions to test whether a tid falls within
the range of filter tids or ETHOFLD tids respectively.

MFC after: 2 weeks


# ac68deae 09-Dec-2013 Adrian Chadd <adrian@FreeBSD.org>

Print out the full PCIe link negotiation during dmesg.

I found this useful when checking whether a NIC is in a PCIE 3.0 8x slot
or not.

Reviewed by: np
Sponsored by: Netflix, inc.


# d419aaa1 09-Dec-2013 Navdeep Parhar <np@FreeBSD.org>

Unstaticize t4_list and t4_uld_list. This works around a clang
annoyance[1] and allows kgdb to find these symbols.

[1] http://lists.freebsd.org/pipermail/freebsd-hackers/2012-November/041166.html

MFC after: 3 days


# 273ef991 08-Dec-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): save a copy of the RSS map for each port for the driver's use.


# 05337b80 03-Dec-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): T4_SET_SCHED_CLASS and T4_SET_SCHED_QUEUE ioctls to program
scheduling classes in the chip and to bind tx queue(s) to a scheduling
class respectively. These can be used for various kinds of tx traffic
throttling (to force selected tx queues to drain at a fixed Kbps rate,
or a % of the port's total bandwidth, or at a fixed pps rate, etc.).

Obtained from: Chelsio


# 245a0bd4 21-Nov-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): update the internal list of device features.

MFC after: 3 days


# 1192eeb8 06-Nov-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Tidy up the display for payload memory statistics (pm_stats).

# sysctl -n dev.t4nex.0.misc.pm_stats
# sysctl -n dev.t5nex.0.misc.pm_stats

MFC after: 1 week


# be2c0121 04-Nov-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Exclude MPS_RPLC_MAP_CTL (0x11114) from the register dump. Turns
out it's a write-only register with strange side effects on read.

Submitted by: gnn
MFC after: 3 days


# 48d05478 14-Oct-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Update T4 and T5 firmwares to 1.9.12.0


# 4cdc1f54 09-Oct-2013 Gleb Smirnoff <glebius@FreeBSD.org>

There are some high performance NICs that count statistics in hardware,
and there are ifnets, that do that via counter(9). Provide a flag that
would skip cache line trashing '+=' operation in ether_input().

Sponsored by: Netflix
Sponsored by: Nginx, Inc.
Reviewed by: melifaro, adrian
Approved by: re (marius)


# 480e603c 29-Aug-2013 Navdeep Parhar <np@FreeBSD.org>

Merge r254386 from user/np/cxl_tuning. Add an INET|INET6 check missing
in said revision.

r254386:
Flush inactive LRO entries periodically.


# 319a31ea 28-Aug-2013 Navdeep Parhar <np@FreeBSD.org>

Change t4_list_lock and t4_uld_list_lock from mutexes to sx'es.

- tom_uninit had to be reworked not to hold the adapter lock (a mutex)
around t4_deactivate_uld, which acquires the uld_list_lock.
- the ifc_match for the interface cloner that creates the tracer ifnet
had to be reworked as the kernel calls ifc_match with the global
if_cloners_mtx held.


# 98005176 28-Aug-2013 Navdeep Parhar <np@FreeBSD.org>

Add hooks in base cxgbe(4) for the iWARP upper-layer driver. Update a
couple of assertions in the TOE driver as well.


# 8a59745f 26-Aug-2013 Navdeep Parhar <np@FreeBSD.org>

Use correct mailbox and PCIe PF number when querying RDMA parameters.


# 2485eeee 20-Aug-2013 Navdeep Parhar <np@FreeBSD.org>

Display P/N information in the description.

Submitted by: gnn
MFC after: 3 days


# 82342de2 02-Aug-2013 Navdeep Parhar <np@FreeBSD.org>

Display temperature sensor data. Shows -1 if sensor not
available on the card.

# sysctl dev.t4nex.0.temperature
# sysctl dev.t5nex.0.temperature


# 6e22f9f3 30-Jul-2013 Navdeep Parhar <np@FreeBSD.org>

Display SGE tunables in the sysctl tree.

dev.t5nex.0.fl_pktshift: payload DMA offset in rx buffer (bytes)
dev.t5nex.0.fl_pad: payload pad boundary (bytes)
dev.t5nex.0.spg_len: status page size (bytes)
dev.t5nex.0.cong_drop: congestion drop setting

Discussed with: scottl


# 23932205 27-Jul-2013 Navdeep Parhar <np@FreeBSD.org>

Display a string instead of a numeric code in the linkdnrc sysctl.

Submitted by: gnn@


# 716c9e1b 26-Jul-2013 Navdeep Parhar <np@FreeBSD.org>

Expand the list of devices claimed by cxgbe(4).


# caf20efc 26-Jul-2013 Navdeep Parhar <np@FreeBSD.org>

Add support for packet-sniffing tracers to cxgbe(4). This works with
all T4 and T5 based cards and is useful for analyzing TSO, LRO, TOE, and
for general purpose monitoring without tapping any cxgbe or cxl ifnet
directly.

Tracers on the T4/T5 chips provide access to Ethernet frames exactly as
they were received from or transmitted on the wire. On transmit, a
tracer will capture a frame after TSO segmentation, hw VLAN tag
insertion, hw L3 & L4 checksum insertion, etc. It will also capture
frames generated by the TCP offload engine (TOE traffic is normally
invisible to the kernel). On receive, a tracer will capture a frame
before hw VLAN extraction, runt filtering, other badness filtering,
before the steering/drop/L2-rewrite filters or the TOE have had a go at
it, and of course before sw LRO in the driver.

There are 4 tracers on a chip. A tracer can trace only in one direction
(tx or rx). For now cxgbetool will set up tracers to capture the first
128B of every transmitted or received frame on a given port. This is a
small subset of what the hardware can do. A pseudo ifnet with the same
name as the nexus driver (t4nex0 or t5nex0) will be created for tracing.
The data delivered to this ifnet is an additional copy made inside the
chip. Normal delivery to cxgbe<n> or cxl<n> will be made as usual.

/* watch cxl0, which is the first port hanging off t5nex0. */
# cxgbetool t5nex0 tracer 0 tx0 (watch what cxl0 is transmitting)
# cxgbetool t5nex0 tracer 1 rx0 (watch what cxl0 is receiving)
# cxgbetool t5nex0 tracer list
# tcpdump -i t5nex0 <== all that cxl0 sees and puts on the wire

If you were doing TSO, a tcpdump on cxl0 may have shown you ~64K
"frames" with no L3/L4 checksum but this will show you the frames that
were actually transmitted.

/* all done */
# cxgbetool t5nex0 tracer 0 disable
# cxgbetool t5nex0 tracer 1 disable
# cxgbetool t5nex0 tracer list
# ifconfig t5nex0 destroy


# 2b66d732 11-Jul-2013 Navdeep Parhar <np@FreeBSD.org>

Attach to the 4x10G T540-CR card.


# 3a760ee7 04-Jul-2013 Navdeep Parhar <np@FreeBSD.org>

- Show the reason why link is down if this information is available.
- Display the temperature and PHY firmware version of the BT PHY.

MFC after: 1 day


# 6eb3180f 04-Jul-2013 Navdeep Parhar <np@FreeBSD.org>

- Make note of interface MTU change if the rx queues exist, and not just
when the interface is up.
- Add a tunable to control the TOE's rx coalesce feature (enabled by
default as it always has been). Consider the interface MTU or the
coalesce size when deciding which cluster zone to use to fill the
offload rx queue's free list. The tunable is:
dev.{t4nex,t5nex}.<N>.toe.rx_coalesce

MFC after: 1 day


# 6300655c 04-Jul-2013 Navdeep Parhar <np@FreeBSD.org>

On-the-fly changes to the interrupt coalescing timer should apply to the
TOE rx queues too.

MFC after: 1 day


# c337fa30 04-Jul-2013 Navdeep Parhar <np@FreeBSD.org>

- Read all TP parameters in one place.
- Read the filter mode, calculate various shifts, and use them
properly during active open (in select_ntuple).

MFC after: 1 day


# f72b68a1 03-Jul-2013 Navdeep Parhar <np@FreeBSD.org>

- Include the T5 firmware with the driver.
- Update the T4 firmware to the latest.
- Minor reorganization and updates to the version macros, etc.

Obtained from: Chelsio
MFC after: 1 day


# 87c7afeb 01-Jul-2013 Navdeep Parhar <np@FreeBSD.org>

Add a sysctl to get the number of filters available.

sysctl dev.t4nex.<N>.nfilters
sysctl dev.t5nex.<N>.nfilters

MFC after: 3 days


# 99428986 27-Jun-2013 Navdeep Parhar <np@FreeBSD.org>

Update T5 register ranges. This is so that regdump skips over registers
with read side-effects.

MFC after: 3 days


# ad13c6af 05-Jun-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Never install a firmware if hw.cxgbe.fw_install is 0.

MFC after: 1 week


# 9050afc0 03-Jun-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Provide accurate hit count for filters on T5 cards. The
location within the TCB and the size have both changed.

MFC after: 1 week


# 9e4ffff1 31-May-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Some more debug sysctls. These work on both T4 and T5 based
cards.

dev.t5nex.0.misc.cim_ma_la: CIM MA logic analyzer
dev.t5nex.0.misc.cim_pif_la: CIM PIF logic analyzer
dev.t5nex.0.misc.mps_tcam: MPS TCAM entries
dev.t5nex.0.misc.tp_la: TP logic analyzer
dev.t5nex.0.misc.ulprx_la: ULPRX logic analyzer

Obtained from: Chelsio
MFC after: 1 week


# 5ada8664 16-May-2013 Konstantin Belousov <kib@FreeBSD.org>

Add dependencies on the firmware, which allows the loading of the cxgb
and cxgbe modules.

Reviewed and approved by: np
MFC after: 1 week


# d607c747 13-May-2013 Navdeep Parhar <np@FreeBSD.org>

Deal correctly with 40G ports that don't have any transceiver plugged
in. Do not claim that they have unknown tranceivers.

MFC after: 3 days


# 959cbee5 03-May-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe: Switch to a better way to install firmware.

MFC after: 1 week


# 249b2994 30-Apr-2013 Navdeep Parhar <np@FreeBSD.org>

Attach to the T580 (2 x 40G) card.

MFC after: 1 week.


# 8cf31b85 29-Apr-2013 Navdeep Parhar <np@FreeBSD.org>

- Provide accurate ifmedia information so that 40G ports/transceivers are
displayed properly in ifconfig, etc.

- Use the same number of tx and rx queues for a 40G port as for a 10G port.

MFC after: 1 week


# c0bc8af9 11-Apr-2013 Navdeep Parhar <np@FreeBSD.org>

Add pciids of the T5 based cards. The ones that I haven't tested with
cxgbe(4) are disabled for now. This will change.

MFC after: 2 weeks


# 77ad3c41 11-Apr-2013 Navdeep Parhar <np@FreeBSD.org>

Cosmetic change (s/wrwc/wcwr/;s/WRWC/WCWR/).

MFC after: 3 days.


# e7fdf38b 11-Apr-2013 Navdeep Parhar <np@FreeBSD.org>

Get rid of a couple of stray \n's.

MFC after: 3 days.


# 53e8e49d 11-Apr-2013 Navdeep Parhar <np@FreeBSD.org>

There is no need for elaborate queries and error checking when trying to
set FW4MSG_ENCAP.

MFC after: 3 days


# 408b98ef 11-Apr-2013 Navdeep Parhar <np@FreeBSD.org>

- Explain clearly why a different firmware is being installed (if/when
it is being installed). Improve other error messages while here.

- Select special FPGA specific configuration profile when appropriate.

MFC after: 3 days


# 13bf4b07 11-Apr-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Ensure that the MOD_LOAD handler runs before either t4nex or
t5nex attach to their devices.

MFC after: 3 days


# d14b0ac1 29-Mar-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This
includes support for the NIC and TOE features of the 40G, 10G, and
1G/100M cards based on the T5.

The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4)
has been updated instead of writing a brand new driver. T5 cards will
show up as cxl (short for cxlgb) ports attached to the t5nex bus driver.

Sponsored by: Chelsio


# d78bd33f 26-Feb-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Consider all the API versions of the interfaces exported by
the firmware (instead of just the main firmware version) when evaluating
firmware compatibility. Document the new "hw.cxgbe.fw_install" knob
being introduced here.

This should fix kern/173584 too. Setting hw.cxgbe.fw_install=2 will
mostly do what was requested in the PR but it's a bit more intelligent
in that it won't reinstall the same firmware repeatedly if the knob is
left set.

PR: kern/173584
MFC after: 5 days


# 0abd31e2 25-Feb-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Ask the card's firmware to pad up tiny CPLs by encapsulating
them in a firmware message if it is able to do so. This works out
better for one of the FIFOs in the chip.

MFC after: 5 days


# c1508f2b 21-Feb-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add sysctls to extract debug information from the chip:

dev.t4nex.X.misc.cim_la logic analyzer dump
dev.t4nex.X.misc.cim_qcfg queue configuration
dev.t4nex.X.misc.cim_ibq_xxx inbound queues
dev.t4nex.X.misc.cim_obq_xxx outbound queues

Obtained from: Chelsio
MFC after: 1 week


# bf3db9eb 08-Feb-2013 Navdeep Parhar <np@FreeBSD.org>

Do not hold locks around hardware context reads.

MFC after: 3 days


# c25f3787 29-Jan-2013 Navdeep Parhar <np@FreeBSD.org>

Provide a statistic to track the number of drops in each of the port's
txq's buf_ring. The aggregate for all the queues of a port is already
provided in ifnet->if_snd.ifq_drops.

MFC after: 3 days.


# 1cdc8899 25-Jan-2013 Navdeep Parhar <np@FreeBSD.org>

Force the 404-BT card (4 x 1G) to use the "uwire" configuration file.

MFC after: 3 days


# 7ca5c863 25-Jan-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe/tom: List IFCAP_TOE6 as supported now that all the required pieces
are in place. You still have to enable it explicitly, after loading the
t4_tom KLD.


# 601fce88 16-Jan-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe: Do a more thorough job in the CLEAR_STATS ioctl.

MFC after: 3 days


# 0a0a697c 14-Jan-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Updates to the hardware L2 table management code.

- Add full support for IPv6 addresses.

- Read the size of the L2 table during attach. Do not assume that PCIe
physical function 4 of the card has all of the table to itself.

- Use FNV instead of Jenkins to hash L3 addresses and drop the private
copy of jhash.h from the driver.

MFC after: 1 week


# b174b658 10-Jan-2013 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add functions to help synchronize "slow" operations (those not
on the fast data path) and use them instead of frobbing the adapter lock
and busy flag directly.

Other changes made while reworking all slow operations:
- Wait for the reply to a filter request (add/delete). This guarantees
that the operation is complete by the time the ioctl returns.
- Tidy up the tid_info structure.
- Do not allow the tx queue size to be set to something that's not a
power of 2.

MFC after: 1 week


# c6719ccd 22-Dec-2012 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): Add support for the T440-LP-CR card. This is the 4x10G low
profile card with a QSFP+ transceiver.

MFC after: 3 days


# 6b946662 19-Oct-2012 Ed Schouten <ed@FreeBSD.org>

Prefer __containerof() over __member2struct().

The former works better with qualifiers, but also properly type checks
the input pointer.


# 8039b7e5 12-Oct-2012 Navdeep Parhar <np@FreeBSD.org>

Temporary fix for kern/172364.

PR: kern/172364
MFC after: 3 days


# 86e02bf2 12-Oct-2012 Navdeep Parhar <np@FreeBSD.org>

Use global knob in the TP_PARA_REG3 register to disable congestion
drops if the user has chosen this behaviour.

MFC after: 3 days


# c2e35e3f 10-Oct-2012 Navdeep Parhar <np@FreeBSD.org>

Add a driver ioctl to clear a port's MAC statistics.

Submitted by: gnn@
MFC after: 3 days


# 8d92e1db 10-Oct-2012 Navdeep Parhar <np@FreeBSD.org>

Add a driver ioctl to read a byte from any device on a port's i2c bus.
This lets userspace read arbitrary information from the SFP+ modules
etc. on this bus.

Reading multiple bytes in the same transaction isn't possible right now.
I'll update the driver once the chip's firmware supports this.

MFC after: 3 days


# 389c8bd5 18-Sep-2012 Gavin Atkinson <gavin@FreeBSD.org>

Align the PCI Express #defines with the style used for the PCI-X
#defines. This also has the advantage that it makes the names more
compact, iand also allows us to correct the non-uniform naming of
the PCIM_LINK_* defines, making them all consistent amongst themselves.

This is a mostly mechanical rename:
s/PCIR_EXPRESS_/PCIER_/g
s/PCIM_EXP_/PCIEM_/g
s/PCIM_LINK_/PCIEM_LINK_/g

When this is MFC'd, #defines will be added for the old names to assist
out-of-tree drivers.

Discussed with: jhb
MFC after: 1 week


# 8a599c08 13-Sep-2012 Navdeep Parhar <np@FreeBSD.org>

Install interrupt handlers early, during attach, for the reason
explained in r239913 by jhb.

MFC after: 1 week


# 57c60f98 13-Sep-2012 Navdeep Parhar <np@FreeBSD.org>

Use native FreeBSD facilities everywhere except the shared code in common/

MFC after: 1 week


# 5f7a6408 16-Aug-2012 Navdeep Parhar <np@FreeBSD.org>

Initialize various DDP parameters in the main cxgbe(4) driver:

- Setup multiple DDP page sizes. When the driver attempts DDP it will
try to combine physically contiguous pages into regions of these sizes.

- Set the indicate size such that the payload carried in the indicate can
be copied in the header mbuf (and the 16K rx buffer can be recycled).

- Set DDP threshold to the max payload that the chip will coalesce and
deliver to the driver (this is ~16K by default, which is also why the
offload rx queue is backed by 16K buffers). If the chip is able to
coalesce up to the max it's allowed to, it's a good sign that the peer
is transmitting in bulk without any TCP PSH.

MFC after: 2 weeks


# 1f1b5a0f 16-Aug-2012 Navdeep Parhar <np@FreeBSD.org>

Add a routine (t4_set_tcb_field) to update arbitrary parts of a hardware
TCB. Filters are programmed by modifying the TCB too (via a different
routine) and the reply to any TCB update is delivered via a
CPL_SET_TCB_RPL. Figure out whether the reply is for a filter-write or
something else and route it appropriately.

MFC after: 2 weeks


# 1b4cc91f 16-Aug-2012 Navdeep Parhar <np@FreeBSD.org>

Allow for a different handler for each type of firmware message.

MFC after: 2 weeks


# 2951cbab 14-Aug-2012 Navdeep Parhar <np@FreeBSD.org>

if_iqdrops should include frames truncated within the chip.

MFC after: 2 weeks


# 9fb8886b 14-Aug-2012 Navdeep Parhar <np@FreeBSD.org>

Convert some fixed parameters to tunables (with reasonable default
values).

- cong_drop specifies what to do on congestion: nothing, backpressure,
or drop.
- fl_pktshift specifies the padding before Ethernet payload.
- fl_pad specifies the boundary upto which to pad Ethernet payload.
- spg_len controls the length of the status page.

MFC after: 2 weeks


# 9a0b948f 03-Jul-2012 Navdeep Parhar <np@FreeBSD.org>

Fix inverted test that resulted in incorrect multicast hw programming.


# 9f1dae79 02-Jul-2012 Navdeep Parhar <np@FreeBSD.org>

Instruct the firmware not to provision resources for TCP offload if the
kernel is being built without TCP_OFFLOAD. But never override
toecaps_allowed if it has been set manually.


# 932b1a5f 29-Jun-2012 Navdeep Parhar <np@FreeBSD.org>

- Assign (don't OR) the CSUM_XXX bits to csum_flags in the rx checksum code.
- Fix TSO/TSO4 mixup.
- Add IFCAP_LINKSTATE to the available/enabled capabilities.


# a1ea9a82 29-Jun-2012 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): support for IPv6 TSO and LRO.

Submitted by: bz (this is a modified version of that patch)


# 9600bf00 29-Jun-2012 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4): support for IPv6 hardware checksumming (rx and tx).


# 2cd9f071 25-Jun-2012 Navdeep Parhar <np@FreeBSD.org>

Allow cxgbe(4) running within a VM to attach to its devices that have been
exported via PCI passthrough.

- Do not check for a specific physical function (PF) before claiming a device.
Different PFs have different device-ids so this check is redundant anyway.

- Obtain the PF# from the WHOAMI register instead of pci_get_function().

- Setup the memory windows using the real BAR0 address, not what the VM says it
is.

Obtained from: Chelsio Communications


# 3c51d154 22-Jun-2012 Navdeep Parhar <np@FreeBSD.org>

Do not allocate extra vectors when adapter is not TOE
capable (or toecaps have been disallowed by the user).

+ one very minor unrelated cleanup in t4_sge.c


# afce448c 22-Jun-2012 Navdeep Parhar <np@FreeBSD.org>

Do not read registers with read side effects while performing a register
dump for cxgbetool.


# 09fe6320 19-Jun-2012 Navdeep Parhar <np@FreeBSD.org>

- Updated TOE support in the kernel.

- Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs.
These are available as t3_tom and t4_tom modules that augment cxgb(4)
and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as
usual with or without these extra features.

- iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the
works and will follow soon.

Build-tested with make universe.

30s overview
============
What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the
capabilities of an interface:
# ifconfig -m | grep TOE

Enable/disable TCP offload on an interface (just like any other ifnet
capability):
# ifconfig cxgbe0 toe
# ifconfig cxgbe0 -toe

Which connections are offloaded? Look for toe4 and/or toe6 in the
output of netstat and sockstat:
# netstat -np tcp | grep toe
# sockstat -46c | grep toe

Reviewed by: bz, gnn
Sponsored by: Chelsio communications.
MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)


# 7a32954c 30-Apr-2012 Navdeep Parhar <np@FreeBSD.org>

Change the default to not use packet counters to generate rx interrupts.
Rely solely on the timer based mechanism.

Update man page to reflect this change.

MFC after: 1 week


# e07f03e8 30-Apr-2012 Navdeep Parhar <np@FreeBSD.org>

Make sure that the firmware version is available in
dev.t4nex.X.firmware_version even if the driver fails to attach
properly. At least it'll be easy to tell what we're dealing with.

MFC after: 1 week


# 62795b70 07-Feb-2012 Navdeep Parhar <np@FreeBSD.org>

Program the MAC exact match table in batches of 7 addresses at
a time when possible. This is more efficient than one at a time.

Submitted by: gnn
MFC after: 3 days


# 17c60e7b 07-Feb-2012 Navdeep Parhar <np@FreeBSD.org>

Acquire the adapter lock before updating fields of the filter structure.

Submitted by: gnn (different version)
MFC after: 3 days


# 65d43cc6 07-Feb-2012 Navdeep Parhar <np@FreeBSD.org>

Remove if_start from cxgb and cxgbe.

Submitted by: jhb
MFC after: 3 days


# bfb08b6b 06-Feb-2012 Navdeep Parhar <np@FreeBSD.org>

cxgbe: reduce diffs with other branches.
Will help future MFCs from HEAD.

MFC after: 3 days


# 733b9277 15-Dec-2011 Navdeep Parhar <np@FreeBSD.org>

Many updates to cxgbe(4)

- Device configuration via plain text config file. Also able to operate
when not attached to the chip as the master driver.

- Generic "work request" queue that serves as the base for both ctrl and
ofld tx queues.

- Generic interrupt handler routine that can process any event on any
kind of ingress queue (via a dispatch table).

- A couple of new driver ioctls. cxgbetool can now install a firmware
to the card ("loadfw" command) and can read the card's memory
("memdump" and "tcb" commands).

- Lots of assorted information within dev.t4nex.X.misc.* This is
primarily for debugging and won't show up in sysctl -a.

- Code to manage the L2 tables on the chip.

- Updates to cxgbe(4) man page to go with the tunables that have changed.

- Updates to the shared code in common/

- Updates to the driver-firmware interface (now at fw 1.4.16.0)

MFC after: 1 month


# 4b7ec270 22-Nov-2011 Marius Strobl <marius@FreeBSD.org>

- There's no need to overwrite the default device method with the default
one. Interestingly, these are actually the default for quite some time
(bus_generic_driver_added(9) since r52045 and bus_generic_print_child(9)
since r52045) but even recently added device drivers do this unnecessarily.
Discussed with: jhb, marcel
- While at it, use DEVMETHOD_END.
Discussed with: jhb
- Also while at it, use __FBSDID.


# 6472ac3d 07-Nov-2011 Ed Schouten <ed@FreeBSD.org>

Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs.

The SYSCTL_NODE macro defines a list that stores all child-elements of
that node. If there's no SYSCTL_DECL macro anywhere else, there's no
reason why it shouldn't be static.


# 59bc8ce0 10-Jun-2011 Navdeep Parhar <np@FreeBSD.org>

- driver ioctl to get SGE context for any given queue.
- sysctls to display the context id, cidx, and pidx of all kinds of queues.

MFC after: 3 days


# 91046633 04-Jun-2011 Navdeep Parhar <np@FreeBSD.org>

Cause backpressure (instead of dropping frames) on congestion.

MFC after: 3 days


# 272cba15 31-May-2011 Navdeep Parhar <np@FreeBSD.org>

Provide hit-count with rest of the information about a filter.

MFC after: 1 week


# 136e410c 31-May-2011 Navdeep Parhar <np@FreeBSD.org>

Firmware device log.

# sysctl dev.t4nex.0.devlog

MFC after: mdf's sysctl+sbuf changes are MFC'd


# 56599263 30-May-2011 Navdeep Parhar <np@FreeBSD.org>

- Specialized ingress queues that take interrupts for other ingress
queues. Try to have a set of these per port when possible, fall back
to sharing a common pool between all ports otherwise.

- One control queue per port (used to be one per hardware channel).

- t4_eth_rx now handles Ethernet rx only.

- sysctls to display pidx/cidx for some queues.

MFC after: 1 week


# 4dba21f1 30-May-2011 Navdeep Parhar <np@FreeBSD.org>

L2 table code. This is enough to get the T4's switch + L2 rewrite
filters working. (All other filters - switch without L2 info rewrite,
steer, and drop - were already fully-functional).

Some contrived examples of "switch" filters with L2 rewriting:

# cxgbetool t4nex0 iport 0 dport 80 action switch vlan +9 eport 3
Intercept all packets received on physical port 0 with TCP port 80 as
destination, insert a vlan tag with VID 9, and send them out of port 3.

# cxgbetool t4nex0 sip 192.168.1.1/32 ivlan 5 action switch \
vlan =9 smac aa:bb:cc:dd:ee:ff eport 0
Intercept all packets (received on any port) with source IP address
192.168.1.1 and VLAN id 5, rewrite the VLAN id to 9, rewrite source mac
to aa:bb:cc:dd:ee:ff, and send it out of port 0.

MFC after: 1 week


# b0775aef 19-May-2011 Navdeep Parhar <np@FreeBSD.org>

Simplify t4_os_find_pci_capability.

MFC after: 3 days


# bc14b14d 18-May-2011 Navdeep Parhar <np@FreeBSD.org>

- Enable per-channel congestion notification.
- Enable PCIe relaxed ordering for all egress queues and rx data buffers.

MFC after: 3 days


# c4343146 16-May-2011 Navdeep Parhar <np@FreeBSD.org>

Add missing header. The test for VLAN_CAPABILITIES later in the file
doesn't make sense without it.

MFC after: 3 days


# 3792a4d2 05-May-2011 Navdeep Parhar <np@FreeBSD.org>

Bump up the number of egress queues that the driver is allowed to use.

MFC after: 3 days


# 8820ce5f 04-May-2011 Navdeep Parhar <np@FreeBSD.org>

T4 packet filtering/steering.

- Enable 5-tuple and every-packet lookup.

- Setup the default filter mode to allow filtering/steering based on IP
protocol, ingress port, inner VLAN ID, IP frag, FCoE, and MPS match
type; all combined together. You can also filter based on MAC index,
Ethernet type, IP TOS/IPv6 Traffic Class, and outer VLAN ID but you'll
have to modify the default filter mode and exclude some of the
match-fields in it.

IPv4 and IPv6 SIP/DIP/SPORT/DPORT are always available in all filter
rules.

- Add driver ioctls to get/set the global filter mode.

- Add driver ioctls to program and delete hardware filters. A couple of
the "switch" actions that rewrite Ethernet and VLAN information and
switch the packet out of another port may not work as the L2 code is not
yet in place. Everything else, including all "drop" and "pass" rules
with RSS or absolute qid, should work.

Obtained from: Chelsio Communications


# 657d9381 19-Apr-2011 Navdeep Parhar <np@FreeBSD.org>

Use Toeplitz hash for RSS.

MFC after: 3 days


# f7dfe243 19-Apr-2011 Navdeep Parhar <np@FreeBSD.org>

- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is
now a suitable base for all kinds of egress queues.

- Add control queues (sge_ctrlq) and allocate one of these per hardware
channel. They can be used to program filters and steer traffic (and
more).

MFC after: 1 week


# 2be67d29 14-Apr-2011 Navdeep Parhar <np@FreeBSD.org>

Fix a couple of bad races that can occur when a cxgbe interface is taken
down. The ingress queue lock was unused and has been removed as part of
these changes.

- An in-flight egress update from the SGE must be handled before the
queue that requested it is destroyed. Wait for the update to arrive.

- Interrupt handlers must stop processing rx events for a queue before
the queue is destroyed. Events that have not yet been processed
should be ignored once the queue disappears.

MFC after: 1 week


# 6b49a4ec 14-Apr-2011 Navdeep Parhar <np@FreeBSD.org>

There is no need to request a tx credit flush if such a request is already
pending.

MFC after: 3 days


# bd0d6201 07-Apr-2011 Navdeep Parhar <np@FreeBSD.org>

Modify read/write ioctls to work with 64 bit registers too.

MFC after: 3 days


# 37ba1354 31-Mar-2011 Navdeep Parhar <np@FreeBSD.org>

Update header and related code for firmware 1.3.8

MFC after: 3 days


# a91fea93 23-Mar-2011 Navdeep Parhar <np@FreeBSD.org>

Do not over-allocate MSI interrupts for the case where each ingress
queue has its own interrupt. If the exact number that we need is not a
power of 2 and we're using MSI, then switch to interrupt multiplexing.

While here, replace the magic numbers with something more readable.

MFC after: 3 days


# d986a01a 09-Mar-2011 Navdeep Parhar <np@FreeBSD.org>

Display holdoff timers and packet counts as a list of numbers.

MFC after: 1 week


# 94586193 07-Mar-2011 Navdeep Parhar <np@FreeBSD.org>

cxgbe shouldn't directly know of the UMA zones where network buffers
come from.

MFC after: 1 week


# 99bb3c53 04-Mar-2011 Navdeep Parhar <np@FreeBSD.org>

Be sure to stay within the bounds of the mod_str array when displaying
the transceiver type.


# 29ca78e1 04-Mar-2011 Navdeep Parhar <np@FreeBSD.org>

Store the ifnet rather than the port_info in each txq and rxq struct.

MFC after: 1 week


# 56c2cdaf 04-Mar-2011 Navdeep Parhar <np@FreeBSD.org>

Upgrade the firmware on the card automatically if a better version is
available. Downgrade only for a major version mismatch.

MFC after: 1 week


# ecb79ca4 04-Mar-2011 Navdeep Parhar <np@FreeBSD.org>

Resume tx immediately in response to an SGE egress update from the hardware.

MFC after: 1 week


# 54e4ee71 18-Feb-2011 Navdeep Parhar <np@FreeBSD.org>

cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters.

MFC after: 3 weeks