MFC r341785

sfxge(4): use n Tx queues instead of n + 2 on EF10 HW

On EF10 HW we can avoid sending packets without checksum offload
or with IP-only checksum offload to dedicated queues. Instead, we
can use option descriptors to change offload policy on any queue
during runtime. Thus, we don't need to create two dedicated queues.

Submitted by: Ivan Malov <Ivan.Malov at oktetlabs.ru>
Sponsored by: Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D18390

MFC r341784

sfxge(4): prepare the number of Tx queues on event queue 0 to become
variable

The number of Tx queues on event queue 0 can depend on the NIC family
type, and this property will be leveraged by future patches.
This patch prepares the code for this change.

Submitted by: Ivan Malov <Ivan.Malov at oktetlabs.ru>
Sponsored by: Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D18389

MFC r341782

sfxge(4): populate per-event queue stats in sysctl

In order to find out why the first event queue and corresponding
interrupt is triggered more frequent, it is useful to know which
events go to each event queue.

Sponsored by: Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D18418

MFC r310713

sfxge(4): add possibility to control event queue performance profile

It is ignored on SFN5xxx/6xxx (aka Siena).

Sponsored by: Solarflare Communications, Inc.

MFC r301724

sfxge(4): handle negative ticks difference correctly

ticks are signed int and if statistics is not updated for a long time
(more than INT_MAX ticks, but less than UINT_MAX) difference becomes
negative and less than hz for a long time.

Other option to repeat is simply load driver (which initializes
timestamps to 0) when ticks are negative.

Sponsored by: Solarflare Communications, Inc.

MFC r301309

sfxge(4): always be ready to receive batched events

When the low-latency firmware variant is running, it is reported as not
being capable of batching RX events, but it can still do so if the
FORCE_EV_MERGING flag is set on an RXQ. Therefore we need to handle
batched RX events even if the capability isn't set.

If this bug is fixed in the firmware such that the capability is set
even when running the low-latency firmware variant, it will almost
always be reported so I don't think we lose much by removing the check.

Submitted by: Mark Spender <mspender at solarflare.com>
Sponsored by: Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D6705

MFC r301122

sfxge(4): set moderation in efx_ev_qcreate

This simplifies setting an initial interrupt moderation value, and
avoids most calls to evx_ev_qmoderate from contexts where MCDI is
not allowed (MCDI is need for an EVQ timer workaround in a later patch).

Submitted by: Andy Moreton <amoreton at solarflare.com>
Reviewed by: gnn
Sponsored by: Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D6673

MFC r300607

sfxge(4): cleanup: update copyright to 2016

Sponsored by: Solarflare Communications, Inc.

MFC r301105

sfxge(4): cope with code duplication on SW events composition

Sponsored by: Solarflare Communications, Inc.

MFC r301075

sfxge(4): avoid code duplication in SW events definition

Sponsored by: Solarflare Communications, Inc.

MFC r298735

sfxge(4): do not use RxQ index as label

Labels are limitted by 32 on EF10. It is not sufficient on powerful hosts.
Since only one RxQ is running over each EvQ, zero label may be used.

Reviewed by: gnn
Sponsored by: Solarflare Communications, Inc.
PR: 208267
Differential Revision: https://reviews.freebsd.org/D6121

MFC: r283514

sfxge: add 7xxx NICs family support

Support 7xxx adapters including firmware-assisted TSO and VLAN tagging:

- Solarflare Flareon Ultra 7000 series 10/40G adapters:
- Solarflare SFN7042Q QSFP+ Server Adapter
- Solarflare SFN7142Q QSFP+ Server Adapter

- Solarflare Flareon Ultra 7000 series 10G adapters:
- Solarflare SFN7022F SFP+ Server Adapter
- Solarflare SFN7122F SFP+ Server Adapter
- Solarflare SFN7322F Precision Time Synchronization Server Adapter

- Solarflare Flareon 7000 series 10G adapters:
- Solarflare SFN7002F SFP+ Server Adapter

Support utilities to configure adapters and update firmware.

The work is done by Solarflare developers
(Andy Moreton, Andrew Lee and many others),
Artem V. Andreev <Artem.Andreev at oktetlabs.ru> and me.

Sponsored by: Solarflare Communications, Inc.

MFC: 279351

sfxge: expect required init_state on data path and in periodic calls

With the patch applied the number of instruction events is 1% less and
number of mispredicted branch events is 5% less under multistream TCP
traffic load close to line rate.

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 279182

sfxge: correct event queue interrupt moderation timer quanta

Submitted by: Andrew Lee <alee at solarflare.com>
Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 279177

sfxge: assert event queue lock in event handlers

It is useful to highlight lock context.

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 278940

sfxge: add driver context member with number of event queues

Mainly to unify with similar member for transmit and receive queues.
It will be used in the future for resources allocation processing.

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 278250

sfxge: make lock names unique

Lock name should include interface name.
Tx queue and event queue lock name should include queue number.

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 278221

sfxge: Add macros to init, destroy, acquire, release and assert locks

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 277893

sfxge: Use SFXGE_MODERATION to initialize event moderation

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 277889

sfxge: Add evq argument to sfxge_tx_qcomplete()

It removes necessity to get evq pointer by its index in soft context.

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 277886

sfxge: Make it possible to build without EVQ statistics

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 277884

sfxge: Change sfxge_ev_qpoll() proto to avoid EVQ pointers array access

It was the only place on data path where sc->evq array is accessed.

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 272328

Make size of Tx and Rx rings configurable

Required size of event queue is calculated now.

Submitted by: Andrew Rybchenko <arybchenko at solarflare.com>
Sponsored by: Solarflare Communications, Inc.

MFC: 272325

cleanup: code style fixes

Remove trailing whitespaces and tabs.
Enclose value in return statements in parentheses.
Use tabs after #define.
Do not skip comparison with 0/NULL in boolean expressions.

Submitted by: Andrew Rybchenko <arybchenko at solarflare.com>
Sponsored by: Solarflare Communications, Inc.

MFC: 263302, 264461, 264772

263302:
fix mbuf leak if it does not fit in software queue

264461:
Commit various fixes for the SolarFlare drivers, in particular
this set of patches fixes support for systems with > 32 cores.

Details include

sfxge: RXQ index (not label) comes from FW in flush done/failed events

Change the second argument name of the efx_rxq_flush_done_ev_t and
efx_rxq_flush_failed_ev_t prototypes to highlight that RXQ index (not label)
comes from FW in flush done and failed events.

sfxge: TXQ index (not label) comes from FW in flush done event

Change the second argument name of the efx_txq_flush_done_ev_t prototype to
highlight that TXQ index (not label) comes from FW in flush done event.

sfxge: use TXQ type as label to support more than 32 TXQs

There are 3 TXQs in event queue 0 and 1 TXQ (with TCP/UDP checksum offload)
in all other event queues.

264772:
Check that port is started when MAC filter is set

The MAC filter set may be called without softc_lock held in the case of
SIOCADDMULTI and SIOCDELMULTI ioctls. The ioctl handler checks IFF_DRV_RUNNING
flag which implies port started, but it is not guaranteed to remain.
softc_lock shared lock can't be held in the case of these ioctls processing,
since it results in failure where kernel complains that non-sleepable
lock is held in sleeping thread.

Both problems are repeatable on LAG with LACP proto bring up.

Submitted by: Andrew Rybchenko <Andrew.Rybchenko at oktetlabs.ru>
Sponsored by: Solarflare Communications, Inc.

MFC: r283514

sfxge: add 7xxx NICs family support

Support 7xxx adapters including firmware-assisted TSO and VLAN tagging:

- Solarflare Flareon Ultra 7000 series 10/40G adapters:
- Solarflare SFN7042Q QSFP+ Server Adapter
- Solarflare SFN7142Q QSFP+ Server Adapter

- Solarflare Flareon Ultra 7000 series 10G adapters:
- Solarflare SFN7022F SFP+ Server Adapter
- Solarflare SFN7122F SFP+ Server Adapter
- Solarflare SFN7322F Precision Time Synchronization Server Adapter

- Solarflare Flareon 7000 series 10G adapters:
- Solarflare SFN7002F SFP+ Server Adapter

Support utilities to configure adapters and update firmware.

The work is done by Solarflare developers
(Andy Moreton, Andrew Lee and many others),
Artem V. Andreev <Artem.Andreev at oktetlabs.ru> and me.

Sponsored by: Solarflare Communications, Inc.

MFC: 279351

sfxge: expect required init_state on data path and in periodic calls

With the patch applied the number of instruction events is 1% less and
number of mispredicted branch events is 5% less under multistream TCP
traffic load close to line rate.

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 279182

sfxge: correct event queue interrupt moderation timer quanta

Submitted by: Andrew Lee <alee at solarflare.com>
Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 279177

sfxge: assert event queue lock in event handlers

It is useful to highlight lock context.

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 278940

sfxge: add driver context member with number of event queues

Mainly to unify with similar member for transmit and receive queues.
It will be used in the future for resources allocation processing.

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 278250

sfxge: make lock names unique

Lock name should include interface name.
Tx queue and event queue lock name should include queue number.

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 278221

sfxge: Add macros to init, destroy, acquire, release and assert locks

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 277893

sfxge: Use SFXGE_MODERATION to initialize event moderation

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 277889

sfxge: Add evq argument to sfxge_tx_qcomplete()

It removes necessity to get evq pointer by its index in soft context.

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 277886

sfxge: Make it possible to build without EVQ statistics

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 277884

sfxge: Change sfxge_ev_qpoll() proto to avoid EVQ pointers array access

It was the only place on data path where sc->evq array is accessed.

Sponsored by: Solarflare Communications, Inc.
Approved by: gnn (mentor)

MFC: 272328

Make size of Tx and Rx rings configurable

Required size of event queue is calculated now.

Submitted by: Andrew Rybchenko <arybchenko at solarflare.com>
Sponsored by: Solarflare Communications, Inc.

MFC: 272325

cleanup: code style fixes

Remove trailing whitespaces and tabs.
Enclose value in return statements in parentheses.
Use tabs after #define.
Do not skip comparison with 0/NULL in boolean expressions.

Submitted by: Andrew Rybchenko <arybchenko at solarflare.com>
Sponsored by: Solarflare Communications, Inc.

MFC: 263302, 264461, 264772

263302:
fix mbuf leak if it does not fit in software queue

264461:
Commit various fixes for the SolarFlare drivers, in particular
this set of patches fixes support for systems with > 32 cores.

Details include

sfxge: RXQ index (not label) comes from FW in flush done/failed events

Change the second argument name of the efx_rxq_flush_done_ev_t and
efx_rxq_flush_failed_ev_t prototypes to highlight that RXQ index (not label)
comes from FW in flush done and failed events.

sfxge: TXQ index (not label) comes from FW in flush done event

Change the second argument name of the efx_txq_flush_done_ev_t prototype to
highlight that TXQ index (not label) comes from FW in flush done event.

sfxge: use TXQ type as label to support more than 32 TXQs

There are 3 TXQs in event queue 0 and 1 TXQ (with TCP/UDP checksum offload)
in all other event queues.

264772:
Check that port is started when MAC filter is set

The MAC filter set may be called without softc_lock held in the case of
SIOCADDMULTI and SIOCDELMULTI ioctls. The ioctl handler checks IFF_DRV_RUNNING
flag which implies port started, but it is not guaranteed to remain.
softc_lock shared lock can't be held in the case of these ioctls processing,
since it results in failure where kernel complains that non-sleepable
lock is held in sleeping thread.

Both problems are repeatable on LAG with LACP proto bring up.

Submitted by: Andrew Rybchenko <Andrew.Rybchenko at oktetlabs.ru>
Sponsored by: Solarflare Communications, Inc.