- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Align the PCI Express #defines with the style used for the PCI-X
#defines. This also has the advantage that it makes the names more
compact, iand also allows us to correct the non-uniform naming of
the PCIM_LINK_* defines, making them all consistent amongst themselves.

This is a mostly mechanical rename:
s/PCIR_EXPRESS_/PCIER_/g
s/PCIM_EXP_/PCIEM_/g
s/PCIM_LINK_/PCIEM_LINK_/g

When this is MFC'd, #defines will be added for the old names to assist
out-of-tree drivers.

Discussed with: jhb
MFC after: 1 week

- Updated TOE support in the kernel.

- Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs.
These are available as t3_tom and t4_tom modules that augment cxgb(4)
and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as
usual with or without these extra features.

- iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the
works and will follow soon.

Build-tested with make universe.

30s overview
============
What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the
capabilities of an interface:
# ifconfig -m | grep TOE

Enable/disable TCP offload on an interface (just like any other ifnet
capability):
# ifconfig cxgbe0 toe
# ifconfig cxgbe0 -toe

Which connections are offloaded? Look for toe4 and/or toe6 in the
output of netstat and sockstat:
# netstat -np tcp | grep toe
# sockstat -46c | grep toe

Reviewed by: bz, gnn
Sponsored by: Chelsio communications.
MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)

There is no need to test __FreeBSD_version for features that have
been around for a long time now (7.1-ish or even earlier); assume
they are present. These includes MSI, TSO, LRO, VLAN, INTR_FILTERS,
FIRMWARE, etc.

Also, eliminate some dead code and clean up in other places as part
of this quick once-over.

MFC after: 1 week

cxgb(4) updates, including:
- support for the new Gen-2, BT, and LP-CR cards.
- T3 firmware 7.7.0
- shared "common code" updates.

Approved by: gnn (mentor)
Obtained from: Chelsio
MFC after: 1 month

Greatly simplify cxgb by removing almost all of the custom mbuf management logic

- remove mbuf iovec - useful, but adds too much complexity when isolated to
the driver

- remove driver private caching - insufficient benefit over UMA to justify
the added complexity and maintenance overhead

- remove separate logic for managing multiple transmit queues, with the
new drbr routines the control flow can be made to much more closely resemble
legacy drivers

- remove dedicated service threads, with per-cpu callouts one can get the same
benefit much more simply by registering a callout 1 tick in the future if there
are still buffered packets

- remove embedded mbuf usage - Jeffr's changes will (I hope) soon be integrated
greatly reducing the overhead of using kernel APIs for reference counting
clusters

- add hysteresis to descriptor coalescing logic

- add coalesce threshold sysctls to allow users to decide at run-time
between optimizing for forwarding / UDP or optimizing for TCP

- add once per second watchdog to effectively close the very rare races
occurring from coalescing

- incorporate Navdeep's changes to the initialization path required to
convert port and adapter locks back to ordinary mutexes (silencing BPF
LOR complaints)

- enable prefetches in get_packet and tx cleaning

Reviewed by: navdeep@
MFC after: 2 weeks

Add backward compatibility ifdefs for non-multiq kernels

- bump __FreeBSD version to reflect added buf_ring, memory barriers,
and ifnet functions

- add memory barriers to <machine/atomic.h>
- update drivers to only conditionally define their own

- add lockless producer / consumer ring buffer
- remove ring buffer implementation from cxgb and update its callers

- add if_transmit(struct ifnet *ifp, struct mbuf *m) to ifnet to
allow drivers to efficiently manage multiple hardware queues
(i.e. not serialize all packets through one ifq)
- expose if_qflush to allow drivers to flush any driver managed queues

This work was supported by Bitgravity Inc. and Chelsio Inc.

For now our LRO code (tcp_lro.c) only supports IPv4 properly thus
only enable if INET is on.

Reviewed by: kmacy
MFC after: 2 months

Update cxgb include paths to not require prefixing with dev/cxgb

Submitted by: Chelsio Inc.

Allow cxgb to be unified across versions by making newer features conditional

Submitted by: Chelsio Inc
MFC after: 3 days

Fix issue with tom loading by moving cxgb_log_tcb in to tom

MFC after: 3 days

Further whitespace and copyright cleanups to minimize the
delta with RELENG_7.

import vendor fixes to cxgb

conditionally define PANIC_IF, remove 'unlikely'

- update firmware to 5.0
- add support for T3C
- add DDP support (zero-copy receive)
- fix TOE transmit of large requests
- fix shutdown so that sockets don't remain in CLOSING state indefinitely
- register listeners when an interface is brought up after tom is loaded
- fix setting of multicast filter
- enable link at device attach
- exit tick handler if shutdown is in progress
- add helper for logging TCB
- add sysctls for dumping transmit queues

- note that TOE wxill not be MFC'd until after 7.0 has been finalized

MFC after: 3 days

remove superfluous locking from dequeue

Add extensive sanity checking to buf_ring

Add buf_ring_full utility function, make sure dequeue/enqueue see the latest
indexes

Make TCP offload work on HEAD (modulo negative interaction between sbcompress
and t3_push_frames).
- Import latest changes to cxgb_main.c and cxgb_sge.c from toestack p4 branch
- make driver local copy of tcp_subr.c and tcp_usrreq.c and override tcp_usrreqs so
TOE can also functions on versions with unmodified TCP

- add cxgb back to the build

overlead mbuf fields for use by toe

Import updated support code for the TOM (tcp offload module).

pull in changes made to RELENG_6 version in the process of doing the MFC

Supported by: Chelsio
Approved by: re (blanket)

Add back in support for normal mbuf chaining on RX under DISABLE_MBUF_IOVEC

Approved by: re(blanket)
Supported by: Chelsio

- integrate most recent changes from vendor branch and upgrade to firmware revision 4.5.5
- add filter support
- further improvements for T304
- recover gracefully from spurious immediate packets

Approved by: re(blanket)
Supported by: Chelsio
MFC after: 3 days

- Increase descriptors per call to start
- enqueue per-txq task
- fix per-txq task initialization

Approved by: re (blanket)

Tuning for small packet handling
- Double the number of descriptors that a single call to send can use
- Quadruple the number of descriptors that can be reclaimed per pass
- only run reclaim twice per second
- increase coalesce timer from 3.5us to 5us

fix printf warning on 64-bit platforms

(MFp4)
- upgrade to reflect state of 1.0.0.86
- move from firmware rev 3.2 to 4.0.0
- import driver bits for offload functionality
- remove binary distribution clause from top level files as it
runs counter to the intent of purely supporting the hardware

MFC after: 3 days

Convert driver RX path over to using mbuf iovec

make MSI-X the default and allocate up to mp_ncpus queues per port

MFC after: 3 days

Synchronize with version 1.0.071 of Chelsio's common code
(with the notable exception of improvements for using multiple TX queues)

This adds support for the T3B2 ASIC rev

Obtained from: Chelsio
MFC after: 3 days

#define L1_CACHE_BYTES for non-x86

define prefetch as a no-op macro for non-x86 arches

#define memory barrier macros for the non-i386 && non-amd64 case

add cxgb_config.h to define values that are defined in the Makefile when compiled as a
module

move prefetch out of cxgb_sge.c into header under arch conditional compilation

First of several commits for driver support for the Chelsio T3B 10 Gigabit
Ethernet adapter.

Reviewed by: scottl, sam

For those interested in the preliminary performance work see below.

Plots of mxge vs. cxgb running netpipe:

blocksize vs. bandwidth:
http://www.fsmware.com/chelsio.random/bsvsbw.gif

blocksize vs. RTT:
First of several commits for driver support for the Chelsio T3B 10 Gigabit
Ethernet adapter.

Reviewed by: scottl, sam

For those interested in the preliminary performance work see below.

Plots of mxge vs. cxgb running netpipe:

blocksize vs. bandwidth:
http://www.fsmware.com/chelsio.random/bsvsbw.gif

blocksize vs. RTT:
http://www.fsmware.com/chelsio.random/bsvstime.gif

blocksize vs. RTT for block sizes <= 10kb:
http://www.fsmware.com/chelsio.random/bsvstime_10kb.gif
http://www.fsmware.com/chelsio.random/bsvstime_10kb3.gif