MFC r343990:
net80211: hide casts for 'i_seq' field offset calculation inside
ieee80211_getqos() and reuse it in various places.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

net80211: drop some unused variables / local macros

Most of them left after some commits (r178354, r191544, r287197 etc.);
some were never used.

Found by: Clang Static Analyzer

net80211: hide subtype mask & shift in function call.

Hide subtype mask/shift (which is used for index calculation
in ieee80211_mgt_subtype_name[] array) in function call.

Tested with RTL8188CUS, STA mode.

Reviewed by: adrian
Differential Revision: https://reviews.freebsd.org/D5369

net80211: fix panic for DWDS vaps

Since r248069, TX IC lock must obtained before calling ieee80211_encap()
and/or ieee80211_parent_xmitpkt().

Tested with: Intel 3945BG.

net80211: eliminate copy-paste nearby ieee80211_check_rxseq()

Approved by: adrian (mentor)
Differential Revision: https://reviews.freebsd.org/D4043

net80211: add few missing subtype names.

- Add definitions for Timing Advertisement and Control Wrapper frames.
- Refresh ieee80211_mgt_subtype_name and ieee80211_ctl_subtype_name
arrays.
- Count Timing Advertisement frames as discarded management frames in all
modes.

Approved by: adrian (mentor)
Differential Revision: https://reviews.freebsd.org/D5331

net80211: free node reference in the ieee80211_parent_xmitpkt() when error happened.

Move error handling into ieee80211_parent_xmitpkt() instead of spreading it
between functions.

Submitted by: <s3erios@gmail.com>
Differential Revision: https://reviews.freebsd.org/D3772

Begin plumbing ieee80211_rx_stats through the receive path.

Smart NICs with firmware (eg wpi, iwn, the new atheros parts, the intel 7260
series, etc) support doing a lot of things in firmware. This includes but
isn't limited to things like scanning, sending probe requests and receiving
probe responses. However, net80211 doesn't know about any of this - it still
drives the whole scan/probe infrastructure itself.

In order to move towards suppoting smart NICs, the receive path needs to
know about the channel/details for each received packet. In at least
the iwn and 7260 firmware (and I believe wpi, but I haven't tried it yet)
it will do the scanning, power-save and off-channel buffering for you -
all you need to do is handle receiving beacons and probe responses on
channels that aren't what you're currently on. However the whole receive
path is peppered with ic->ic_curchan and manual scan/powersave handling.
The beacon parsing code also checks ic->ic_curchan to determine if the
received beacon is on the correct channel or not.[1]

So:

* add freq/ieee values to ieee80211_rx_stats;
* change ieee80211_parse_beacon() to accept the 'current' channel
as an argument;
* modify the iv_input() and iv_recv_mgmt() methods to include the rx_stats;
* add a new method - ieee80211_lookup_channel_rxstats() - that looks up
a channel based on the contents of ieee80211_rx_stats;
* if it exists, use it in the mgmt path to switch the current channel
(which still defaults to ic->ic_curchan) over to something determined
by rx_stats.

This is enough to kick-start scan offload support in the Intel 7260
driver that Rui/I are working on. It also is a good start for scan
offload support for a handful of existing NICs (wpi, iwn, some USB
parts) and it'll very likely dramatically improve stability/performance
there. It's not the whole thing - notably, we don't need to do powersave,
we should not scan all channels, and we should leave probe request sending
to the firmware and not do it ourselves. But, this allows for continued
development on the above features whilst actually having a somewhat
working NIC.

TODO:

* Finish tidying up how the net80211 input path works.
Right now ieee80211_input / ieee80211_input_all act as the top-level
that everything feeds into; it should change so the MIMO input routines
are those and the legacy routines are phased out.

* The band selection should be done by the driver, not by the net80211
layer.

* ieee80211_lookup_channel_rxstats() only determines 11b or 11g channels
for now - this is enough for scanning, but not 100% true in all cases.
If we ever need to handle off-channel scan support for things like
static-40MHz or static-80MHz, or turbo-G, or half/quarter rates,
then we should extend this.

[1] This is a side effect of frequency-hopping and CCK modes - you
can receive beacons when you think you're on a different channel.
In particular, CCK (which is used by the low 11b rates, eg beacons!)
is decodable from adjacent channels - just at a low SNR.
FH is a side effect of having the hardware/firmware do the frequency
hopping - it may pick up beacons transmitted from other FH networks
that are in a different phase of hopping frequencies.

Properly update TX statistics for wlan(4).
ieee80211_pwrsave() can fail due to queue overflow, check its return code
and increment oerrors counter when it fails. Also handle more error cases
and update oerrors counter when we don't send mbuf due to some errors.
Return ENETDOWN when parent interface isn't ready. Update obytes and omcasts
counters in corresponding places.

PR: 184626
Differential Revision: https://reviews.freebsd.org/D2621
Reviewed by: adrian
MFC after: 1 week

Do not check sequence number for QoS Null frames; set it for generated QoS Null
frames to 0

From IEEE Std. 802.11-2012, 8.3.2.1 "Data frame format", p. 415 (513):
"The Sequence Control field for QoS (+)Null frames is ignored by the receiver
upon reception."

At this moment, any <mode>_input() function interprets them as regular QoS data
frames with TID = 0. As a result, stations, that use another TX sequence for
QoS Null frames (e.g. wpi(4), where (QoS) Null frames are generated by the
firmware), may experience significant packet loss with any other NIC in hostap
mode.

Tested:

* wpi(4) (author)
* iwn(4) - Intel 5100, STA mode (me)

PR: kern/200128
Submitted by: Andriy Voskoboinyk <s3erios@gmail.com>

Mechanically convert to if_inc_counter().

Rename definition of IEEE80211_FC1_WEP to IEEE80211_FC1_PROTECTED.

The origin of WEP comes from IEEE Std 802.11-1997 where it defines
whether the frame body of MAC frame has been encrypted using WEP
algorithm or not.
IEEE Std. 802.11-2007 changes WEP to Protected Frame, indicates
whether the frame is protected by a cryptographic encapsulation
algorithm.

Reviewed by: adrian, rpaulo

The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare
to this event, adding if_var.h to files that do need it. Also, include
all includes that now are included due to implicit pollution via if_var.h

Sponsored by: Netflix
Sponsored by: Nginx, Inc.

Convert net80211 over to using if_transmit for the dispatch from the
upper layer(s).

This eliminates the if_snd queue from net80211. Yay!

This unfortunately has a few side effects:

* It breaks ALTQ to net80211 for now - sorry everyone, but fixing
parallelism and eliminating the if_snd queue is more important
than supporting this broken traffic scheduling model. :-)

* There's no VAP and IC flush methods just yet - I think I'll add
some NULL methods for now just as placeholders.

* It reduces throughput a little because now net80211 will drop packets
rather than buffer them if the driver doesn't do its own buffering.
This will be addressed in the future as I implement per-node software
queues.

Tested:

* ath(4) and iwn(4) in STA operation

Bring over my initial work from the net80211 TX locking branch.

This patchset implements a new TX lock, covering both the per-VAP (and
thus per-node) TX locking and the serialisation through to the underlying
physical device.

This implements the hard requirement that frames to the underlying physical
device are scheduled to the underlying device in the same order that they
are processed at the VAP layer. This includes adding extra encapsulation
state (such as sequence numbers and CCMP IV numbers.) Any order mismatch
here will result in dropped packets at the receiver.

There are multiple transmit contexts from the upper protocol layers as well
as the "raw" interface via the management and BPF transmit paths.
All of these need to be correctly serialised or bad behaviour will result
under load.

The specifics:

* add a new TX IC lock - it will eventually just be used for serialisation
to the underlying physical device but for now it's used for both the
VAP encapsulation/serialisation and the physical device dispatch.

This lock is specifically non-recursive.

* Methodize the parent transmit, vap transmit and ic_raw_xmit function
pointers; use lock assertions in the parent/vap transmit routines.

* Add a lock assertion in ieee80211_encap() - the TX lock must be held
here to guarantee sensible behaviour.

* Refactor out the packet sending code from ieee80211_start() - now
ieee80211_start() is just a loop over the ifnet queue and it dispatches
each VAP packet send through ieee80211_start_pkt().

Yes, I will likely rename ieee80211_start_pkt() to something that
better reflects its status as a VAP packet transmit path. More on
that later.

* Add locking around the management and BAR TX sending - to ensure that
encapsulation and TX are done hand-in-hand.

* Add locking in the mesh code - again, to ensure that encapsulation
and mesh transmit are done hand-in-hand.

* Add locking around the power save queue and ageq handling, when
dispatching to the parent interface.

* Add locking around the WDS handoff.

* Add a note in the mesh dispatch code that the TX path needs to be
re-thought-out - right now it's doing a direct parent device transmit
rather than going via the vap layer. It may "work", but it's likely
incorrect (as it bypasses any possible per-node power save and
aggregation handling.)

Why not a per-VAP or per-node lock?

Because in order to ensure per-VAP ordering, we'd have to hold the
VAP lock across parent->if_transmit(). There are a few problems
with this:

* There's some state being setup during each driver transmit - specifically,
the encryption encap / CCMP IV setup. That should eventually be dragged
back into the encapsulation phase but for now it lives in the driver TX path.
This should be locked.

* Two drivers (ath, iwn) re-use the node->ni_txseqs array in order to
allocate sequence numbers when doing transmit aggregation. This should
also be locked.

* Drivers may have multiple frames queued already - so when one calls
if_transmit(), it may end up dispatching multiple frames for different
VAPs/nodes, each needing a different lock when handling that particular
end destination.

So to be "correct" locking-wise, we'd end up needing to grab a VAP or
node lock inside the driver TX path when setting up crypto / AMPDU sequence
numbers, and we may already _have_ a TX lock held - mostly for the same
destination vap/node, but sometimes it'll be for others. That could lead
to LORs and thus deadlocks.

So for now, I'm sticking with an IC TX lock. It has the advantage of
papering over the above and it also has the added advantage that I can
assert that it's being held when doing a parent device transmit.
I'll look at splitting the locks out a bit more later on.

General outstanding net80211 TX path issues / TODO:

* Look into separating out the VAP serialisation and the IC handoff.
It's going to be tricky as parent->if_transmit() doesn't give me the
opportunity to split queuing from driver dispatch. See above.

* Work with monthadar to fix up the mesh transmit path so it doesn't go via
the parent interface when retransmitting frames.

* Push the encryption handling back into the driver, if it's at all
architectually sane to do so. I know it's possible - it's what mac80211
in Linux does.

* Make ieee80211_raw_xmit() queue a frame into VAP or parent queue rather
than doing a short-cut direct into the driver. There are QoS issues
here - you do want your management frames to be encapsulated and pushed
onto the stack sooner than the (large, bursty) amount of data frames
that are queued. But there has to be a saner way to do this.

* Fragments are still broken - drivers need to be upgraded to an if_transmit()
implementation and then fragmentation handling needs to be properly fixed.

Tested:

* STA - AR5416, AR9280, Intel 5300 abgn wifi
* Hostap - AR5416, AR9160, AR9280
* Mesh - some testing by monthadar@, more to come.

Mechanically substitute flags from historic mbuf allocator with
malloc(9) flags within sys.

Exceptions:

- sys/contrib not touched
- sys/mbuf.h edited manually

Fix some corner cases in the net80211 sequence number retransmission
handling.

The current sequence number code does a few things incorrectly:

* It didn't try eliminating duplications from HT nodes. I guess it's assumed
that out of order / retransmission handling would be handled by the AMPDU RX
routines. If a HT node isn't doing AMPDU RX, then retransmissions need to
be eliminated. Since most of my debugging is based on this (as AMPDU TX
software packet aggregation isn't yet handled), handle this corner case.

* When a sequence number of 4095 was received, any subsequent sequence number
is going to be (by definition) less than 4095. So if the following sequence
number (0) doesn't initially occur and the retransmit is received, it's
incorrectly eliminated by the IEEE80211_FC1_RETRY && SEQ_LEQ() check.
Try to handle this better.

This almost completely eliminates out of order TCP statistics showing up during
iperf testing for the 11a, 11g and non-aggregate 11n AMPDU RX case. The only
other packet loss conditions leading to this are due to baseband resets or
heavy interference.

Make sure to only accept and handle action frames which are for us. In
promiscuous mode we might receive stuff which otherwise gets filtered
by hardware.

Add a new mgmt subtype "ACTION NO ACK" defined in 802.11n-2009, while here
clean up parts of the *_recv_mgmt() functions.
- make sure appropriate counters are bumped and debug messages are printed
- order the unhandled subtypes by value and add a few missing ones
- fix some whitespace nits
- remove duplicate code in adhoc_recv_mgmt()
- remove a useless comment, probably left in while c&p

When taking the AMPDU reorder fastpath, need_tap wasn't being
initialized. Initialize on declaration to avoid this.

Found with: clang static analyzer

Add ieee80211_ageq; a facility for staging packets that require
long-term work before they can be serviced. Packets are tagged and
assigned an age (in seconds) at the point they are added to the
queue. If a packet is not retrieved before it's age expires it is
reclaimed. Tagging can take two forms: a reference to an ieee80211_node
(as happens in the tx path) or an opaque token in cases where there
is no reference or the node structure is not stable (i.e. it's going
to be destroyed).

o add ic_stageq to replace the per-node wds staging queue used for
dynamic wds
o add ieee80211_mac_hash for building ageq tokens; this computes a
32-bit hash from an 802.11 mac address (copied from the bridge)
o while here fix a stray ';' noticed in IEEE80211_PSQ_INIT

Reviewed by: rpaulo
Approved by: re (kensmith)

Revamp 802.11 action frame handling:
o add a new facility for components to register send+recv handlers
o ieee80211_send_action and ieee80211_recv_action now use the registered
handlers to dispatch operations
o rev ieee80211_send_action api to enable passing arbitrary data
o rev ieee80211_recv_action api to pass the 802.11 frame header as it may
be difficult to locate
o update existing IEEE80211_ACTION_CAT_BA and IEEE80211_ACTION_CAT_HT handling
o update mwl for api rev

Reviewed by: rpaulo
Approved by: re (kensmith)

ieee80211_dwds_mcast(): check the correct mbuf ptr after encap.

Fix handling of devices w/o radiotap support:
o do not attach DLT_IEEE802_11_RADIO unless both tx and rx headers are
present; this is assumed in the capture code paths
o verify the above with asserts in ieee80211_radiotap_{rx,tx}
o add missing checks for active taps before calling ieee80211_radiotap_rx

Overhaul monitor mode handling:
o replace DLT_IEEE802_11 support in net80211 with DLT_IEEE802_11_RADIO
and remove explicit bpf support from wireless drivers; drivers now
use ieee80211_radiotap_attach to setup shared data structures that
hold the radiotap header for each packet tx/rx
o remove rx timestamp from the rx path; it was used only by the tdma support
for debugging and was mostly useless due to it being 32-bits and mostly
unavailable
o track DLT_IEEE80211_RADIO bpf attachments and maintain per-vap and
per-com state when there are active taps
o track the number of monitor mode vaps
o use bpf tap and monitor mode vap state to decide when to collect radiotap
state and dispatch frames; drivers no longer explicitly directly check
bpf state or use bpf calls to tap frames
o handle radiotap state updates on channel change in net80211; drivers
should not do this (unless they bypass net80211 which is almost always
a mistake)
o update various drivers to be more consistent/correct in handling radiotap
o update ral to include TSF in radiotap'd frames
o add promisc mode callback to wi

Reviewed by: cbzimmer, rpaulo, thompsa

print both fc bytes when hitting a protocol version mismatch

o add missing 802.3 bpf tap
o add 802.11 bpf tap to output path now that it's removed from ieee80211_encap

o update dwds mcast handling after hoisting ieee80211_encap: frames need
to be encapsulated before dispatching to the driver
o eliminate M_WDS now that we call ieee80211_encap directly and can supply
the wds vap to indicate a 4-address frame should be created

split Atheros SuperG support out into it's own file that's included only
with a new IEEE80211_SUPPORT_SUPERG option

follow prevailing style

convert calls to IFQ_HANDOFF to if_transmit

Fix checks for fast frames negotiation. ni_ath_flags holds the
capabilities reported by the ap. These need to be cross-checked
against the local configuration in the vap. Previously we were
only checking the ap capabilities which meant that if an ap reported
it was ff-capable but we were not setup to use them we'd try to do
ff aggregation and drop the frame.

There are a number of problems to be fixed here but applying this
fix immediately as the problem causes all traffic to stop (and has
not workaround).

Reported by: Ashish Shukla

Cleanup AMPDU handling:

For receive:
o explicitly tag rx frames w/ M_AMPDU instead of passing frames through
the reorder processing according to the node having HT and the frame
being QoS data
o relax ieee80211_ampdu_reorder asserts to allow any frame to be passed
in, unsuitable frames are returned to the caller for normal processing;
this permits drivers that cannot inspect the PLCP to mark all data
frames as potential ampdu candidates with only a small penalty
o add M_AMPDU_MPDU to identify frames resubmitted from the reorder q

For transmit:
o tag aggregation candidates with M_AMPDU_MPDU
o fix the QoS ack policy set in ampdu subframes; we only support immediate
BA streams which should be marked for "normal ack" to get implicit block
ack behaviour; interestingly certain vendor parts BA'd frames with the
11e BA ack policy set
o do not assign a sequence # to aggregation candidates; this must be done
when frames are submitted for transmit (NB: this can/will be handled
better when aggregation is pulled up to net80211)

no need to stop the sw beacon miss timer; it's not used with adhoc or wds

Multi-bss (aka vap) support for 802.11 devices.

Note this includes changes to all drivers and moves some device firmware
loading to use firmware(9) and a separate module (e.g. ral). Also there
no longer are separate wlan_scan* modules; this functionality is now
bundled into the wlan module.

Supported by: Hobnob and Marvell
Reviewed by: many
Obtained from: Atheros (some bits)