net80211 / drivers: remove public use of ieee80211_node_incref()

ieee80211_node_incref() is the FreeBSD implementation of
ieee80211_ref_node(). Not being interested in the node returned
it was used as a shortcut in 3 drivers (ath, uath, wpi).
Replace the call with the public KPI of ieee80211_ref_node() and
ignore the result.
This leaves us with the single internal call going
ieee80211_ref_node() -> ieee80211_node_incref() and that should
help increasing portability but also limiting the places to trace
for node reference operations.

Sponsored by: The FreeBSD Foundation
MFC after: 4 weeks

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

ath(4): Fix two typos in source code comments

- s/overriden/overridden/

MFC after: 3 days

Provide MS() and SM() macros for 80211 and wireless drivers.

We have (two versions) of MS() and SM() macros which we use throughout
the wireless code. Change all but three places (ath_hal, rtwn, and rsu)
to the newly provided _IEEE80211_MASKSHIFT() and _IEEE80211_SHIFTMASK()
macros. Also change one internal case using both _S and _M instead of
just _S away from _M (one of the reasons rtwn and rsu were not changed).

This was done semi-mechanically. No functional changes intended.

Requested by: gnn (D26091)
Reviewed by: adrian (pre line wrap)
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC (d/b/a "Netgate")
Differential Revision: https://reviews.freebsd.org/D26539

ath: clean up empty lines in .c and .h files

[ath] reset hardware if this particular mac bug is seen.

I have to dig into why I'm seeing it on chips as late as the AR9380 era
stuff (as it's marked as an AR5416 bug, but who knows!) but i'm seeing
aggregate TX frames complete with no blockack bit set. So, everything
should be treated as a failure and do a hardware reset for good measure.

Tested:

* AR9380, STA mode
* AR9580 (5GHz), AP mode

[ath_rate_sample] Limit the tx schedules for A-MPDU ; don't take short retries
into account and remove the requirement that the MCS rate is "higher" if we're
considering a new rate.

Ok, another fun one.

* In order for reliable non-software retried higher MCS rates, the TX schedules
(inconsistently!) use hard-coded lower rates at the end of the schedule.
Now, hard-coded is a problem because (a) it means that aggregate formation
is limited by the SLOWEST rate, so I never formed large AMDU frames for
3 stream rates, and (b) if the AP disables lower rates as base rates, it
complains about "unknown rix" every frame you transmit at that rate.

So, for now just disable the third and fourth schedule entry for AMPDUs.
Now I'm forming 32k and 64k aggregates for the higher density MCS rates
much more reliably.

It would be much nicer if the rate schedule stuff wasn't fixed but instead
I'd just populate ath_rc_series[] when I fetch the rates. This is all a
holdover of ye olde pre-11n stuff and I really just need to nuke it.

But for now, ye hack.

* The check for "is this MCS rate better" based on MCS itself is just garbage.
It meant things like going MCS0->7 would be fine, and say 0->8->16 is fine,
(as they're equivalent encoding but 1,2,3 spatial streams), BUT it meant
going something like MCS7->11 would fail even though it's likely that
MCS11 would just be better, both for EWMA/BER and throughput.

So for now just use the average tx time. The "right" way for this comparison
would be to compare PHY bitrates rather than MCS / rate indexes, but I'm not
yet there. The bit rates ARE available in the PHY index, but honestly
I have a lot of other cleaning up to here before I think about that.

* Don't include the RTS/CTS retry count (and thus time) into the average tx time
caluation. It just makes temporarily failures make the rate look bad by
QUITE A LOT, as RTS/CTS exchanges are (a) long, and (b) mostly irrelevant
to the actual rate being tried. If we keep hitting RTS/CTS failures then
there's something ELSE wrong on the channel, not our selected rate.

[ath] [ath_rate] Extend ath_rate_sample to better handle 11n rates and aggregates.

My initial rate control code was .. suboptimal. I wanted to at least get MCS
rates sent, but it didn't do anywhere near enough to handle low signal level links
or remotely keep accurate statistics.

So, 8 years later, here's what I should've done back then.

* Firstly, I wasn't at all tracking packet sizes other than the two buckets
(250 and 1600 bytes.) So, extend it to include 4096, 8192, 16384, 32768 and
65536. I may go add 2048 at some point if I find it's useful.

This is important for a few reasons. First, when forming A-MPDU or AMSDU
aggregates the frame sizes are larger, and thus the TX time calculation
is woefully, increasingly wrong. Secondly, the behaviour of 802.11 channels
isn't some fixed thing, both due to channel conditions and radios themselves.
Notably, there was some observations done a few years ago on 11n chipsets
which noticed longer aggregates showed an increase in failed A-MPDU sub-frame
reception as you got further along in the transmit time. It could be due to
a variety of things - transmitter linearity, channel conditions changing,
frequency/phase drift, etc - but the observation was to potentially form
shorter aggregates to improve BER.

* .. and then modify the ath TX path to report the length of the aggregate sent,
so as the statistics kept would line up with the correct bucket.

* Then on the rate control look-up side - i was also only using the first frame
length for an A-MPDU rate control lookup which isn't good enough here.
So, add a new method that walks the TID software queue for that node to
find out what the likely length of data available is. It isn't ALL of the
data in the queue because we'll only ever send enough data to fit inside the
block-ack window, so limit how many bytes we return to roughly what ath_tx_form_aggr()
would do.

* .. and cache that in the first ath_buf in the aggregate so it and the eventual
AMPDU length can be returned to the rate control code.

* THEN, modify the rate control code to look at them both when deciding which bucket
to attribute the sent frame on. I'm erring on the side of caution and using the
size bucket that the lookup is based on.

Ok, so now the rate lookups and statistics are "more correct". However, MCS rates
are not the same as 11abg rates in that they're not a monotonically incrementing
set of faster rates and you can't assume that just because a given MCS rate fails,
the next higher one wouldn't work better or be a lower average tx time.

So, I had to do a bunch of surgery to the best rate and sample rate math.
This is the bit that's a WIP.

* First, simplify the statistics updates (update_stats()) to do a single pass on
all rates.
* Next, make sure that each rate average tx time is updated based on /its/ failure/success.
Eg if you sent a frame with { MCS15, MCS12, MCS8 } and MCS8 succeeded, MCS15 and MCS
12 would have their average tx time updated for /their/ part of the transmission,
not the whole transmission.
* Next, EWMA wasn't being fully calculated based on the /failures/ in each of the
rate attempts. So, if MCS15, MCS12 failed above but MCS8 didn't, then ensure
that the statistics noted that /all/ subframes failed at those rates, rather than
the eventual set of transmitted/sent frames. This ensures the EWMA /and/ average
TX time are updated correctly.
* When picking a sample rate and initial rate, probe rates aroud the current MCS
but limit it to MCS0..7 /for all spatial streams/, rather than doing crazy things
like hitting MCS7 and then probing MCS8 - MCS8 is basically MCS0 but two spatial
streams. It's a /lot/ slower than MCS7. Also, the reverse is true - if we're at
MCS8 then don't probe MCS7 as part of it, it's not likely to succeed.
* Fix bugs in pick_best_rate() where I was /immediately/ choosing the highest MCS
rate if there weren't any frames yet transmitted. I was defaulting to 25% EWMA and
.. then each comparison would accept the higher rate. Just skip those; sampling
will fill in the details.

So, this seems to work a lot better. It's not perfect; I'm still seeing a lot of
instability around higher MCS rates because there are bursts of loss/retransmissions
that aren't /too/ bad. But i'll keep iterating over this and tidying up my hacks.

Ok, so why this still something I'm poking at? rather than porting minstrel_ht?

ath_rate_sample tries to minimise airtime, not maximise throughput. I have
extended it with an EWMA based on sub-frame success/failures - high MCS rates
that have partially successful receptions still show super short average frame
times, but a /lot/ of retransmits have to happen for that to work.
So for MCS rates I also track this EWMA and ensure that the rates I'm choosing
don't have super crappy packet failures. I don't mind not getting lower
peak throughput versus minstrel_ht; instead I want to see if I can make "minimise
airtime" work well.

Tested:

* AR9380, STA mode
* AR9344, STA mode
* AR9580, STA/AP mode

[ath] [ath_rate] Add some extra data into the rate control lookup.

Right now (well, since I did this in 2011/2012) the rate control code
makes some super bad choices for 11n aggregates/rates, and it tracks
statistics even more questionably.

It's been long enough and I'm now trying to use it again daily, so let's
start by:

* telling the rate control code if it's an aggregate or not;
* being clearer about the TID - yes it can be extracted from the
ath_buf but this way it can be overridden by the caller without
changing the TID itself.

(This is for doing experiments with voice/video QoS at some point..)

* Return an optional field to limit how long the aggregate is in
microseconds. Right now the rate control code supplies a rate table
and the ath aggr form code will look at the rate table and limit
the aggregate size to 4ms at the slowest rate. Yeah, this is pretty
terrible.

* Add some more TODO comments around handling txpower, rate and
handling filtered frames status so if I continue to have spoons for
this I can go poke at it.

[net80211] convert all of the WME use over to a temporary copy of WME info.

This removes the direct WME info access in the ieee80211com struct and instead
provides a method of fetching the data. Right now it's a no-op but eventually
it'll turn into a per-VAP method for drivers that support it (eg iwn, iwm,
upcoming ath10k work) as things like p2p support require this kind of behaviour.

Tested:

* ath(4), STA and AP mode

TODO:

* yes, this is slightly stack size-y, but it is an important first step
to get drivers migrated over to a sensible WME API. A lot of per-phy things
need to be converted to per-VAP before P2P, 11ac firmware, etc stuff shows up.

sys/dev: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

[ath] prepare for "correct" group (bcast/mcast) address frame handling and software/hardware queue TID mapping.

When I initially did this 11n TX work in days of yonder, my 802.11 standards
clue was ... not as finely tuned. One of the things in 802.11-2012 (which
I guess technically was after I did this work, but I'm sure it was like this
in the previous rev?) is that among other traffic classes, three things are
important:

* group addressed frames should be default non-QoS, even if they're QoS frames, and
* group addressed frames should have a seqno out of a different space than the
per-TID QoS one; and because of this
* group addressed frames, being non-QoS, should never be in the Block-ACK window
for TX.

Now, net80211 and now this code cheats by using the non-QOS TID, but ideally
we'd introduce a separate seqno space just for multicast/group traffic for
TX and RX comparison.

Later extensions (eg reliable multicast / multimedia) express what one should do
when doing multicast traffic in a TID. Now, technically we /could/ do group traffic
as QoS traffic and throw it into a per-TID seqno space, but this definitely
introduces ordering issues when you take into account things like CABQ behaviour.
(Ie, if some traffic in the TID goes into the CABQ and some doesn't, because
it's doing a split of multicast and non-multicast traffic, then you have
seqno ordering issues.)

So, until someone implements 802.11vv reliable multicast / multimedia extensions,
group traffic is non-QoS.

Next, software/hardware queue TID mapping. In the past I believed the WME tagging
of frames because well, net80211 had a habit of tagging things like management
traffic with it. But, then we also map QoS traffic categories to TIDs as well.
So, we should obey the TID! But! then it put some management traffic into higher
WME categories too, as those frames don't have QoS TIDs. But! It'd do things like
put things like QoS action frames into higher WME categories, when they should
be kept in-order with the rest of the traffic for that TID. So! Given all of this,
the ath(4) driver does overrides to not trust the WME category.

I .. am undoing some of this. Now, the TID has a 1:1 mapping to the hardware
queue. The TID is the primary source of truth now for all QoS traffic.
The WME is only used for non-QoS traffic. This now means that any TID traffic
queued should be consistently queued regardless of WME, so things like the
"TX finished, do more TX" that is occuring right now for transmit handling
should be "better".

The consistent {TID, WME} -> hardware queue mapping is important for
transmit completion. It's used to schedule more traffic for that
particular TID, because that {many TID}:{1 TXQ} mapping in ath_tx_tid_sched()
is used for driving completion. Ie, when the hardware queue completes,
it'll walk that list of scheduled TIDs attached to that TXQ.

The eventual aim is to get ready for some other features around putting
some data into other hardware queues (eg for better PS-POLL support,
uAPSD, support, correct-er TDMA support, etc) which requires that
I tidy all of this up in preparation for then introducing further
TID scheduling that isn't linked to a hardware TXQ (likely a per-WME, per-TID
driver queue, and a per-node driver queue) to enable that.

Tested:

* AR9380, STA mode
* AR9380, AR9580, AP mode

[ath] log seqno, type and subtype when assigning sequence numbers for A-MPDU.

This is just to improve adrian-debugging.

[ath] fix thresholds for deciding to queue to the software queue and populate hardware frames

This is two fixes, which establishes what I /think/ is pretty close to the
theoretical PHY maximum speed on the AR9380 devices.

* When doing A-MPDU on a TID, don't queue to the hardware directly if
the hardware queue is busy. This gives us time to get more packets
queued up (and the hardware is busy, so there's no point in queuing
more to the hardware right now) to potentially form an A-MPDU.

This fixes up the throughput issue I was seeing where a couple hundred
single frames were being sent a second interspersed between A-MPDU
frames. It just happened that the software queue had exactly one
frame in it at that point. Queuing it until the hardware finishes
transmitting isn't exactly costly.

* When determining whether to dequeue from a software node/TID queue into
the hardware queue, fix up the checks to work right for EDMA chips
(ar9380 and later.) Before it was not dispatching anything until
the FIFO was empty. Now we allow it to dispatch another aggregate
up to the hardware aggregate limit, like I intended with the earlier
work.

This allows a 5GHz HT40, short-GI, "htprotmode off" test at MCS23
to achieve 357 Mbit/sec in a one-way UDP test. The stars have to be
aligned /just right/ so there are no retries but it can happen.
Just don't expect it to work in an OTA test if your 2yo is running
around the room - MCS23 is very very sensitive to channel conditions.

Tested:

* AR9380 STA (test) -> AR9580 hostap

TODO:

* More thorough testing on pre-AR9380 chips (AR5416, AR9160, AR9280)
* (Finally) teach ath_rate_sample about throughput/latency rather than
air time, so I can get good transmit rates with a 2yo running around.

net80211: remove IEEE80211_RADIOTAP_TSFT field from transmit definitions.

This field may be used for received frames only.

Differential Revision: https://reviews.freebsd.org/D3826
Differential Revision: https://reviews.freebsd.org/D3827

[ath] set the relevant TOA/TOD locationing bits when trying to do locationing.

* Don't do RTS/CTS - experiments show that we get ACK frames for each of them
and this ends up causing the timestamps to look all funny.
* Set the HAL_TXDESC_POS bit, so the AR9300 HAL sets up the hardware to return
location and CSI information.

[ath] [ath_hal] break out the duration calculation to optionally include SIFS.

The pre-11n calculations include SIFS, but the 11n ones don't.

The reason is that (mostly) the 11n hardware is doing the SIFS calculation
for us but the pre-11n hardware isn't. This means that we're over-shooting
the times in the duration field for non-11n frames on 11n hardware, which
is OK, if not a little inefficient.

Now, this is all fine for what the hardware needs for doing duration math
for ACK, RTS/CTS, frame length, etc, but it isn't useful for doing PHY
duration calculations. Ie, given a frame to TX and its timestamp, what
would the end of the actual transmission time be; and similar for an
RX timestamp and figuring out its original length.

So, this adds a new field to the duration routines which requests
SIFS or no SIFS to be included. All the callers currently will call
it requesting SIFS, so this /should/ be a glorious no-op. I'm however
planning some future work around airtime fairness and positioning which
requires these routines to have SIFS be optional.

Notably though, the 11n version doesn't do any SIFS addition at the moment.
I'll go and tweak and verify all of the packet durations before I go and
flip that part on.

Tested:

* AR9330, STA mode
* AR9330, AP mode
* AR9380, STA mode

[ath] commit initial bluetooth coexistence support for the MCI NICs.

This is the initial framework to call into the MCI HAL routines and drive
the basic state engine.

The MCI bluetooth coex model uses a command channel between wlan and
bluetooth, rather than a 2-wire or 3-wire signaling protocol to control things.
This means the wlan and bluetooth chip exchange a lot more information and
signaling, even at the per-packet level. The NICs in question can share
the input LNA and output PA on the die, so they absolutely can't stomp
on each other in a silly fashion. It also allows for the bluetooth side
to signal when profiles come and go, so the driver can take appropriate
control. There's also the possibility of dynamic bluetooth/wlan duty cycle
control which I haven't yet really played with.

It configures things up with a static "wlan wins everything" coexistence,
configures up the available 2GHz channel map for bluetooth, sets a static
duty cycle for bluetooth/wifi traffic priority and drives the basics needed to
keep the MCI HAL code happy.

It doesn't do any actual coexistence except to default to "wlan wins everything",
which at least demonstrates that things do indeed work. Bluetooth inquiry frames
still trump wifi (including beacons), so that demonstrates things really do
indeed seem to work.

Tested:

* AR9462 (WB222), STA mode + bt
* QCA9565 (WB335), STA mode + bt

TODO:

* .. the rest of coexistence. yes, bluetooth, not people. That stuff's hard.
* It doesn't do the initial BT side calibration, which requires a WLAN chip
reset. I'll fix up the reset path a bit more first before I enable that.
* The 1-ant and 2-ant configuration bits aren't being set correctly in
if_ath_btcoex.c - I'll dig into that and fix it in a subsequent commit.
* It's not enabled by default for WB222/WB225 even though I believe it now
can be - I'll chase that up in a subsequent commit.

Obtained from: Qualcomm Atheros, Linux ath9k

dev/ath: minor spelling fixes in comments.

No functional change.

Reviewed by: adrian

[ath] turn the BA hardware bug back into a printf().

I saw this happen a couple of times and all I saw was a dump of the
transmit descriptors. Log the message for now so I can see whta happened.

net80211: move ieee80211_free_node() call on error from ic_raw_xmit() to ieee80211_raw_output().

This doesn't free the mbuf upon error; the driver ic_raw_xmit method is still
doing that.

Submitted by: <s3erios@gmail.com>
Differential Revision: https://reviews.freebsd.org/D3774

net80211: separate mbuf cleanup from ieee80211_fragment()

* Create ieee80211_free_mbuf() which frees a list of mbufs.
* Use it in the fragment transmit path and ath / uath transmit paths.
* Call it in xmit_pkt() if the transmission fails; otherwise fragments
may be leaked.

This should be a big no-op.

Submitted by: <s3erios@gmail.com>
Differential Revision: https://reviews.freebsd.org/D3769

Replay r286410. Change KPI of how device drivers that provide wireless
connectivity interact with the net80211 stack.

Historical background: originally wireless devices created an interface,
just like Ethernet devices do. Name of an interface matched the name of
the driver that created. Later, wlan(4) layer was introduced, and the
wlanX interfaces become the actual interface, leaving original ones as
"a parent interface" of wlanX. Kernelwise, the KPI between net80211 layer
and a driver became a mix of methods that pass a pointer to struct ifnet
as identifier and methods that pass pointer to struct ieee80211com. From
user point of view, the parent interface just hangs on in the ifconfig
list, and user can't do anything useful with it.

Now, the struct ifnet goes away. The struct ieee80211com is the only
KPI between a device driver and net80211. Details:

- The struct ieee80211com is embedded into drivers softc.
- Packets are sent via new ic_transmit method, which is very much like
the previous if_transmit.
- Bringing parent up/down is done via new ic_parent method, which notifies
driver about any changes: number of wlan(4) interfaces, number of them
in promisc or allmulti state.
- Device specific ioctls (if any) are received on new ic_ioctl method.
- Packets/errors accounting are done by the stack. In certain cases, when
driver experiences errors and can not attribute them to any specific
interface, driver updates ic_oerrors or ic_ierrors counters.

Details on interface configuration with new world order:
- A sequence of commands needed to bring up wireless DOESN"T change.
- /etc/rc.conf parameters DON'T change.
- List of devices that can be used to create wlan(4) interfaces is
now provided by net.wlan.devices sysctl.

Most drivers in this change were converted by me, except of wpi(4),
that was done by Andriy Voskoboinyk. Big thanks to Kevin Lo for testing
changes to at least 8 drivers. Thanks to pluknet@, Oliver Hartmann,
Olivier Cochard, gjb@, mmoll@, op@ and lev@, who also participated in
testing.

Reviewed by: adrian
Sponsored by: Netflix
Sponsored by: Nginx, Inc.

Remove most of the references of ifp->if_softc and replace with
references to ic->ic_softc.

This is in preparation for gleb's ifnet work.

Tested:

* ath(4), STA mode
* ath(4), hostap mode
* make universe

Revert the wifi ifnet changes until things are more baked and tested.

* 286410
* 286413
* 286416

The initial commit broke a variety of debug and features that aren't
in the GENERIC kernels but are enabled in other platforms.

Change KPI of how device drivers that provide wireless connectivity interact
with the net80211 stack.

Historical background: originally wireless devices created an interface,
just like Ethernet devices do. Name of an interface matched the name of
the driver that created. Later, wlan(4) layer was introduced, and the
wlanX interfaces become the actual interface, leaving original ones as
"a parent interface" of wlanX. Kernelwise, the KPI between net80211 layer
and a driver became a mix of methods that pass a pointer to struct ifnet
as identifier and methods that pass pointer to struct ieee80211com. From
user point of view, the parent interface just hangs on in the ifconfig
list, and user can't do anything useful with it.

Now, the struct ifnet goes away. The struct ieee80211com is the only
KPI between a device driver and net80211. Details:

- The struct ieee80211com is embedded into drivers softc.
- Packets are sent via new ic_transmit method, which is very much like
the previous if_transmit.
- Bringing parent up/down is done via new ic_parent method, which notifies
driver about any changes: number of wlan(4) interfaces, number of them
in promisc or allmulti state.
- Device specific ioctls (if any) are received on new ic_ioctl method.
- Packets/errors accounting are done by the stack. In certain cases, when
driver experiences errors and can not attribute them to any specific
interface, driver updates ic_oerrors or ic_ierrors counters.

Details on interface configuration with new world order:
- A sequence of commands needed to bring up wireless DOESN"T change.
- /etc/rc.conf parameters DON'T change.
- List of devices that can be used to create wlan(4) interfaces is
now provided by net.wlan.devices sysctl.

Most drivers in this change were converted by me, except of wpi(4),
that was done by Andriy Voskoboinyk. Big thanks to Kevin Lo for testing
changes to at least 8 drivers. Thanks to Olivier Cochard, gjb@, mmoll@,
op@ and lev@, who also participated in testing. Details here:

https://wiki.freebsd.org/projects/ifnet/net80211

Still, drivers: ndis, wtap, mwl, ipw, bwn, wi, upgt, uath were not
tested. Changes to mwl, ipw, bwn, wi, upgt are trivial and chances
of problems are low. The wtap wasn't compilable even before this change.
But the ndis driver is complex, and it is likely to be broken with this
commit. Help with testing and debugging it is appreciated.

Differential Revision: D2655, D2740
Sponsored by: Nginx, Inc.
Sponsored by: Netflix

Use device_printf() instead of if_printf(). No functional changes.

Mechanically convert to if_inc_counter().

Bring over some initial power save management support, reset path
fixes and beacon programming / debugging into the ath(4) driver.

The basic power save tracking:

* Add some new code to track the current desired powersave state; and
* Add some reference count tracking so we know when the NIC is awake; then
* Add code in all the points where we're about to touch the hardware and
push it to force-wake.

Then, how things are moved into power save:

* Only move into network-sleep during a RUN->SLEEP transition;
* Force wake the hardware up everywhere that we're about to touch
the hardware.

The net80211 stack takes care of doing RUN<->SLEEP<->(other) state
transitions so we don't have to do it in the driver.

Next, when to wake things up:

* In short - everywhere we touch the hardware.
* The hardware will take care of staying awake if things are queued
in the transmit queue(s); it'll then transit down to sleep if
there's nothing left. This way we don't have to track the
software / hardware transmit queue(s) and keep the hardware
awake for those.

Then, some transmit path fixes that aren't related but useful:

* Force EAPOL frames to go out at the lowest rate. This improves
reliability during the encryption handshake after 802.11
negotiation.

Next, some reset path fixes!

* Fix the overlap between reset and transmit pause so we don't
transmit frames during a reset.
* Some noisy environments will end up taking a lot longer to reset
than normal, so extend the reset period and drop the raise the
reset interval to be more realistic and give the hardware some
time to finish calibration.
* Skip calibration during the reset path. Tsk!

Then, beacon fixes in station mode!

* Add a _lot_ more debugging in the station beacon reset path.
This is all quite fluid right now.
* Modify the STA beacon programming code to try and take
the TU gap between desired TSF and the target TU into
account. (Lifted from QCA.)

Tested:

* AR5210
* AR5211
* AR5212
* AR5413
* AR5416
* AR9280
* AR9285

TODO:

* More AP, IBSS, mesh, TDMA testing
* Thorough AR9380 and later testing!
* AR9160 and AR9287 testing

Obtained from: QCA

Rewrite the cleanup code to, well, actually work right.

The existing cleanup code was based on the Atheros reference driver
from way back and stuff that was in Linux ath9k. It turned out to be ..
rather silly.

Specifically:

* The whole method of determining whether there's hardware-queued frames
was fragile and the BAW would never quite work right afterwards.

* The cleanup path wouldn't correctly pull apart aggregate frames in the
queue, so frames would not be freed and the BAW wouldn't be correctly
updated.

So to implement this:

* Pull the aggregate frames apart correctly and handle each separately;
* Make the atid->incomp counter just track the number of hardware queued
frames rather than try to figure it out from the BAW;
* Modify the aggregate completion path to handle it as a single frame
(atid->incomp tracks the one frame now, not the subframes) and
remove the frames from the BAW before completing them as normal frames;
* Make sure bf->bf_next is NULled out correctly;
* Make both aggregate session and non-aggregate path frames now be
handled via the incompletion path.

TODO:

* kill atid->incomp; the driver tracks the hardware queued frames
for each TID and so we can just use that.

This is a stability fix that should be merged back to stable/10.

Tested:

* AR5416, STA

MFC after: 7 days

* Modify the debugging output from pause/resume to note the TID and STA
MAC
* Now that the paused < 0 bugs have been identified, make the DPRINTF()
a device_printf() again. Anything else that shows up here needs to be
fixed immediately.

Tested:

* AR5416, STA mode

MFC after: 7 days

Make sure bf_next is NULL'ed out when we're completing up an aggregate
frame through the cleanup path.

Whilst here, fix the indenting for something I messed up.

Tested:

* AR5416, STA mode

Fix a cleanup hang if cleanup gets called _during_ an active cleanup.

During power save testing I noticed that the cleanup code is being
called during a RUN->RUN state transition. It's because the net80211
stack is treating that (for reasons I don't quitey know yet) as a
reassociation and this calls the node cleanup code. The reason it's
seeing a RUN->RUN transition is because during active power save
stuff it's possible that the RUN->SLEEP and SLEEP->RUN transitions
happen so quickly that the deferred net80211 vap state code
"loses" a transition, namely the intermediary SLEEP transition.

So, this was causing the node reassociation code to sometimes be called
twice in quick succession and this would result in ath_tx_tid_cleanup()
to be called again. The code calling it would always call pause, and
then only call resume if the TID didn't have "cleanup_inprogress" set.
Unfortunately it didn't check if it was already set on entry, so it
would pause but not call resume. Thus, paused would be called more
than once (once before each entry into ath-tx_tid_cleanup()) but resume
would only be called once when the cleanup state was finished.

This doesn't entirely fix all of the issues seen in the cleanup path
but it's a necessary first step.

Since this is a stability fix, it should be merged to stable/10 at some
point.

Tested:

* AR5416, STA mode

MFC after: 7 days

Add some debugging and forcing of the BAW to match what the current
tracked BAW actually is.

The net80211 code that completes a BAR will set tid->txa_start (the
BAW start) to whatever value was called when sending the BAR.
Now, in case there's bugs in my driver code that cause the BAW
to slip along, we should make sure that the new BAW we start
at is actually what we currently have it at, not what we've sent.

This totally breaks the specification and so this stays a printf().
If it happens then I need to know and fix it.

Whilst here, add some debugging updates:

* add TID logging to places where it's useful;
* use SEQNO().

Don't do continue inside the scheduler loop; we really need to check
if we've hit the end of the list and cycled around to the first
node again.

Obtained from: DragonflyBSD

Correct the actual definition of ath_tx_tid_filt_comp_single() to
match how it's used.

This is another bug that led to aggregate traffic hanging because
the BAW tracking stopped being accurate. In this instance, a filtered
frame that exceeded retries would return a non-error, which would
mean the caller would never remove it from the BAW. But it wouldn't
be added to the filtered list, so it would be lost forever. There'd
thus be a hole in the BAW that would never get transmitted and
this leads to a traffic hang.

Tested:

* Routerstation Pro, AR9220 AP

Add a comment explaining the obvious.

Don't resume a TID on each filtered frame completion - only do it if
we did suspend it.

The whole suspend/resume TID queue thing is supposed to be a matched
reference count - a subsystem (eg addba negotiation, BAR transmission,
filtered frames, etc) is supposed to call pause() once and then resume()
once.

ath_tx_tid_filt_comp_complete() is called upon the completion of any
filtered frame, regardless of whether the driver had aleady seen
a filtered frame and called pause().

So only call resume() if tid->isfiltered = 1, which indicates that
we had called pause() once.

This fixes a seemingly whacked and different problem - traffic hangs.

What was actually going on:

* There'd be some marginal link with crappy behaviour, causing filtered
frames and BAR TXing to occur;
* A BAR TX would occur, setting the new BAW (block-ack window) to seqno n;
* .. and pause() would be called, blocking further transmission;
* A filtered frame completion would occur from the hardware, but with
tid->isfiltered = 0 which indiciates we haven't actually marked
the queue yet as filtered;
* ath_tx_tid_filt_comp_complete() would call resume(), continuing
transmission;
* Some frames would be queued to the hardware, since the TID is now no
longer paused;
* .. and if some make it out and ACked successfully, the new BAW
may be seqno n+1 or more;
* .. then the BAR TX completes and sets the new seqno back to n.

At this point the BAW tracking would be loopy because the BAW
start was modified but the BAW ring buffer wasn't updated in lock
step.

Tested:

* Routerstation Pro + AR9220 AP

Throw the flush messages behind ATH_DEBUG_RESET as well.

These are needed to diagnose TX hangs that I and hiren are seeing.
Without it, the only way we'll see debugging is by having ATH_DEBUG_SW_TX
enabled and that is going to be very, very spammy.

ATH_DEBUG_RESET is fine; it's only going to be done during stuck beacon
situations in AP mode.

Whilst I'm here, and now that it's behind debugging, let's just disable
the "print only one" conditional. I'll eventually make it more tunable.

Tested:

* AR9220, hostap mode.

Call ieee80211_dump_pkt() based on IFF_DUMPPKTS().

MFC after: 3 days

Rename definition of IEEE80211_FC1_WEP to IEEE80211_FC1_PROTECTED.

The origin of WEP comes from IEEE Std 802.11-1997 where it defines
whether the frame body of MAC frame has been encrypted using WEP
algorithm or not.
IEEE Std. 802.11-2007 changes WEP to Protected Frame, indicates
whether the frame is protected by a cryptographic encapsulation
algorithm.

Reviewed by: adrian, rpaulo

Include <sys/ktr.h>, since we need it if ATH_DEBUG is defined.

The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare
to this event, adding if_var.h to files that do need it. Also, include
all includes that now are included due to implicit pollution via if_var.h

Sponsored by: Netflix
Sponsored by: Nginx, Inc.

Add a missing comma.

Move a lot of debugging printf's to DPRINTF.

Approved by: adrian
MFC after: 2 weeks

Log the MAC address of the node in question rather than the pointer.

Shuffle around the cleanup unpause calls a bit.

Migrate ath(4) to now use if_transmit instead of the legacy if_start
and if queue mechanism; also fix up (non-11n) TX fragment handling.

This may result in a bit of a performance drop for now but I plan on
debugging and resolving this at a later stage.

Whilst here, fix the transmit path so fragment transmission works.

The TX fragmentation handling is a bit more special. In order to
correctly transmit TX fragments, there's a bunch of corner cases that
need to be handled:

* They must be transmitted back to back, in the same order..
* .. ie, you need to hold the TX lock whilst transmitting this
set of fragments rather than interleaving it with other MSDUs
destined to other nodes;
* The length of the next fragment is required when transmitting, in
order to correctly set the NAV field in the current frame to the
length of the next frame; which requires ..
* .. that we know the transmit duration of the next frame, which ..
* .. requires us to set the rate of all fragments to the same length,
or make the decision up-front, etc.

To facilitate this, I've added a new ath_buf field to describe the
length of the next fragment. This avoids having to keep the mbuf
chain together. This used to work before my 11n TX path work because
the ath_tx_start() routine would be handed a single mbuf with m_nextpkt
pointing to the next frame, and that would be maintained all the way
up to when the duration calculation was done. This doesn't hold
true any longer - the actual queuing may occur at any point in the
future (think ath_node TID software queuing) so this information
needs to be maintained.

Right now this does work for non-11n frames but it doesn't at all
enforce the same rate control decision for all frames in the fragment.
I plan on fixing this in a followup commit.

RTS/CTS has the same issue, I'll look at fixing this in a subsequent
commit.

Finaly, 11n fragment support requires the driver to have fully
decided what the rate scenario setup is - including 20/40MHz,
short/long GI, STBC, LDPC, number of streams, etc. Right now that
decision is (currently) made _after_ the NAV field value is updated.
I'll fix all of this in subsequent commits.

Tested:

* AR5416, STA, transmitting 11abg fragments
* AR5416, STA, 11n fragments work but the NAV field is incorrect for
the reasons above.

TODO:

* It would be nice to be able to queue mbufs per-node and per-TID so
we can only queue ath_buf entries when it's time to assemble frames
to send to the hardware.

But honestly, we should just do that level of software queue management
in net80211 rather than ath(4), so I'm going to leave this alone for now.

* More thorough AP, mesh and adhoc testing.

* Ensure that net80211 doesn't hand us fragmented frames when A-MPDU has
been negotiated, as we can't do software retransmission of fragments.

* .. set CLRDMASK when transmitting fragments, just to ensure.

Implement a separate hardware queue threshold for aggregate and non-aggr
traffic.

When transmitting non-aggregate traffic, we need to keep the hardware
busy whilst transmitting or small bursts in txdone/tx latency will
kill us.

This restores non-aggregate iperf performance, especially when doing
TDMA.

Tested:

* AR5416<->AR5416, TDMA
* AR5416 STA <-> AR9280 AP

More non-ATH_DEBUG build fixes.

Be (very) careful about how to add more TX DMA work.

The list-based DMA engine has the following behaviour:

* When the DMA engine is in the init state, you can write the first
descriptor address to the QCU TxDP register and it will work.

* Then when it hits the end of the list (ie, it either hits a NULL
link pointer, OR it hits a descriptor with VEOL set) the QCU
stops, and the TxDP points to the last descriptor that was transmitted.

* Then when you want to transmit a new frame, you can then either:
+ write the head of the new list into TxDP, or
+ you write the head of the new list into the link pointer of the
last completed descriptor (ie, where TxDP points), then kick
TxE to restart transmission on that QCU>

* The hardware then will re-read the descriptor to pick up the link
pointer and then jump to that.

Now, the quirks:

* If you write a TxDP when there's been no previous TxDP (ie, it's 0),
it works.

* If you write a TxDP in any other instance, the TxDP write may actually
fail. Thus, when you start transmission, it will re-read the last
transmitted descriptor to get the link pointer, NOT just start a new
transmission.

So the correct thing to do here is:

* ALWAYS use the holding descriptor (ie, the last transmitted descriptor
that we've kept safe) and use the link pointer in _THAT_ to transmit
the next frame.

* NEVER write to the TxDP after you've done the initial write.

* .. also, don't do this whilst you're also resetting the NIC.

With this in mind, the following patch does basically the above.

* Since this encapsulates Sam's issues with the QCU behaviour w/ TDMA,
kill the TDMA special case and replace it with the above.

* Add a new TXQ flag - PUTRUNNING - which indicates that we've started
DMA.

* Clear that flag when DMA has been shutdown.

* Ensure that we're not restarting DMA with PUTRUNNING enabled.

* Fix the link pointer logic during TXQ drain - we should always ensure
the link pointer does point to something if there's a list of frames.
Having it be NULL as an indication that DMA has finished or during
a reset causes trouble.

Now, given all of this, i want to nuke axq_link from orbit. There's now HAL
methods to get and set the link pointer of a descriptor, so what we
should do instead is to update the right link pointer.

* If there's a holding descriptor and an empty TXQ list, set the
link pointer of said holding descriptor to the new frame.

* If there's a non-empty TXQ list, set the link pointer of the
last descriptor in the list to the new frame.

* Nuke axq_link from orbit.

Note:

* The AR9380 doesn't need this. FIFO TX writes are atomic. As long as
we don't append to a list of frames that we've already passed to the
hardware, all of the above doesn't apply. The holding descriptor stuff
is still needed to ensure the hardware can re-read a completed
descriptor to move onto the next one, but we restart DMA by pushing in
a new FIFO entry into the TX QCU. That doesn't require any real
gymnastics.

Tested:

* AR5210, AR5211, AR5212, AR5416, AR9380 - STA mode.

Add some more debugging printf()s to complain if the ath_buf tx queue
doesn't match the actual hardware queue this frame is queued to.

I'm trying to ensure that the holding buffers are actually being queued
to the same TX queue as the holding buffer that they end up on.
I'm pretty sure this is all correct so if this complains, it'll be due
to some kind of subtle broken-ness that needs fixing.

This is only done for legacy hardware, not EDMA hardware.

Tested:

* AR5416 STA mode, very lightly

Tidy up the debugging - don't bother printing out TID pointers; now
that we are printing out the MAC address in these fields, just printing
out the TID is enough.

Limit the number of software queued frames when doing non-aggregation.

This should prevent the TX queue being filled with non-aggregate frames,
causing starvation and non-fair queue behaviour.

Implement my first cut at "correct" node power-save and
PS-POLL support.

This implements PS-POLL awareness i nthe

* Implement frame "leaking", which allows for a software queue
to be scheduled even though it's asleep
* Track whether a frame has been leaked or not
* Leak out a single non-AMPDU frame when transmitting aggregates
* Queue BAR frames if the node is asleep
* Direct-dispatch the rest of control and management frames.
This allows for things like re-association to occur (which involves
sending probe req/resp as well as assoc request/response) when
the node is asleep and then tries reassociating.
* Limit how many frames can set in the software node queue whilst
the node is asleep. net80211 is already buffering frames for us
so this is mostly just paranoia.
* Add a PS-POLL method which leaks out a frame if there's something
in the software queue, else it calls net80211's ps-poll routine.
Since the ath PS-POLL routine marks the node as having a single frame
to leak, either a software queued frame would leak, OR the next queued
frame would leak. The next queued frame could be something from the
net80211 power save queue, OR it could be a NULL frame from net80211.

TODO:

* Don't transmit further BAR frames (eg via a timeout) if the node is
currently asleep. Otherwise we may end up exhausting management frames
due to the lots of queued BAR frames.

I may just undo this bit later on and direct-dispatch BAR frames
even if the node is asleep.

* It would be nice to burst out a single A-MPDU frame if both ends
support this. I may end adding a FreeBSD IE soon to negotiate
this power save behaviour.

* I should make STAs timeout of power save mode if they've been in power
save for more than a handful of seconds. This way cards that get
"stuck" in power save mode don't stay there for the "inactivity" timeout
in net80211.

* Move the queue depth check into the driver layer (ath_start / ath_transmit)
rather than doing it in the TX path.

* There could be some naughty corner cases with ps-poll leaking.
Specifically, if net80211 generates a NULL data frame whilst another
transmitter sends a normal data frame out net80211 output / transmit,
we need to ensure that the NULL data frame goes out first.
This is one of those things that should occur inside the VAP/ic TX lock.
Grr, more investigations to do..

Tested:

* STA: AR5416, AR9280
* AP: AR5416, AR9280, AR9160

Improve the debugging output - use the MAC address rather than various
pointer values everywhere.

Oops, commit the other half of r250606.

Simplify this bit of code!

When doing BAW tracking, don't dereference a NULL pointer if the BAW
slot is actually NULL.

There's some races (likely in the BAR handling, sigh) which is causing
the pause/resume code to not be called completely symmetrically.

I'll chase down the root cause of that soon; this at least works around
the bug and tells me when it happens.

Use the new net80211 method to fetch the node TX power, rather than
directly referencing ni->ni_txpower.

This provides the hardware with a slightly more accurate idea of
the maximum TX power to be using.

This is part of a series to get per-packet TPC to work (better).

Tested:

* AR5416, hostap mode

Mark a couple of places where I think the dmamap isn't being unmapped
before the TX path is being aborted.

Right now it's in the TDMA code and I can live with that; but it really
should get fixed.

I'll do a more thorough audit of this code soon.

Ensure that we only call the busdma unmap/flush routines once, when
the buffer is being freed.

* When buffers are cloned, the original mapping isn't copied but it
wasn't freeing the mapping until later. To be safe, free the
mapping when the buffer is cloned.

* ath_freebuf() now no longer calls the busdma sync/unmap routines.

* ath_tx_freebuf() now calls sync/unmap.

* Call sync first, before calling unmap.

Tested:

* AR5416, STA mode

Use ATH_MAX_SCATTER rather than ATH_TXDESC.

ATH_MAX_SCATTER is used to size the ath_buf DMA segment array.
We thus should use it when checking sizes of things.

Implement the replacement EDMA FIFO code.

(Yes, the previous code temporarily broke EDMA TX. I'm sorry; I should've
actually setup ATH_BUF_FIFOEND on frames so txq->axq_fifo_depth was
cleared!)

This code implements a whole bunch of sorely needed EDMA TX improvements
along with CABQ TX support.

The specifics:

* When filling/refilling the FIFO, use the new TXQ staging queue
for FIFO frames

* Tag frames with ATH_BUF_FIFOPTR and ATH_BUF_FIFOEND correctly.
For now the non-CABQ transmit path pushes one frame into the TXQ
staging queue without setting up the intermediary link pointers
to chain them together, so draining frames from the txq staging
queue to the FIFO queue occurs AMPDU / MPDU at a time.

* In the CABQ case, manually tag the list with ATH_BUF_FIFOPTR and
ATH_BUF_FIFOEND so a chain of frames is pushed into the FIFO
at once.

* Now that frames are in a FIFO pending queue, we can top up the
FIFO after completing a single frame. This means we can keep
it filled rather than waiting for it drain and _then_ adding
more frames.

* The EDMA restart routine now walks the FIFO queue in the TXQ
rather than the pending queue and re-initialises the FIFO with
that.

* When restarting EDMA, we may have partially completed sending
a list. So stamp the first frame that we see in a list with
ATH_BUF_FIFOPTR and push _that_ into the hardware.

* When completing frames, only check those on the FIFO queue.
We should never ever queue frames from the pending queue
direct to the hardware, so there's no point in checking.

* Until I figure out what's going on, make sure if the TXSTATUS
for an empty queue pops up, complain loudly and continue.
This will stop the panics that people are seeing. I'll add
some code later which will assist in ensuring I'm populating
each descriptor with the correct queue ID.

* When considering whether to queue frames to the hardware queue
directly or software queue frames, make sure the depth of
the FIFO is taken into account now.

* When completing frames, tag them with ATH_BUF_BUSY if they're
not the final frame in a FIFO list. The same holding descriptor
behaviour is required when handling descriptors linked together
with a link pointer as the hardware will re-read the previous
descriptor to refresh the link pointer before contiuning.

* .. and if we complete the FIFO list (ie, the buffer has
ATH_BUF_FIFOEND set), then we don't need the holding buffer
any longer. Thus, free it.

Tested:

* AR9380/AR9580, STA and hostap
* AR9280, STA/hostap

TODO:

* I don't yet trust that the EDMA restart routine is totally correct
in all circumstances. I'll continue to thrash this out under heavy
multiple-TXQ traffic load and fix whatever pops up.

Remove the mcast path calls to ath_hal_gettxdesclinkptr() for axq_link -
they're no longer needed for the legacy path and they're not wanted
for the EDMA path.

Tested:

* AR9280, hostap + CABQ
* AR9380/AR9580, hostap + CABQ

Migrate the multicast queue assembly code to not use the axq_link pointer
and instead use the HAL method to set the link pointer.

Tested:

* AR9280, hostap mode, CABQ frames being queued and transmitted

Move the TXQ lock earlier in this routine - so to correctly protect the
link pointer check.

Overhaul the TXQ locking (again!) as part of some beacon/cabq timing
related issues.

Moving the TX locking under one lock made things easier to progress on
but it had one important side-effect - it increased the latency when
handling CABQ setup when sending beacons.

This commit introduces a bunch of new changes and a few unrelated changs
that are just easier to lump in here.

The aim is to have the CABQ locking separate from other locking.
The CABQ transmit path in the beacon process thus doesn't have to grab
the general TX lock, reducing lock contention/latency and making it
more likely that we'll make the beacon TX timing.

The second half of this commit is the CABQ related setup changes needed
for sane looking EDMA CABQ support. Right now the EDMA TX code naively
assumes that only one frame (MPDU or A-MPDU) is being pushed into each
FIFO slot. For the CABQ this isn't true - a whole list of frames is
being pushed in - and thus CABQ handling breaks very quickly.

The aim here is to setup the CABQ list and then push _that list_ to
the hardware for transmission. I can then extend the EDMA TX code
to stamp that list as being "one" FIFO entry (likely by tagging the
last buffer in that list as "FIFO END") so the EDMA TX completion code
correctly tracks things.

Major:

* Migrate the per-TXQ add/removal locking back to per-TXQ, rather than
a single lock.

* Leave the software queue side of things under the ATH_TX_LOCK lock,
(continuing) to serialise things as they are.

* Add a new function which is called whenever there's a beacon miss,
to print out some debugging. This is primarily designed to help
me figure out if the beacon miss events are due to a noisy environment,
issues with the PHY/MAC, or other.

* Move the CABQ setup/enable to occur _after_ all the VAPs have been
looked at. This means that for multiple VAPS in bursted mode, the
CABQ gets primed once all VAPs are checked, rather than being primed
on the first VAP and then having frames appended after this.

Minor:

* Add a (disabled) twiddle to let me enable/disable cabq traffic.
It's primarily there to let me easily debug what's going on with beacon
and CABQ setup/traffic; there's some DMA engine hangs which I'm finally
trying to trace down.

* Clear bf_next when flushing frames; it should quieten some warnings
that show up when a node goes away.

Tested:

* AR9280, STA/hostap, up to 4 vaps (staggered)
* AR5416, STA/hostap, up to 4 vaps (staggered)

TODO:

* (Lots) more AR9380 and later testing, as I may have missed something here.
* Leverage this to fix CABQ hanling for AR9380 and later chips.
* Force bursted beaconing on the chips that default to staggered beacons and
ensure the CABQ stuff is all sane (eg, the MORE bits that aren't being
correctly set when chaining descriptors.)

Now that the tx map field is correctly populated for both edma and
legacy chips, just use that.

Why'd I keep this here? remove it entirely now.

Fix two bugs:

* when pulling frames off of the TID queue, the ATH_TID_REMOVE()
macro decrements the axq_depth field. So don't do it twice.

* in ath_tx_comp_cleanup_aggr(), bf wasn't being reset to bf_first
before walking the buffer list to complete buffers; so those buffers
will leak.

Remove a now incorrect comment.

This comment dates back to my initial stab at TX aggregation completion,
where I didn't even bother trying to do software retries.

Disable the hw TID != buffer TID check.

I can 100% reliably trigger this on TID 1 traffic by using iperf -S 32
<client fields> to create traffic that maps to TID 1.

The reference driver doesn't do this check.

Disable debugging entries about BAW issues. I haven't seen any issues
to do with BAW tracking in the last 9 months or so.

A couple of quick tidyups:

* Delete this debugging print - I used it when debugging the initial
TX descriptor chaining code. It now works, so let's toss it.
It just confuses people if they enable TX descriptor debugging as they
get two slightly different versions of the same descriptor.

* Indenting.

Pull out the if_transmit() work and revert back to ath_start().

My changed had some rather significant behavioural changes to throughput.
The two issues I noticed:

* With if_start and the ifnet mbuf queue, any temporary latency
would get eaten up by some mbufs being queued. With ath_transmit()
queuing things to ath_buf's, I'd only get 512 TX buffers before I
couldn't queue any further frames.

* There's also some non-zero latency involved with TX being pushed
into a taskqueue via direct dispatch. Any time the scheduler didn't
immediately schedule the ath TX task would cause extra latency.
Various 1ge/10ge drivers implement both direct dispatch (if the TX
lock can be acquired) and deferred task transmission (if the TX lock
can't be acquired), with frames being pushed into a drbd queue.
I'll have to do this at some point, but until I figure out how to
deal with 802.11 fragments, I'll have to wait a while longer.

So what I saw:

* lots of extra latency, specially under load - if the taskqueue
wasn't immediately scheduled, things went pear shaped;

* any extra latency would result in TX ath_buf's taking their sweet time
being replenished, so any further calls to ath_transmit() would drop
mbufs.

* .. yes, there's no explicit backpressure here - things are just dropped.
Eek.

With this, the general performance has gone up, but those subtle if_start()
related race conditions are back. For some reason, this is doubly-obvious
with the AR5416 NIC and I don't quite understand why yet.

There's an unrelated issue with AR5416 performance in STA mode (it's
fine in AP mode when bridging frames, weirdly..) that requires a little
further investigation. Specifically - it works fine on a Lenovo T40
(single core CPU) running a March 2012 9-STABLE kernel, but a Lenovo T60
(dual core) running an early November 2012 kernel behaves very poorly.
The same hardware with an AR9160 or AR9280 behaves perfectly.

Methodize the process of adding the software TX queue to the taskqueue.

Move it (for now) to the TX taskqueue.

Migrate the TX sending code out from under the ath0 taskq and into
the separate ath0 TX taskq.

Whilst here, make sure that the TX software scheduler is also
running out of the TX task, rather than the ath0 taskqueue.

Make sure that the tx taskqueue is blocked/unblocked as necessary.

This allows for a little more parallelism on multi-core machines,
as well as (eventually) supporting a higher task priority for TX
tasks, allowing said TX task to preempt an already running RX or
TX completion task.

Tested:

* AR5416, AR9280 hostap and STA modes

Fix this routine to acutally break out and not set clrdmask if any
of the TIDs are currently marked as "filtered."

Migrate CLRDMASK to be a per-node flag, rather than a per-TID flag.

This is easily possible now that the TX is protected by a single
lock, rather than a per-TXQ (and thus per-TID) lock.

Only set CLRDMASK if none of the destinations are filtered.
This likely will need some tuning when it comes time to do UASPD/PS-POLL
TX, however at that point it should be manually set anyway.

Tested:

* AR9280, STA mode

TODO:

* More thorough testing in AP mode
* test other chipsets, just to be safe/sure.

Implement frame (data) transmission using if_transmit(), rather than
if_start().

This removes the overlapping data path TX from occuring, which
solves quite a number of the potential TX queue races in ath(4).
It doesn't fix the net80211 layer TX queue races and it doesn't
fix the raw TX path yet, but it's an important step towards this.

This hasn't dropped the TX performance in my testing; primarily
because now the TX path can quickly queue frames and continue
along processing.

This involves a few rather deep changes:

* Use the ath_buf as a queue placeholder for now, as we need to be
able to support queuing a list of mbufs (ie, when transmitting
fragments) and m_nextpkt can't be used here (because it's what is
joining the fragments together)

* if_transmit() now simply allocates the ath_buf and queues it to
a driver TX staging queue.

* TX is now moved into a taskqueue function.

* The TX taskqueue function now dequeues and transmits frames.

* Fragments are handled correctly here - as the current API passes
the fragment list as one mbuf list (joined with m_nextpkt) through
to the driver if_transmit().

* For the couple of places where ath_start() may be called (mostly
from net80211 when starting the VAP up again), just reimplement
it using the new enqueue and taskqueue methods.

What I don't like (about this work and the TX code in general):

* I'm using the same lock for the staging TX queue management and the
actual TX. This isn't required; I'm just being slack.

* I haven't yet moved TX to a separate taskqueue (but the taskqueue is
created); it's easy enough to do this later if necessary. I just need
to make sure it's a higher priority queue, so TX has the same
behaviour as it used to (where it would preempt existing RX..)

* I need to re-review the TX path a little more and make sure that
ieee80211_node_*() functions aren't called within the TX lock.
When queueing, I should just push failed frames into a queue and
when I'm wrapping up the TX code, unlock the TX lock and
call ieee80211_node_free() on each.

* It would be nice if I could hold the TX lock for the entire
TX and TX completion, rather than this release/re-acquire behaviour.
But that requires that I shuffle around the TX completion code
to handle actual ath_buf free and net80211 callback/free outside
of the TX lock. That's one of my next projects.

* the ic_raw_xmit() path doesn't use this yet - so it still has
sequencing problems with parallel, overlapping calls to the
data path. I'll fix this later.

Tested:

* Hostap - AR9280, AR9220
* STA - AR5212, AR9280, AR5416

There's no need to use a TXQ pointer here; we specifically need the
hardware queue ID when queuing to EDMA descriptors.

This is a small part of trying to reduce the size of ath_buf entries.

Mechanically substitute flags from historic mbuf allocator with
malloc(9) flags in sys/dev.

Don't grab the PCU lock inside the TX lock.

Delete the per-TXQ locks and replace them with a single TX lock.

I couldn't think of a way to maintain the hardware TXQ locks _and_ layer
on top of that per-TXQ software queuing and any other kind of fine-grained
locks (eg per-TID, or per-node locks.)

So for now, to facilitate some further code refactoring and development
as part of the final push to get software queue ps-poll and u-apsd handling
into this driver, just do away with them entirely.

I may eventually bring them back at some point, when it looks slightly more
architectually cleaner to do so. But as it stands at the present, it's
not really buying us much:

* in order to properly serialise things and not get bitten by scheduling
and locking interactions with things higher up in the stack, we need to
wrap the whole TX path in a long held lock. Otherwise we can end up
being pre-empted during frame handling, resulting in some out of order
frame handling between sequence number allocation and encryption handling
(ie, the seqno and the CCMP IV get out of sequence);

* .. so whilst that's the case, holding the lock for that long means that
we're acquiring and releasing the TXQ lock _inside_ that context;

* And we also acquire it per-frame during frame completion, but we currently
can't hold the lock for the duration of the TX completion as we need
to call net80211 layer things with the locks _unheld_ to avoid LOR.

* .. the other places were grab that lock are reset/flush, which don't happen
often.

My eventual aim is to change the TX path so all rejected frame transmissions
and all frame completions result in any ieee80211_free_node() calls to occur
outside of the TX lock; then I can cut back on the amount of locking that
goes on here.

There may be some LORs that occur when ieee80211_free_node() is called when
the TX queue path fails; I'll begin to address these in follow-up commits.

Until I figure out what to do here, remind myself that this needs some
rate control 'adjustment' when NOACK is set.

ALQ logging enhancements:

* upon setup, tell the alq code what the chip information is.
* add TX/RX path logging for legacy chips.
* populate the tx/rx descriptor length fields with a best-estimate.
It's overly big (96 bytes when AH_SUPPORT_AR5416 is enabled)
but it'll do for now.

Whilst I'm here, add CURVNET_RESTORE() here during probe/attach as a
partial solution to fixing crashes during attach when the attach fails.
There are other attach failures that I have to deal with; those'll come
later.

Make sure the final descriptor in an aggregate has rate control information.

This was broken by me when merging the 802.11n aggregate descriptor chain
setup with the default descriptor chain setup, in preparation for supporting
AR9380 NICs.

The corner case here is quite specific - if you queue an aggregate frame
with >1 frames in it, and the last subframe has only one descriptor making
it up, then that descriptor won't have the rate control information
copied into it. Look at what happens inside ar5416FillTxDesc() if
both firstSeg and lastSeg are set to 1.

Then when ar5416ProcTxDesc() goes to fill out ts_rate based on the
transmit index, it looks at the rate control fields in that descriptor
and dutifully sets it to be 0.

It doesn't happen for non-aggregate frames - if they have one descriptor,
the first descriptor already has rate control info.

I removed the call to ath_hal_setuplasttxdesc() when I migrated the
code to use the "new" style aggregate chain routines from the HAL.
But I missed this particular corner case.

This is a bit inefficient with MIPS boards as it involves a few redundant
writes into non-cachable memory. I'll chase that up when it matters.

Tested:

* AR9280 STA mode, TCP iperf traffic
* Rui Paulo <rpaulo@> first reported this and has verified it on
his AR9160 based AP.

PR: kern/173636

Add some debugging to try and catch an invalid TX rate (0x0) that is
being reported.

Remove this; i incorrectly committed the wrong (debug) changes in my
previous commit.

Don't call av_set_tim() if it's NULL.

This happens during a scan in STA mode; any queued data frames will
be power save queued but as there's no TIM in STA mode, it panics.

This was introduced by me when I disabled my driver-aware power save
handling support.

Add some hooks into the driver to attach, detach and record EDMA descriptor
events.

This is primarily for the TX EDMA and TX EDMA completion. I haven't yet
tied it into the EDMA RX path or the legacy TX/RX path.

Things that I don't quite like:

* Make the pointer type 'void' in ath_softc and have if_ath_alq*()
return a malloc'ed buffer. That would remove the need to include
if_ath_alq.h in if_athvar.h.
* The sysctl setup needs to be cleaned up.

Convert the aggregate descriptor path over to use the same API as
the non-aggregate path.

I "cheated" by using some TX setup code in our HAL that isn't present
in the atheros HAL (or Linux ath9k.)

The old path for forming aggregates was:

* setup the rate control in the first descriptor;
* call chaintxdesc() on all the frames;
* call setupfirsttxdesc() on the first descrpitor in the first
frame;
* call setuplasttxdesc() on the last descriptor in the last frame.

The new path for forming aggregates looks like the non-aggregate path:

* call setuptxdesc() on the first descriptor in the first frame;
* setup the rate control in the first descriptor;
* call filltxdesc() on each descriptor in the frame;
* if it's an aggregate - call set11n_aggr_{first, middle, last} as
appropriate (see the code for a description of what is "appropriate".)

Now, this is done primarily for the AR9300 HAL - it doesn't implement
the first set of aggregate functions. It just has the older methods
and the "first/middle/last" aggregate methods. So, let's convert the
code to use these.

Note: the AR5416 HAL in FreeBSD had that code (from me, a while ago)
and a previous commit brought it up to behave the same as the AR9300
HAL routines.

There's some further tidyups to be done - specifically, avoid doing
multiple calls to the 11n descriptor functions. I shouldn't call
clr11n_aggr(), then set11n_aggr_middle(), then also set11n_aggr_first().
On (at least MIPS) the TX descriptors are in non-cachable memory and
this will cause multiple slow writes.

I'll debug/tidy that up in a future commit.

Tested:

* AR9280, STA
* AR9280/AR9160, AP
* AR9380, STA (using a local, closed source HAL, sorry!)

I give up - introduce a TX lock to serialise TX operations.

I've tried serialising TX using queues and such but unfortunately
due to how this interacts with the locking going on elsewhere in the
networking stack, the TX task gets delayed, resulting in quite a
noticable throughput loss:

* baseline TCP for 2x2 11n HT40 is ~ 170mbit/sec;
* TCP for TX task in the ath taskq, with the RX also going on - 80mbit/sec;
* TCP for TX task in a separate, second taskq - 100mbit/sec.

So for now I'm going with the Linux wireless stack approach - lock tx
early. The linux code does in the wireless stack, before the 802.11
state stuff happens and before it's punted to the driver.
But TX locking needs to also occur at the driver layer as the TX
completion code _also_ begins to drain the ifnet TX queue.

Whilst I'm here, add some KTR traces for the TX path.

Note:

* This really should be done at the net80211 layer (as well, at least.)
But that'll have to wait for a little more thought to happen.

Begin fleshing out some software queue awareness for TIM handling with
the power save queue.

* introduce some new ATH_NODE lock protected fields, tracking the
net80211 psq and TIM state;
* when doing buffer transitions - ie, when sending and completing
buffers - check the state of the SWQ and update the TIM appropriately.
* when clearing the TIM bit, if the SWQ is not empty then delay clearing
it.

This is racy, but it's no less racy than the current net80211 power
save queue management code. Specifically, with multiple TX threads,
it's quite plausible that parallel state updates will race and the
TIM will be left in an inconsistent state. I'll address that in
a follow-up commit.

Since it's not immediately obvious whether the current TX path handles
fragment rate lookups correctly, add a comment describing exactly that.

The assumption in the fragment duration code is the duration of the next
fragment will match the rate used by the current fragment. But I think
a rate lookup is being done for _each_ fragment. For older pre-sample
rate control this would almost always be the case, but for sample
it may be incorrect more often then correct.

Stop abusing the ATH_TID_*() queue macros for filtered frames and give
them their own macro set.

Fix the non-TDMA build.

Migrate the TID TXQ accesses to a new set of macros, rather than reusing
the ATH_TXQ_* macros.

* Introduce the new macros;
* rename the TID queue and TID filtered frame queue so the compiler
tells me I'm using the wrong macro.

These should correspond 1:1 to the existing code.

Pause and unpause the software queues for a given node based on the
net80211 node power save state.

* Add an ATH_NODE_UNLOCK_ASSERT() check
* Add a new node field - an_is_powersave
* Pause/unpause the queue based on the node state
* Attempt to handle net80211 concurrency issues so the queue
doesn't get paused/unpaused more than once at a time from
the net80211 power save code.

Whilst here (and breaking my usual rule), set CLRDMASK when a queue
is unpaused, regardless of whether the queue has some pending traffic.
This means the first frame from that TID (now or later) will hvae
CLRDMASK set.

Also whilst here, bump the swretrymax counters whenever the
filtered frames code expires a frame. Again, breaking my rule, but
this is just a statistics thing rather than a functional change.

This doesn't fix ps-poll (but it doesn't break it too much worse
than it is at the present) or correcting the TID updates.
That's next on the list.

Tested:
* AR9220 AP (Atheros AP96 reference design)
* Macbook Pro and LG Optimus 1 Android phone, both setting
and clearing power save state (but not using PS-POLL.)

Map the non-QoS TID to the voice queue, in order to ensure important
things like EAPOL frames make it out.

After a whole bunch of hacking/testing, I discovered that they weren't
being early-dropped by the stack (but I should look at ensuring that
later..) but were even making to the hardware transmit queue.
They were mostly even being received by the remote end. However, the
remote end was completely ignoring them.

This didn't happen under 150-170MBit TCP tests as I'm guessing the TX
queue stayed very busy and the STA didn't do any scanning. However, when
doing 100Mbit/s of TCP traffic, the STA would do background scanning -
which involves it coming in and out of powersave mode with the AP.

Now, this is a total and utter hack around the real problems, which are:

* I need to implement proper power save handling and integrate it into
the filtered frames support, so the driver/stack doesn't send frames
whilst the station is actually in sleep;

* .. but frames were actually making it to the STA (macbook pro) and
the AP did receive an ACK; but a tcpdump on the receiving side showed
the EAPOL frame never made it. So the stack was dropping it for
some reason;

* Importantly - the EAPOL frames are currently going into the non-QoS
TID, which maps to the BE queue and is susceptible to that queue being
busy doing other things, but;

* There's other traffic going on in the non-QoS TID from other contexts
when scanning is going on and it's possible there's some races causing
sequence number/IV issues, but;

* Importantly importantlly, I think the interaction with TID 16 multicast
traffic in power save mode is causing issues - since I -believe- the
sequence number space being used by the EAPOL frames on TID 16 overlaps
with the multicast frames that have sequence numbers allocated and
are then stuffed on the cabq. Since with EAPOL frames being in TID 16
and queued to the BE queue, it's going to be waiting to be serviced
with all of the aggregate traffic going on - and if the CABQ gets
emptied beforehand, those TID 16 multicast frames with sequence numbers
will go out beforehand.

Now, there's quite likely a bunch of "stuff happening slightly out of
sequence" going on due to the nature of the TX path (read: lots of
overlapping and concurrent ath_start() and ath_raw_xmit() calls going
on, sigh) but I thought I had caught them all and stuffed each TID TX
behind a lock (that lasted as long as it needed to in order to get
the frame onto the relevant destination queue - thus keeping things
in order.)

Unfortunately the last problem is the big one and I'm going to stare at
it some more. If it _is_

So this is a work around for now to ensure that EAPOL frames actually
make it out before any other stuff in the non-QoS TID and HOPEFULLY
before the CABQ gets active.

I'm now going to spend a little time in the TX path figuring out exactly
why the sender is rejecting things. There's two (well, three if you count
EAPOL contents invalid) possibilities:

* The sequence number is out of order (ie, something else like the multicast
traffic on CABQ) is going out first on TID 16;
* The CCMP IV is out of order (similar to above - but less likely, as the
TX key for multicast traffic is different to unicast traffic);
* EAPOL contents strangely invalid.

AP: Ubiquiti RSPRO, AR9160/AR9220 NICs
STA: Macbook Pro, Broadcom 11n NIC

Oops - don't do the clrdmask check in ath_tx_xmit_normal() - the wrong
lock may be held.

Kim reported that the TID lock wasn't held when ath_tx_update_clrdmask()
was called. Well, the underlying hardware TXQ for that TID.

I'm betting it's the cabq stuff. ath_tx_xmit_normal() can be called
for both real and software cabq. For software cabq, the real destination
txq is different to the txq. So, the lock check will fail.

Reported by: Kim Culhan <w8hdkim@gmail.com>

Call ath_tx_tid_unsched() after the node has been flushed, so the
state can be printed correctly.

Migrate the ath(4) KTR logging to use an ATH_KTR() macro.

This should eventually be unified with ATH_DEBUG() so I can get both
from one macro; that may take some time.

Add some new probes for TX and TX completion.

Prepare for software retransmission of non-aggregate frames but ensure
it's disabled.

The previous commit to enable CLRDMASK setting didn't do it at all
correctly for non-aggregate sessions - so the CLRDMASK bit would be
cleared and never re-set.

* move ath_tx_update_clrdmask() to be called by functions that setup
descriptors and queue frames to the hardware, rather than scattered
everywhere.

* Force CLRDMASK to be set on all non-aggregate session frames being
transmitted.

* Use ath_tx_normal_comp() now on non-aggregate sessoin frames
that are queued via ath_tx_xmit_normal(). That way the TID hwq is
updated and they can trigger (eventual) filter frame queue resets
and software retransmits.

There's still a bit more work to do in this area to reverse the silly
short-sightedness on my part, however it's likely going to be better
to fix this now than just reverting the patch.

Thanks to people on the freebsd-wireless@ mailing list for promptly
pointing this out.

In (eventual) preparation for supporting disabling the whole 11n/software
retry path - add some code to make it obvious (to me!) how to disable
the software tx path.

Introduce the CLRDMASK gating based on tid->clrdmask, enabling filtered
frames to occur.

* Create a new function which will set the bf_flags CLRDMASK bit
if required.
* For raw frames, always set CLRDMASK.
* For BAR, ADDBA frames, always set CLRDMASK.
* For everything else, check if CLRDMASK needs to be set before
calling tx_setds() or tx_setds11n().
* When unpausing a queue or drain/resetting it, set tid->clrdmask=1
just to ensure traffic starts flowing.

What I need to do:

* Modify that function to _clear_ the CLRDMASK if it's not required,
or retried frames may have CLRDMASK set when they don't need to.
(Which isn't a huge deal, but..)

Whilst I'm here:

* ath_tx_normal_xmit() should really act like the AMPDU session TX
functions - any incomplete frames will end up being assigned
ath_tx_normal_comp() which will decrement tid->hwq_depth - but that
won't have been incremented.

So whilst I'm here, add a comment to do that.

* Fix the debug print function to be slightly clearer about things;
it's not a good sign when I can't interpret my own debugging output.

I've done some testing on AR9280/AR5416/AR9160 STA and AP modes.

Place the comment where it should be.

Add a work-around for some strange net80211 BAR races in the wireless
stack.

There are unfortunately quite a few odd cases in BAR TX and BAR TX
retransmission that I haven't yet fully diagnosed. So for now, add
this work-around so the resume() function isn't called too often,
decrementing pause to -1 (and causing things to stay paused.)

Oops - take a copy of ath_tx_status from the buffer before the TX processing
is done.

The aggregate path was definitely accessing 'ts' before it was actually
being assigned.

This had the side effect of over-filtering frames, since occasionally that
bit would be '1'.

Whilst here, do the same thing in the non-aggregate completion function -
as calling the filter function may also invalidate bf.

Pointy hat to: adrian, for not noticing this over many, many code reviews.

Implement my first cut at filtered frames in aggregation sessions.

The hardware can optionally "filter" frames if successive transmissions
to a given node (ie, "entry in the keycache") fail. That way the hardware
can implement a kind of early abort of all the other frames queued to
that destination, rather than simply trying to TX each frame to that
destination (and failing.)

The background:

* If a frame comes back as being filtered, the hardware didn't try to
TX it (or it was outside the TX burst opportunity.) So, take it as a hint
that some (but not all, see below) frames to the destination may be
filtered.

* If the CLRDMASK bit is set in a TX descriptor, the "filter to this
destination" bit in the keycache entry is cleared and TX to that host
will be unconditionally retried.

* Right now everything has the CLRDMASK bit set, so filtered frames
tend to be aggregates and frames that fall outside of the WME burst
window. It was a bit worse in the past as I had messed up the TX
flags and CLRDMASK wasn't being set on aggregate frames.

The annoying bits:

* It's easy (ish) to do for aggregate session frames - firstly, they
can be retried in any order as long as they're within the BAW, and
there's already a bunch of infrastructure tracking how many frames
the TID has queued to the hardware (tid->hwq_depth.) However, for
frames that bypassed the software queue, hwq_depth doesn't get
incremented. I'll fix that in a subsequent commit.

* For non-aggregate session frames, the only retries that can occur
are ones for sequence numbers that hvaen't successfully been TXed yet.
Since there's no re-ordering going on in non-aggregate sessions, if any
subsequent seqno frames make it out, any filtered frames before that
seqno need to be dropped.

Hence why this initially is just for aggregate session frames.

* Since there may be intermediary frames to the destination that
have CLRDMASK set - for example, any directly dispatched management
frames to that destination - it's possible that there will be some
filtered frames followed up by some non filtered frames. Thus,
it can't be assumed that once you see a filtered frame for the given
destination node, all subsequent frames for all TIDs will be filtered.

Ok, with that in mind:

* Create a per-TID filtered frame queue for frames that the hardware
returns as filtered.

* Track filtered frames per-tid, rather than per-node. It just makes
the locking much easier.

* When a filtered frame appears in the completion function, the node
transitions to "filtered", and all subsequent completed error frames
(filtered or otherwise) are put on the filtered frame queue. The TID
is paused once (during the transition from non-filtered to filtered).

* If a filtered frame retry count exceeds SWMAX_RETRIES, a BAR should be
sent.

* Once all the frames queued to the hardware for the given filtered frame
TID, transition back from filtered frame to non-filtered frame, which
means pre-pending all the filtered frames onto the head of the software
queue, clearing the filtered frame state and unpausing the TID.

Things get quite hairy around handling completion (aggr, non-aggr, norm,
direct-dispatched frames to a hardware queue); whether it's an "error",
"cleanup" or "BAR" state as well as filtered, which order to do things
in (eg do filtered BEFORE checking for BAR, as the filter completion
may be needed to actually transmit a BAR frame.)

This work has definitely reminded me that I have to tidy up all the locking
and remove some of the ridiculous lock/unlock/lock/unlock going on in the
completion functions.

It's also reminded me that I should really split out TID versus hardware TXQ
locking, even if the underlying locking is still the destination hardware TXQ.

Finally, this is all pre-requisite for working on AP mode power save support
(PS-POLL, uAPSD) as well as improving performance to misbehaving nodes (as
they can transition into filter mode, stopping any TX until everything has
caught up.)

Finally (ish) - this should also be done for non-aggregate sessions as
there are still plenty of laptops and mobile devices that don't speak
802.11n but do wish for stable, useful power save AP support where packets
aren't simply dropped. This requires software retransmission for
non-aggregate sessions to be implemented, which includes the caveats I've
mentioned above.

Finally finally - this doesn't yet do anything about the CLRDMASK bit in the
TX descriptor. That's still unconditionally set to 1. I'll debug the
current work (mostly ensuring I haven't busted up the hairy transitions
between BAR, filtered, error (all frames in an aggregate failing) and
cleanup (when transitioning from aggregation -> non-aggregation.))

Finally finally finally - this is all original work by yours truely, rather
than ported from the Atheros internal driver codebase or Linux ath9k.

Tested:
* AR9280, AR5416 in STA mode
* AR9280, AR9130 in hostap mode
* Lots and lots of iperf testing in very marginal and non-marginal conditions,
complete with inducing filtered frames + BAR TX conditions.

Take credit for the work I've done in this source file.

Clear the correct descriptor when going through the chained together
gather DMA descriptor list.

Pointy hat to: adrian@, for even USING bf->bf_desc here instead of 'ds'.

Make sure the aggregate fields are properly cleared - both in the
ath_buf and when forming a non-aggregate frame.

The non-11n setds function is called when TXing aggregate frames (and
yes, I should fix this!) and the non-11n TX aggregation code doesn't clear
the delimiter field. I figure it's nicer to do that.

Ensure that single-frame aggregate session frames are retransmitted
with the correct configuration.

Occasionally an aggregate TX would fail and the first frame would be
retransmitted as a non-AMPDU frame. Since bfs_aggr=1 and bfs_nframes > 1
(from the previous AMPDU attempt), the aggr completion function would be
called and be very confused about what's going on.

Noticed by: Kim <w8hdkim@gmail.com>
PR: kern/171394

Fix a build issue when ATH_DEBUG isn't defined - just initialise and use
qnum.

Wrap debugging in #ifdef ATH_DEBUG

Advance the descriptor pointer by sc->sc_tx_desclen bytes, rather than
sizeof(struct ath_desc). This isn't correct for EDMA TX descriptors.

This popped up during iperf tests. Ping tests never created frames that
had enough segments to overflow into a second descriptor. However,
an iperf TCP test would do that after a few seconds; the second descriptor
would almost always certainly have garbage.

Tested:

* AR9380, STA mode
* AR9280, STA mode (802.11n TX, legacy TX)

When assembling the descriptor list, make sure that the "first" descriptor
is marked correctly.

The existing logic assumed that the first descriptor is i == 0, which
doesn't hold for EDMA TX. In this instance, the first time filltxdesc()
is called can be up to i == 3.

So for a two-buffer descriptor:

* firstSeg is set to 0;
* lastSeg is set to 1;
* the ath_hal_filltxdesc() code will treat it as the last segment in
a descriptor chain and blank some of the descriptor fields, causing
the TX to stop.

When firstSeg is set to 1 (regardless of lastSeg), it overrides the
lastSeg setting. Thus, ath_hal_filltxdesc() won't blank out these
fields.

Tested: AR9380, STA mode. With this, association is successful.

Extend the non-aggregate TX descriptor chain routine to be aware of:

* the descriptor ID, and
* the multi-buffer support that the EDMA chips support.

This is required for successful MAC transmission of multi-descriptor
frames. The MAC simply hangs if there are NULL buffers + 0 length pointers,
but the descriptor did have TxMore set.

This won't be done for the 11n aggregate path, as that will be modified
to use the newer API (ie, ath_hal_filltxdesc() and then set first|middle|
last_aggr), which will deprecate some of the current code.

TODO:

* Populate the numTxMaps field in the HAL, then make sure that's fetched
by the driver. Then I can undo that hack.

Tested:

* AR9380, AP mode, TX'ing non-aggregate 802.11n frames;
* AR9280, STA/AP mode, doing aggregate and non-aggregate traffic.

Revert the ath_tx_draintxq() method, and instead teach it the minimum
necessary to "do" EDMA.

It was just using the TX completion status for logging information about
the descriptor completion. Since with EDMA we don't know this without
checking the TX completion FIFO, we can't provide this information.
So don't.

Break out ath_draintxq() into a method and un-methodize ath_tx_processq().

Now that I understand what's going on with this, I've realised that
it's going to be quite difficult to implement a processq method in
the EDMA case. Because there's a separate TX status FIFO, I can't
just run processq() on each EDMA TXQ to see what's finished.
i have to actually run the TX status queue and handle individual
TXQs.

So:

* unmethodize ath_tx_processq();
* leave ath_tx_draintxq() as a method, as it only uses the completion status
for debugging rather than actively completing the frames (ie, all frames
here are failed);
* Methodize ath_draintxq().

The EDMA ath_draintxq() will have to take care of running the TX
completion FIFO before (potentially) freeing frames in the queue.

The only two places where ath_tx_draintxq() (on a single TXQ) are used:

* ath_draintxq(); and
* the CABQ handling in the beacon setup code - it drains the CABQ before
populating the CABQ with frames for a new beacon (when doing multi-VAP
operation.)

So it's quite possible that once I methodize the CABQ and beacon handling,
I can just drop ath_tx_draintxq() in its entirety.

Finally, it's also quite possible that I can remove ath_tx_draintxq()
in the future and just "teach" it to not check the status when doing
EDMA.

Add the AR9300 HAL ID in to the 11n check routine.

I was having TX hang issues, which I root caused to having the
legacy ath_hal_setupxtxdesc() called, rather than the 11n rate scenario
setup code. This meant that rate control information wasn't being
put into frames, causing the MAC to stall/hang.

Correct re-initialise the link pointer to be the final descriptor in
the last buffer.

This fixes traffic stalls that were occuring with stuck beacon events.

PR: kern/170433

Migrate the 802.11n ath_hal_chaintxdesc() API to use a buffer/segment
array, similar to what filltxdesc() uses.

This removes the last reference to ds_data in the TX path outside of
debugging statements. These need to be adjusted/fixed.

Tested:

* AR9280 STA/AP with iperf TCP traffic

Migrate the ath_hal_filltxdesc() API to take a list of buffer/seglen values.

The existing API only exposes 'seglen' (the current buffer (segment) length)
with the data buffer pointer set in 'ds_data'. This is fine for the legacy
DMA engine but it won't work for the EDMA engines.

The EDMA engine has a significantly different TX descriptor layout.

* The legacy DMA engine had a ds_data pointer at the same offset in the
descriptor for both TX and RX buffers;
* The EDMA engine has no ds_data for RX - the data is DMAed after the
descriptor;
* The EDMA engine has support for 4 TX buffer/segment pairs in the TX
DMA descriptor;
* The EDMA TX completion is in a different FIFO, and the driver will
'link' the status completion entry to a QCU by a "QCU ID".
I don't know why it's just not filled in by the hardware, alas.

So given that, here are the changes:

* Instead of directly fondling 'ds_data' in ath_desc, change the
ath_hal_filltxdesc() to take an array of buffer pointers as well
as segment len pointers;
* The EDMA TX completion status wants a descriptor and queue id.
This (for now) uses bf_state.bfs_txq and will extract the hardware QCU
ID from that.
* .. and this is ugly and wasteful; it should change to just store
the QCU in the bf_state and save 3/7 bytes in the process.

Now, the weird crap:

* The aggregate TX path was using bf_state->bfs_txq for the TXQ, rather than
taking a function argument. I've tidied that up.
* The multicast queue frames get put on a software TXQ and then that is
appended to the hardware CABQ when appropriate. So for now, make sure
that bf_state->bfs_txq points at the CABQ when adding frames to the
multicast queue.
* .. but the multicast queue TX path for now doesn't use the software
queue and instead
(a) directly sets up the descriptor contents at that point;
(b) the frames on the vap->avp_mcastq are then just appended wholesale
to the CABQ.
So for now, I don't have to worry about making the multicast path
work with aggregation or the per-TID software queue. Phew.

What's left to do:

* I need to modify the 11n ath_hal_chaintxdesc() API to do the same.
I'll do that in a subsequent commit.
* Remove bf_state.bfs_txq entirely and store the QCU as appropriate.
* .. then do the runtime "is this going on the right HWQ?" checks using
that, rather than comparing pointer values.

Tested on:

* AR9280 STA/AP
* AR5416 STA/AP

Fix an issue that crept in with the previous descriptor tidyup.

When forming aggregates, the last descriptor was now not being
correctly setup - instead, the "setuplasttxdesc" call was being
handed the first descriptor in the last subframe, rather than the
last descriptor in the last subframe.

This showed up as "bad series0 hwrate" messages, as the final
descriptor just didn't have any of the rate control information
squirreled away.

Tested:
* AR9280 STA -> 11n AP, iperf TCP

Allow 802.11n hardware to support multi-rate retry when RTS/CTS is
enabled.

The legacy (pre-802.11n) hardware doesn't support this - although
the AR5212 era hardware supports MRR, it doesn't have all the bits
needed to support MRR + RTS/CTS. The AR5416 and later support
a packet duration and RTS/CTS flags per rate scenario, so we should
support it.

Tested:

* AR9280, STA

PR: kern/170302

Shuffle the call to ath_hal_setuplasttxdesc() to _after_ the rate control
code is called and remove it from ath_buf_set_rate().

For the legacy (non-11n API) TX routines, ath_hal_filltxdesc() takes care
of setting up the intermediary and final descriptors right, complete
with copying the rate control info into the final descriptor so the
rate modules can grab it.

The 11n version doesn't do this - ath_hal_chaintxdesc() doesn't
copy the rate control bits over, nor does it clear isaggr/moreaggr/
pad delimiters. So the call to setuplasttxdesc() is needed here.

So:

* legacy NICs - never call the 11n rate control stuff, so filltxdesc
copies the rate control info right;
* 11n NICs transmitting legacy or 11n non-aggregate frames -
ath_hal_set11nratescenario() is called to setup rate control and
then ath_hal_filltxdesc() chains them together - so the rate control
info is right;
* 11n aggregate frames - set11nratescenario() is called, then
ath_hal_chaintxdesc() is called to chain a list of aggregate and subframes
together. This requires a call to ath_hal_setuplasttxdesc() to complete
things.

Tested:

* AR9280 in station mode

TODO:

* I really should make sure that the descriptor contents get blanked
out correctly or garbage left over from aggregate frames may show
up in non-aggregate frames, leading to badness.

Push the rate control and descriptor chaining into the descriptor "set"
functions, for both legacy and 802.11n.

This will simplify supporting the EDMA chipsets as these two descriptor
setup functions can just be overridden in their entirety, hiding all of
the subtle differences in setting things up.

It's not a permanent solution, as eventually the AR5416 HAL should grow
similar versions of the 11n descriptor functions and then those can be
used.

TODO:

* Push the "clr11naggr" call into the legacy setds, just to ensure
that retried frames don't end up with the aggregate bits set
inappropriately;
* Remove the "setlasttxdesc" call from the 11n TX path and push it
into setds_11n.
* Ensure that setds_11n will work correctly for non-aggregate frames;
* .. and then when it does, just unconditionally call "setds_11n" for
11n NICs and "setds" for non-11n NICs.

Migrate some more TX side setup routines to be methods.

Break out the hardware handoff and TX DMA restart code into methods.

These (and a few others) will differ based on the underlying DMA
implementation.

For the EDMA NICs, simply stub them out in a fashion which will let
me focus on implementing the necessary descriptor API changes.

Shuffle the rate control call to be consistent with non-aggregate TX.

The correct ordering for non-aggregate TX is:

* call ath_hal_setuptxdesc() to setup the first TX descriptor complete
with the first TX rate/try count;
* call ath_hal_setupxtxdesc() to setup the multi-rate retry;
* .. or for 802.11n NICs, call ath_hal_set11nratescenario() for MRR and
802.11n flags;
* then call ath_hal_filltxdesc() to setup intermediary descriptors
in a multi-descriptor single frame.

The call to ath_hal_filltxdesc() routines seem to correctly (consistently?)
handle the intermediary descriptor flags, including copying the rate
control information to the final descriptor in the frame. That's used
by the rate control module rather than the hardware.

Tested:

* Only on AR9280 STA mode, however it should work on other chips in
both STA and AP mode.

Modify ath_descdma_setup() to take a descriptor size parameter.

The AR9300 and later descriptors are 128 bytes, however I'd like to make
sure that isn't used for earlier chips.

* Populate the TX descriptor length field in the softc with
sizeof(ath_desc)

* Use this field when allocating the TX descriptors

* Pre-AR93xx TX/RX descriptors will use the ath_desc size; newer ones will
query the HAL for these sizes.

Begin separating out the TX DMA setup in preparation for TX EDMA support.

* Introduce TX DMA setup/teardown methods, mirroring what's done in
the RX path.

Although the TX DMA descriptor is setup via ath_desc_alloc() /
ath_desc_free(), there TX status descriptor ring will be allocated
in this path.

* Remove some of the TX EDMA capability probing from the RX path and
push it into the new TX EDMA path.

Begin modifying the descriptor allocation functions to support a variable
sized TX descriptor.

This is required for the AR93xx EDMA support which requires 128 byte
TX descriptors (which is significantly larger than the earlier
hardware.)

Convert the TX path to use the new HAL methods for accessing the
TX descriptor link pointers.

This is required for the AR93xx and later chipsets.

The RX path is slightly different - the legacy RX path directly
accesses ath_desc->ds_link for now, however this isn't at all done
for EDMA (FIFO) RX.

Now, for those performing a little software archeology here:

This is all a bit sub-optimal. "struct ath_desc" is only really relevant
for the pre-AR93xx NICs - where ds_link and ds_data is always in the
same location.

The AR93xx and later NICs have different descriptor layouts altogether.

Now, for AR93xx and later NICs, you should never directly reference
ds_link and ds_data, as:

* the RX descriptors don't have either - the data is _after_ the RX
descriptor. They're just one large buffer. There's also no need for
a per-descriptor RX buffer size as they're all fixed sizes.

* the TX descriptors have 4 buffer and 4 length fields _and_ a link
pointer. Each frame takes up one TX FIFO pointer, but it can contain
multiple subframes (either multiple frames in a buffer, and/or
multiple frames in an aggregate/RIFS burst.)

* .. so, when TX frames are queued to a hardware queue, the link
pointer is ONLY for buffers in that frame/aggregate. The next frame
starts in a new FIFO pointer.

* Finally, descriptor completion status is in a different ring.
I'll write something up about that when its time to do so.

This was inspired by Linux ath9k and the reference driver but is a
reimplementation.

Obtained from: Linux ath9k, Qualcomm Atheros

Fix build when ATH_DEBUG is not defined.

Print the TX buffer if this error condition is asserted.

I need to figure out why this is occuring. Hopefully I can get enough
descriptor dumps to figure it out.

Make sure the BAR TX session pause is correctly unpaused when a node
is reassociating.

PR: kern/169432

Disable this warning debug for now, as I'm now aware of the particular
situation where it's occuring.

Whilst I'm here, flesh out a more descriptive description.

Implement a separate, smaller pool of ath_buf entries for use by management
traffic.

* Create sc_mgmt_txbuf and sc_mgmt_txdesc, initialise/free them appropriately.
* Create an enum to represent buffer types in the API.
* Extend ath_getbuf() and _ath_getbuf_locked() to take the above enum.
* Right now anything sent via ic_raw_xmit() allocates via ATH_BUFTYPE_MGMT.
This may not be very useful.
* Add ATH_BUF_MGMT flag (ath_buf.bf_flags) which indicates the current buffer
is a mgmt buffer and should go back onto the mgmt free list.
* Extend 'txagg' to include debugging output for both normal and mgmt txbufs.
* When checking/clearing ATH_BUF_BUSY, do it on both TX pools.

Tested:

* STA mode, with heavy UDP injection via iperf. This filled the TX queue
however BARs were still going out successfully.

TODO:

* Initialise the mgmt buffers with ATH_BUF_MGMT and then ensure the right
type is being allocated and freed on the appropriate list. That'd save
a write operation (to bf->bf_flags) on each buffer alloc/free.

* Test on AP mode, ensure that BAR TX and probe responses go out nicely
when the main TX queue is filled (eg with paused traffic to a TID,
awaiting a BAR to complete.)

PR: kern/168170

Oops, return the newly allocated buffer to the queue, not the completed
buffer.

PR: kern/168170

Replace the direct sc_txbuf manipulation with a pair of functions.

This is preparation work for having a separate ath_buf queue for
management traffic.

PR: kern/168170

Fix uninitialised reference.

Noticed by: John Hay <jhay@meraka.org.za>

Wrap the whole (software) TX path from ifnet dequeue to software queue
(or direct dispatch) behind the TXQ lock (which, remember, is doubling
as the TID lock too for now.)

This ensures that:

(a) the sequence number and the CCMP PN allocation is done together;
(b) overlapping transmit paths don't interleave frames, so we don't
end up with the original issue that triggered kern/166190.

Ie, that we don't end up with seqno A, B in thread 1, C, D in
thread 2, and they being queued to the software queue as "A C D B"
or similar, leading to the BAW stalls.

This has been tested:

* both STA and AP modes with INVARIANTS and WITNESS;
* TCP and UDP TX;
* both STA->AP and AP->STA.

STA is a Routerstation Pro (single CPU MIPS) and the AP is a dual-core
Centrino.

PR: kern/166190

Add another TID lock.

Make sure the frames are queued to the head of the list, not the tail.
See previous commit.

PR: kern/166190

When scheduling frames in an aggregate session, the frames should be
scheduled from the head of the software queue rather than trying to
queue the newly given frame.

This leads to some rather unfortunate out of order (but still valid
as it's inside the BAW) frame TX.

This now:

* Always queues the frame at the end of the software queue;
* Tries to direct dispatch the frame at the head of the software queue,
to try and fill up the hardware queue.

TODO:

* I should likely try to queue as many frames to the hardware as I can
at this point, rather than doing one at a time;
* ath_tx_xmit_aggr() may fail and this code assumes that it'll schedule
the TID. Otherwise TX may stall.

PR: kern/166190

Retried frames need to be inserted in the head of the list, not the tail.

This is an unfortunate byproduct of how the routine is used - it's called
with the head frame on the queue, but if the frame is failed, it's inserted
into the tail of the queue.

Because of this, the sequence numbers would get all shuffled around and
the BAW would be bumped past this sequence number, that's now at the
end of the software queue. Then, whenever it's time for that frame
to be transmitted, it'll be immediately outside of the BAW and TX will
stall until the BAW catches up.

It can also result in all kinds of weird duplicate BAW frames, leading
to hilarious panics.

PR: kern/166190

Finish undoing the previous commit - this part of the code is no longer
required.

PR: kern/166190

Introduce a new lock debug which is specifically for making sure the
_TID_ lock is held.

For now the TID lock is also the TXQ lock. This is just to make sure
that the right TXQ lock is held for the given TID.

Revert r233227 and followup commits as it breaks CCMP PN replay detection.

This showed up when doing heavy UDP throughput on SMP machines.

The problem with this is because the 802.11 sequence number is being
allocated separately to the CCMP PN replay number (which is assigned
during ieee80211_crypto_encap()).

Under significant throughput (200+ MBps) the TX path would be stressed
enough that frame TX/retry would force sequence number and PN allocation
to be out of order. So once the frames were reordered via 802.11 seqnos,
the CCMP PN would be far out of order, causing most frames to be discarded
by the receiver.

I've fixed this in some local work by being forced to:

(a) deal with the issues that lead to the parallel TX causing out of
order sequence numbers in the first place;
(b) fix all the packet queuing issues which lead to strange (but mostly
valid) TX.

I'll begin fixing these in a subsequent commit or five.

PR: kern/166190

Avoid using hard-coded numbers here.

Fix up some corner cases with aggregation handling.

I've come across a weird scenario in net80211 where two TX streams will
happily attempt to setup an aggregation session together.
If we're very lucky, it happens concurrently on separate CPUs and the
total lack of locking in the net80211 aggregation code causes this stuff
to race. Badly.

So >1 call would occur to the ath(4) addba start, but only one call would
complete to addba complete or timeout. The TID would thus stay paused.

The real fix is to implement some proper per-node (or maybe per-TID)
locking in net80211, which then could be leveraged by the ath(4) TX
aggregation code.

Whilst I'm at it, shuffle around the debugging messages a bit.
I like to keep people on their toes.

For now, add a quick debugging patch to log when the hw TXQ != the TID/AC.

Rename ath_tx_cleanup() -> ath_tx_tid_cleanup() in order to not clash
with a symbol in if_ath.c

Migrate the bulk of the RX routines out from if_ath.c to if_ath_rx.[ch].

* migrate the rx processing out into if_ath_rx.c
* migrate the TSF functions into if_ath_tsf.h, as inlines

This is in prepration for supporting the EDMA RX routines, required to
support the AR93xx series NICs.

TODO:

* ath_start() shouldn't be private, but it's called as part of
the RX path. I should likely migrate ath_rx_tasklet() back into
if_ath.c and then return this to be 'static'. The RX code really
shouldn't need to see TX routines (and vice versa.)

* ath_beacon_* should be in if_ath_beacon.[ch].

* ath_tdma_* should be in if_ath_tdma.[ch] ...

Migrate ath_debug and sc_debug from an int to a uint64_t / QUAD;
add some more BAR debugging logic.

* Change the definition of ath_debug and ath_softc.sc_debug from
int to uint64_t;
* Change the relevant sysctls;
* Add a new BAR TX debugging field;
* Use this in if_ath_tx.

This has been tested by using the sysctl program, which happily allows
for fields > 32 bits to be configured.

Handle non-xretry errors the same as xretry errors for now.

Although I _should_ handle the other errors in various ways (specifically
errors like FILT), treating them as having transmitted successfully
is completely wrong. Here, they'd be counted as successful and the BAW
would be advanced.. but the RX side wouldn't have received them.

The specific errors I've been seeing here are HAL_TXERR_FILT.

This patch does fix the issue - I've tested it using -i 0.001 pings
(enough to start aggregation) and now the behaviour is correct:

* The RX side never sees a "moved window" error, and
* The TX side sends BARs as needed, with the RX side correctly handling
them.

PR: kern/167902

Fix a couple of sc_ac2q[] mappings that were using the TID, not the AC.

PR: kern/167588

Remove some of the redundant locking done in the TX completion path,
when checking whether BAR frames need to be checked.

Migrate the net80211 TX aggregation state to be from per-AC to per-TID.

TODO:

* Test mwl(4) more thoroughly!

Reviewed by: bschmidt (for iwn)

After reviewing the mcast/sleep station code a little, undo some brain
damage which I committed when I had less clue about such things.

Don't ever put normal data frames on the mcast software queue.
Just put mcast frames there if needed.

Pass the txq decision into ath_tx_normal_setup(), as we've already made
the decision. Don't re-do it.

Whilst i'm here, add another random debugging statement.

Do a dma sync before the descriptors are chained together.

I need to find a better place to do this..

Break out the legacy duration and protection code into routines,
call these after rate control selection is done.

The duration/protection code wasn't working - it expected the rix to
be valid. Unfortunately after I moved the rate control selection into
late in the process, the rix value isn't valid and thus the protection/
duration code would get things wrong.

HT frames are now correctly protected with an RTS and for the AR5416,
this involves having the aggregate frames be limited to 8K.

TODO:

* Fix up the DMA sync to occur just before the frame is queued to the
hardware. I'm adjusting the duration here but not doing the DMA
flush.

* Doubly/triply ensure that the aggregate frames are being limited to
the correct size, or the AR5416 will get unhappy when TXing RTS-protected
aggregates.

Remove duplicate txflags field from ath_buf.

rename bf_state.bfs_flags to bf_state.bfs_txflags, as that is what
it effectively is.

Implement BAR TX.

A BAR frame must be transmitted when an frame in an A-MPDU session fails
to transmit - it's retried too often, or it can't be cloned for
re-transmission. The BAR frame tells the remote side to advance the
left edge of the block-ack window (BAW) to a new value.

In order to do this:

* TX for that particular node/TID must be paused;
* The existing frames in the hardware queue needs to be completed, whether
they're TXed successfully or otherwise;
* The new left edge of the BAW is then communicated to the remote side
via a BAR frame;
* Once the BAR frame has been sucessfully TXed, aggregation can resume;
* If the BAR frame can't be successfully TXed, the aggregation session
is torn down.

This is a first pass that implements the above. What needs to be done/
tested:

* What happens during say, a channel reset / stuck beacon _and_ BAR
TX. It _should_ be correctly buffered and retried once the
reset has completed. But if a bgscan occurs (and they shouldn't,
grr) the BAR frame will be forcibly failed and the aggregation session
will be torn down.

Yes, another reason to disable bgscan until I've figured this out.

* There's way too much locking going on here. I'm going to do a couple
of further passes of sanitising and refactoring so the (re) locking
isn't so heavy. Right now I'm going for correctness, not speed.

* The BAR TX can fail if the hardware TX queue is full. Since there's
no "free" space kept for management frames, a full TX queue (from eg
an iperf test) can race with your ability to allocate ath_buf/mbufs
and cause issues. I'll knock this on the head with a subsequent
commit.

* I need to do some _much_ more thorough testing in hostap mode to ensure
that many concurrent traffic streams to different end nodes are correctly
handled. I'll find and squish whichever bugs show up here.

But, this is an important step to being able to flip on 802.11n by default.
The last issue (besides bug fixes, of course) is HT frame protection and
I'll address that in a subsequent commit.

Now that I've fixed the BAW TX hangs, disable this verbose debugging
again.

Correctly handle AR_MoreAggr when assembling multi-descriptor final frames.

Linux ath9k doesn't have this issue as it doesn't try queuing multi-
descriptor frames to the hardware.

Before, I was only setting the first and last descriptor in the final
frame correctly - and that was done by accident. The first descriptor in
the last sub-frame was being correctly updated by ath_tx_setds_11n();
the last descriptor in the last sub-frame was being correctly updated
by ath_buf_set_rate(). But both of those are "incorrect".

The correct behaviour is:

* AR_IsAggr is set for all descriptors for all subframes in an aggregate.
* AR_MoreAggr is set for all descriptors for all non-final sub-frames
in an aggregate.

Ie, all descriptors in the last sub-frame of an aggregate must have this
field set to 0.

I still need to do a couple of extra passes to ensure the pad delimiter
field is being correctly handled in all descriptors in the last sub-frame.

Defer the rescheduling of TID -> TXQ frames in some instances.

Right now ath_txq_sched() is mainly called from the TX ath_tx_processq()
routine, which is (mostly) done as part of the taskqueue. It shouldn't
be called outside the taskqueue.

But now that I'm about to flip back on BAR TX, I'm going to start
stressing the ath_tx_tid_pause() and ath_tx_tid_resume() paths.
What I don't want to have happen is a reschedule of the TID traffic
_during_ the completion of TX frames.

Ideally I'd like to have a way to flag back up to the processing code
that the current hardware queue should be rechecked for software TID
queue frames. But for now, this should suffice for the BAR TX case.

I may eventually delete this code once I've brought some further
sanity to the general TX queue/completion path.

Use the assigned sequence number when checking if a retried packet is
within the BAW.

This regression was introduced in ane earlier commit by me to fix the
BAW seqno allocation-but-not-insertion-into-BAW race. Since it was only
ever using the to-be allocated sequence number, any frame retries
with the first frame in the BAW still in the software queue would
have constantly failed, as ni_txseqs[tid] would always be outside
the BAW.

TODO:

* Extract out the mostly common code here in the agg and non-agg ADDBA
case and stuff it into a single function.

PR: kern/166357

Add some more debugging to try and nail down exactly what's going on when
I see traffic stalls.

It turns out that the bug isn't because the first and last frame in the
BAW is in the software queue. It is more likely that it's because
the first frame in the BAW is still in the software queue and thus there's
no more room to allocate and do subsequent TX.

PR: kern/166357

Add some further debugging to try and aid tracking down what the state of
things were just before a full software queue is drained.

Delay sequence number allocation for A-MPDU until just before the frame
is queued to the hardware.

Because multiple concurrent paths can execute ath_start(), multiple
concurrent paths can push frames into the software/hardware TX queue
and since preemption/interrupting can occur, there's the possibility
that a gap in time will occur between allocating the sequence number
and queuing it to the hardware.

Because of this, it's possible that a thread will have allocated a
sequence number and then be preempted by another thread doing the same.
If the second thread sneaks the frame into the BAW, the (earlier) sequence
number of the first frame will be now outside the BAW and will result
in the frame being constantly re-added to the tail of the queue.
There it will live until the sequence numbers cycle around again.

This also creates a hole in the RX BAW tracking which can also cause
issues.

This patch delays the sequence number allocation to occur only just before
the frame is going to be added to the BAW. I've been wanting to do this
anyway as part of a general code tidyup but I've not gotten around to it.
This fixes the PR.

However, it still makes it quite difficult to try and ensure in-order
queuing and dequeuing of frames. Since multiple copies of ath_start()
can be run at the same time (eg one TXing process thread, one TX completion
task/one RX task) the driver may end up having frames dequeued and pushed
into the hardware slightly/occasionally out of order.

And, to make matters more annoying, net80211 may have the same behaviour -
in the non-aggregation case, the TX code allocates sequence numbers
before it's thrown to the driver. I'll open another PR to investigate
this and potentially introduce some kind of final-pass TX serialisation
before frames are thrown to the hardware. It's also very likely worthwhile
adding some debugging code into ath(4) and net80211 to catch when/if this
does occur.

PR: kern/166190

Fix a panic introduced in a previous commit - non-beaconing modes (eg STA)
don't setup the avp mcast queue.

This is a bit annoying though - it turns out the mcast queue isn't
initialised for STA mode but it's then touched to see whether anything
is in it. That should be fixed in a subsequent commit.

Noticed by: gperez@entel.upc.edu
PR: kern/165895

Don't flood the cabq/mcastq with frames.

In a very noisy 2.4GHz environment (with HT/40 enabled, making it worse)
I saw the following occur:

* the air was considered "busy" a lot of the time;
* the cabq time is quite short due to staggered beacons being enabled;
* it just wasn't able to keep up TX'ing CABQ frames;
* .. and the cabq would swallow up all the TX ath_buf's.

This patch introduces a twiddle which allows the maximum cabq depth to be
set, forcing further frames to be dropped.

It defaults to the TX buffer count at the moment, so the default behaviour
isn't changed.

I've also started fleshing out a similar setup for the data path, so
it doesn't swallow up all the available TX buffers and preventing management
frames (such as ADDBA) out.

PR: kern/165895

Document that we may end up with some suboptimal handling of data
frames with stations in power saving mode.

I'm not (yet) sure how to handle TX'ing aggregates frames to stations
that are in power saving mode, or whether that's even a feasible thing
to do. So in order to (mostly) not forget, leave a couple of comments
in the code.

The code presently assumes that the aggregation TID state for an ath_node
is locked not by the ath_node lock or a node+TID lock, but behind the
hardware queue said TID maps to. This assumption is going to be
incorrect for stations in power saving mode as we'll be TX'ing frames
on the multicast queue.

In any case, I'm afraid its a "later problem". :/

Correctly initialise the TXQ link pointer to the last descriptor in
the last buffer in the list.

The current behaviour (due to me, so pointy hat is firmly on my head here)
was incorrect - it was setting the link pointer to the last descriptor
of the _first_ buffer in the TXQ. Instead, it should have set it to the
last descriptor in the _last_ buffer in the TXQ.

This showed up as occasional TX stalls with frames in the TXQ but no
TX progress being made. Further inspection showed the TXQ looked like
it contained multiple "lists" of frames - there'd be a list of correct
frames, then a NULL link pointer, but there'd be a next buffer in the
list.

Since this code is only called upon an interface reset, it's likely
this only began showing up when I started doing stress testing
in environments which annoy the radios enough to cause lockups.

I've not yet any TX stalls with this patch applied.

PR: kern/165866

style(9) changes. This shouldn't change functionality.

If frames are dumped out of the queue, let's at least see what they are.

This shows that the majority of the weird traffic I see here are probe
frames that haven't been sent out, but I can also trigger this condition
by doing ICMP w/ -i 0.3 - enough to trigger the TX during actual scanning,
but not fast enough to stop scanning from occuring.

PR: kern/163689

Flesh out the TX aggregation completion statistics.

* Failall is now named just that.
* Add TX ok and TX fail, for aggregate frame sub-frames.

This will break athstats; a followup commit wil resolve this.

Sponsored by: Hobnob, Inc.

Flesh out some slightly dirty reset/channel change serialisation code
for the ath(4) driver.

Currently, there's nothing stopping reset, channel change and general
TX/RX from overlapping with each other. This wasn't a big deal with
pre-11n traffic as it just results in some dropped frames.
It's possible this may have also caused some inconsistencies and
badly-setup hardware.

Since locks can't be held across all of this (the Linux solution)
due to LORs with the network stack locks, some state counter
variables are used to track what parts of the code the driver is
currently in.

When the hardware is being reset, it disables the taskqueue and
waits for pending interrupts, tx, rx and tx completion before
it begins the reset or channel change.

TX and RX both abort if called during an active reset or channel
change.

Finally, the reset path now doesn't flush frames if ATH_RESET_NOLOSS
is set. Instead, completed TX and RX frames are passed back up to
net80211 before the reset occurs.

This is not without problems:

* Raw frame xmit are just dropped, rather than placed on a queue.
The net80211 stack should be the one which queues these frames
rather than the driver.

* It's all very messy. It'd be better if these hardware operations
were serialised on some kind of work queue, rather than hoping
they can be run in parallel.

* The taskqueue block/unblock may occur in parallel with the
newstate() function - which shuts down the taskqueue and restarts
it once the new state is known. It's likely these operations should
be refcounted so the taskqueue is restored once no other areas
in the code wish to suspend operations.

* .. interrupt disable/enable should likely be refcounted as well.

With this work, the driver does not drop frames during stuck beacon
or fatal errors and thus 11n traffic continues to run correctly.
Default and full resets however do still drop frames and it's possible
this may occur, causing traffic loss and session stalls.

Sponsored by: Hobnob, Inc.

If software retransmit occurs with an ath_buf marked ATH_BUF_BUSY,
it's cloned and that clone is retransmitted. This means that the
ath_buf pointer squirreled away on the baw window array is suddenly
wrong and was causing all kinds of console output.

This updates the pointer in that particular BAW slot to the new
ath_buf after ensuring that:

* the new and old buffers have the same seqno;
* the current slot pointer matches the old buffer pointer.

This quietens the debugging output (again), restoring said debugging
to only signify when a broken condition has occured.

Sponsored by: Hobnob, Inc.

Introduce TX aggregation and software TX queue management
for Atheros AR5416 and later wireless devices.

This is a very large commit - the complete history can be
found in the user/adrian/if_ath_tx branch.

Legacy (ie, pre-AR5416) devices also use the per-software
TXQ support and (in theory) can support non-aggregation
ADDBA sessions. However, the net80211 stack doesn't currently
support this.

In summary:

TX path:

* queued frames normally go onto a per-TID, per-node queue
* some special frames (eg ADDBA control frames) are thrown
directly onto the relevant hardware queue so they can
go out before any software queued frames are queued.
* Add methods to create, suspend, resume and tear down an
aggregation session.
* Add in software retransmission of both normal and aggregate
frames.
* Add in completion handling of aggregate frames, including
parsing the block ack bitmap provided by the hardware.
* Write an aggregation function which can assemble frames into
an aggregate based on the selected rate control and channel
configuration.
* The per-TID queues are locked based on their target hardware
TX queue. This matches what ath9k/atheros does, and thus
simplified porting over some of the aggregation logic.
* When doing TX aggregation, stick the sequence number allocation
in the TX path rather than net80211 TX path, and protect it
by the TXQ lock.

Rate control:

* Delay rate control selection until the frame is about to
be queued to the hardware, so retried frames can have their
rate control choices changed. Frames with a static rate
control selection have that applied before each TX, just
to simplify the TX path (ie, not have "static" and "dynamic"
rate control special cased.)
* Teach ath_rate_sample about aggregates - both completion and
errors.
* Add an EWMA for tracking what the current "good" MCS rate is
based on failure rates.

Misc:

* Introduce a bunch of dirty hacks and workarounds so TID mapping
and net80211 frame inspection can be kept out of the net80211
layer. Because of the way this code works (and it's from Atheros
and Linux ath9k), there is a consistent, 1:1 mapping between
TID and AC. So we need to ensure that frames going to a specific
TID will _always_ end up on the right AC, and vice versa, or the
completion/locking will simply get very confused. I plan on
addressing this mess in the future.

Known issues:

* There is no BAR frame transmission just yet. A whole lot of
tidying up needs to occur before BAR frame TX can occur in the
"correct" place - ie, once the TID TX queue has been drained.

* Interface reset/purge/etc results in frames in the TX and RX
queues being removed. This creates holes in the sequence numbers
being assigned and the TX/RX AMPDU code (on either side) just
hangs.

* There's no filtered frame support at the present moment, so
stations going into power saving mode will simply have a number
of frames dropped - likely resulting in a traffic "hang".

* Raw frame TX is going to just not function with 11n aggregation.
Likely this needs to be modified to always override the sequence
number if the frame is going into an aggregation session.
However, general raw frame injection currently doesn't work in
general in net80211, so let's just ignore this for now until
this is sorted out.

* HT protection is just not implemented and won't be until the above
is sorted out. In addition, the AR5416 has issues RTS protecting
large aggregates (anything >8k), so the work around needs to be
ported and tested. Thus, this will be put on hold until the above
work is complete.

* The rate control module 'sample' is the only currently supported
module; onoe/amrr haven't been tested and have likely bit rotted
a little. I'll follow up with some commits to make them work again
for non-11n rates, but they won't be updated to handle 11n and
aggregation. If someone wishes to do so then they're welcome to
send along patches.

* .. and "sample" doesn't really do a good job of 11n TX. Specifically,
the metrics used (packet TX time and failure/success rates) isn't as
useful for 11n. It's likely that it should be extended to take into
account the aggregate throughput possible and then choose a rate
which maximises that. Ie, it may be acceptable for a higher MCS rate
with a higher failure to be used if it gives a more acceptable
throughput/latency then a lower MCS rate @ a lower error rate.
Again, patches will be gratefully accepted.

Because of this, ATH_ENABLE_11N is still not enabled by default.

Sponsored by: Hobnob, Inc.
Obtained from: Linux, Atheros

Change the descriptor logic to use bf_lastds to point to the last
descriptor, rather than using the maths involving bf_desc[bf_nseg - 1].

When doing TX aggregation, the status will be updated in the -final-
descriptor of the -final- subframe in an aggregate. Thus bf_lastds
may point to the last descriptor in a completely different ath_buf.

Sponsored by: Hobnob, Inc.

Migrate the STAILQ lists to TAILQs.

A bunch of the 11n TX aggregation logic wants to traverse lists of buffers
in various ways. In order to provide O(1) behaviour in this instance,
use TAILQs.

This does blow out the memory footprint and CPU cycles slightly for some
of these operations. I may convert some of these back to STAILQs once
the rest of the software transmit queue handling has been stabilised.

Sponsored by: Hobnob, Inc.

Add in HT protection but disable it by default.

I'll clear how it's supposed to work with Bernhard and then look
at enabling this in the correct situations.

But this -does- enable HT RTS protection (using the appropriate legacy
rates) if this bit of code is enabled.

Shuffle around the RTS/CTS rate/duration logic.

* Turn ath_tx_calc_ctsduration() into a function that
returns the ctsduration, or -1 for HT rates;
* add a printf() to ath_tx_calc_ctsduration() which will be
very loud if somehow that function is called with an MCS
rate;
* Add ath_tx_get_rtscts_rate() which returns the RTS/CTS
rate to use for the given data rate, incl. the short
preamble flag;
* Only call ath_tx_calc_ctsduration() for non-11n chipsets;
11n chipsets don't require the rtscts duration to be
calculated.

The current code used the fields in ath_set11nratescenario() . Use them
correctly:

* pass in whether to allow the hardware to override the duration field
in the main data frame (durupdate_en) - PS_POLL frames in particular
don't have the duration bit overriden;
* there's no rts/cts duration here; that's done elsehwere

Modify the TX path to set and use the 11n rate scenario bits.

This isn't strictly required to TX (at least non-agg and non-HT40,
non-short-GI) frames; but as it needs to be done anyway, just get
it done.

Linux ath9k uses the rate scenario style path for -all- packets,
legacy or otherwise. This code does much the same.

Beacon TX still uses the legacy, non-rate-scenario TX descriptor
setup. Ath9k also does this.

This 11n rate scenario path is only called for chips in the AR5416
HAL; legacy chips use the previous interface for TX'ing.

Refator the common code which calculates the 802.11g protection duration.

* Add a rather hacky "does this speak the 11n TX descriptor format"
function; which will be later used by the TX path to determine
whether to use the extended features or not.

* Break out the descriptor chaining logic into a separate function;
again so it can be switched out later on for the 11n version when
needed.

* Refactor out the encryption-swizzling code that's common in the
raw and normal TX path.

Migrate the TX path code out of if_ath and into a separate source file.

There's two reasons for this:

* the raw and non-raw TX path shares a lot of duplicate code which should be
refactored;
* the 11n-ready chip TX path needs a little reworking.