Revert r330897:

This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.

Revert with prejudice.

This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.

Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.

Requested by: gjb (re)

Partial merge of the SPDX changes

These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.

No objections from: pfg

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Replay r286410. Change KPI of how device drivers that provide wireless
connectivity interact with the net80211 stack.

Historical background: originally wireless devices created an interface,
just like Ethernet devices do. Name of an interface matched the name of
the driver that created. Later, wlan(4) layer was introduced, and the
wlanX interfaces become the actual interface, leaving original ones as
"a parent interface" of wlanX. Kernelwise, the KPI between net80211 layer
and a driver became a mix of methods that pass a pointer to struct ifnet
as identifier and methods that pass pointer to struct ieee80211com. From
user point of view, the parent interface just hangs on in the ifconfig
list, and user can't do anything useful with it.

Now, the struct ifnet goes away. The struct ieee80211com is the only
KPI between a device driver and net80211. Details:

- The struct ieee80211com is embedded into drivers softc.
- Packets are sent via new ic_transmit method, which is very much like
the previous if_transmit.
- Bringing parent up/down is done via new ic_parent method, which notifies
driver about any changes: number of wlan(4) interfaces, number of them
in promisc or allmulti state.
- Device specific ioctls (if any) are received on new ic_ioctl method.
- Packets/errors accounting are done by the stack. In certain cases, when
driver experiences errors and can not attribute them to any specific
interface, driver updates ic_oerrors or ic_ierrors counters.

Details on interface configuration with new world order:
- A sequence of commands needed to bring up wireless DOESN"T change.
- /etc/rc.conf parameters DON'T change.
- List of devices that can be used to create wlan(4) interfaces is
now provided by net.wlan.devices sysctl.

Most drivers in this change were converted by me, except of wpi(4),
that was done by Andriy Voskoboinyk. Big thanks to Kevin Lo for testing
changes to at least 8 drivers. Thanks to pluknet@, Oliver Hartmann,
Olivier Cochard, gjb@, mmoll@, op@ and lev@, who also participated in
testing.

Reviewed by: adrian
Sponsored by: Netflix
Sponsored by: Nginx, Inc.

Revert the wifi ifnet changes until things are more baked and tested.

* 286410
* 286413
* 286416

The initial commit broke a variety of debug and features that aren't
in the GENERIC kernels but are enabled in other platforms.

Change KPI of how device drivers that provide wireless connectivity interact
with the net80211 stack.

Historical background: originally wireless devices created an interface,
just like Ethernet devices do. Name of an interface matched the name of
the driver that created. Later, wlan(4) layer was introduced, and the
wlanX interfaces become the actual interface, leaving original ones as
"a parent interface" of wlanX. Kernelwise, the KPI between net80211 layer
and a driver became a mix of methods that pass a pointer to struct ifnet
as identifier and methods that pass pointer to struct ieee80211com. From
user point of view, the parent interface just hangs on in the ifconfig
list, and user can't do anything useful with it.

Now, the struct ifnet goes away. The struct ieee80211com is the only
KPI between a device driver and net80211. Details:

- The struct ieee80211com is embedded into drivers softc.
- Packets are sent via new ic_transmit method, which is very much like
the previous if_transmit.
- Bringing parent up/down is done via new ic_parent method, which notifies
driver about any changes: number of wlan(4) interfaces, number of them
in promisc or allmulti state.
- Device specific ioctls (if any) are received on new ic_ioctl method.
- Packets/errors accounting are done by the stack. In certain cases, when
driver experiences errors and can not attribute them to any specific
interface, driver updates ic_oerrors or ic_ierrors counters.

Details on interface configuration with new world order:
- A sequence of commands needed to bring up wireless DOESN"T change.
- /etc/rc.conf parameters DON'T change.
- List of devices that can be used to create wlan(4) interfaces is
now provided by net.wlan.devices sysctl.

Most drivers in this change were converted by me, except of wpi(4),
that was done by Andriy Voskoboinyk. Big thanks to Kevin Lo for testing
changes to at least 8 drivers. Thanks to Olivier Cochard, gjb@, mmoll@,
op@ and lev@, who also participated in testing. Details here:

https://wiki.freebsd.org/projects/ifnet/net80211

Still, drivers: ndis, wtap, mwl, ipw, bwn, wi, upgt, uath were not
tested. Changes to mwl, ipw, bwn, wi, upgt are trivial and chances
of problems are low. The wtap wasn't compilable even before this change.
But the ndis driver is complex, and it is likely to be broken with this
commit. Help with testing and debugging it is appreciated.

Differential Revision: D2655, D2740
Sponsored by: Nginx, Inc.
Sponsored by: Netflix

Use device_printf() instead of if_printf(). No functional changes.

Print out the final_rix if there's a problem.

The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare
to this event, adding if_var.h to files that do need it. Also, include
all includes that now are included due to implicit pollution via if_var.h

Sponsored by: Netflix
Sponsored by: Nginx, Inc.

I give up - just throw the EWMA update into the normal update_stats()
routine.

There were still corner cases where the EWMA update stats are being
called on a rix which didn't have an intermediary stats update; thus
no packets were counted against it. Sigh.

This should fix the crashes I've been seeing on recent -HEAD.

Update the EWMA statistics for each intermediary rate as well as the final
rate.

This fixes two things:

* The intermediary rates now also have their EWMA values changed;
* The existing code was using the wrong value for longtries - so the
EWMA stats were only adjusted for the first rate and not subsequent
rates in a MRR setup.

TODO:

* Merge the EWMA updates into update_stats() now..

I've had some feedback that CCK rates are more reliable than MCS 0
in some very degenerate conditions.

However, until ath_rate_form_aggr() is taught to not form aggregates
if ANY selected rate is non-MCS, this can't yet be enabled.

So, just add a comment.

Improve performance of the Sample rate algorithm on 802.11n networks.

* Don't treat high percentage failures as "sucessive failures" - high
MCS rates are very picky and will quite happily "fade" from low
to high failure % and back again within a few seconds. If they really
don't work, the aggregate will just plain fail.

* Only sample MCS rates +/- 3 from the current MCS. Sample will back off
quite quickly, so there's no need to sample _all_ MCS rates between
a high MCS rate and MCS0; there may be a lot of them.

* Modify the smoothing rate to be 75% rather than 95% - it's more adaptive
but it comes with a cost of being slightly less stable at times.
A per-node, hysterisis behaviour would be nicer.

Improve the sample rate logging.

Fix build

Extend the sample mask from 32 bits to 64 bits.

This is required to support > MCS15 as more than 32 bit rate entries are
suddenly available.

This is quite messy - instead of doing typecasts at each mask operation,
this should be migrated to use a macro and have that do the typecast.

Fix a case of "mis-located braces".

PR: kern/170302

Allow 802.11n hardware to support multi-rate retry when RTS/CTS is
enabled.

The legacy (pre-802.11n) hardware doesn't support this - although
the AR5212 era hardware supports MRR, it doesn't have all the bits
needed to support MRR + RTS/CTS. The AR5416 and later support
a packet duration and RTS/CTS flags per rate scenario, so we should
support it.

Tested:

* AR9280, STA

PR: kern/170302

Introduce a rate table TLV so rate table statistics consumers
know how to map rix -> rate code.

Add a per-node rate control routine for each rate control module.

For now, the only module implement is 'sample', and that's only partially
implemented. The main issue here with reusing this structure in userland
is that it uses 'rix' everywhere, which requires the userland code to
have access to the current HAL rate table.

For now, this is a very large work in progress.

Specific details:

* The rate control information is per-node at the moment and wrapped
in a TLV, to ease parsing and backwards compatibility.
* .. but so I can be slack for now, the userland statistics are just
a copy of the kernel-land sample node state.
* However, for now use a temporary copy and change the rix entries
to dot11rate entries to make it slightly easier to eyeball.

Problems:

* The actual rate information table is unfortunately indexed by rix
and it doesn't contain a rate code. So the userland side of this
currently has no way to extract out a mapping.

TODO:

* Add a TLV payload to dump out the rate control table mapping so
'rix' can be turned into a dot11 / MCS rate.
* .. then remove the temporary copy.

In a complete lack of foresight on my part, my previous commit broke
the assumption that ath_softc doesn't change size based on build time
configuration.

I picked up on this because suddenly radar stuff didn't work; and
although the ath_dfs code was setting sc_dodfs=1, the main ath driver
saw sc_dodfs=0.

So for now, include opt_ath.h in driver source files. This seems like
the sane thing to do anyway.

I'll have to do a pass over the code at some later stage and turn
the radiotap TX/RX structs into malloc'ed memory, rather than in-line
inside of ath_softc. I'd rather like to keep ath_softc the same
layout regardless of configuration parameters.

Pointy hat to: adrian

Add in some debugging code to check whether the current rate table has
been bait-and-switched from the rate control code.

This will avoid the panic that I saw and will avoid sending invalid rates
(eg 11a/11g OFDM rates when in 11b, on 11b-only NICs (AR5211)) where the
rate table is not "big".

It also will point out situations where this occurs for the 11n NICs
which will have sufficiently large rate tables that "invalid rix" doesn't
occur.

I'll try to follow this up with a commit that adds a current operating mode
check. The "rix" is only relevant to the current operating mode and rate
table.

PR: kern/165475

Fix the compile to work when IEEE80211_DEBUG isn't defined.

Sponsored by: Hobnob, Inc.

Introduce TX aggregation and software TX queue management
for Atheros AR5416 and later wireless devices.

This is a very large commit - the complete history can be
found in the user/adrian/if_ath_tx branch.

Legacy (ie, pre-AR5416) devices also use the per-software
TXQ support and (in theory) can support non-aggregation
ADDBA sessions. However, the net80211 stack doesn't currently
support this.

In summary:

TX path:

* queued frames normally go onto a per-TID, per-node queue
* some special frames (eg ADDBA control frames) are thrown
directly onto the relevant hardware queue so they can
go out before any software queued frames are queued.
* Add methods to create, suspend, resume and tear down an
aggregation session.
* Add in software retransmission of both normal and aggregate
frames.
* Add in completion handling of aggregate frames, including
parsing the block ack bitmap provided by the hardware.
* Write an aggregation function which can assemble frames into
an aggregate based on the selected rate control and channel
configuration.
* The per-TID queues are locked based on their target hardware
TX queue. This matches what ath9k/atheros does, and thus
simplified porting over some of the aggregation logic.
* When doing TX aggregation, stick the sequence number allocation
in the TX path rather than net80211 TX path, and protect it
by the TXQ lock.

Rate control:

* Delay rate control selection until the frame is about to
be queued to the hardware, so retried frames can have their
rate control choices changed. Frames with a static rate
control selection have that applied before each TX, just
to simplify the TX path (ie, not have "static" and "dynamic"
rate control special cased.)
* Teach ath_rate_sample about aggregates - both completion and
errors.
* Add an EWMA for tracking what the current "good" MCS rate is
based on failure rates.

Misc:

* Introduce a bunch of dirty hacks and workarounds so TID mapping
and net80211 frame inspection can be kept out of the net80211
layer. Because of the way this code works (and it's from Atheros
and Linux ath9k), there is a consistent, 1:1 mapping between
TID and AC. So we need to ensure that frames going to a specific
TID will _always_ end up on the right AC, and vice versa, or the
completion/locking will simply get very confused. I plan on
addressing this mess in the future.

Known issues:

* There is no BAR frame transmission just yet. A whole lot of
tidying up needs to occur before BAR frame TX can occur in the
"correct" place - ie, once the TID TX queue has been drained.

* Interface reset/purge/etc results in frames in the TX and RX
queues being removed. This creates holes in the sequence numbers
being assigned and the TX/RX AMPDU code (on either side) just
hangs.

* There's no filtered frame support at the present moment, so
stations going into power saving mode will simply have a number
of frames dropped - likely resulting in a traffic "hang".

* Raw frame TX is going to just not function with 11n aggregation.
Likely this needs to be modified to always override the sequence
number if the frame is going into an aggregation session.
However, general raw frame injection currently doesn't work in
general in net80211, so let's just ignore this for now until
this is sorted out.

* HT protection is just not implemented and won't be until the above
is sorted out. In addition, the AR5416 has issues RTS protecting
large aggregates (anything >8k), so the work around needs to be
ported and tested. Thus, this will be put on hold until the above
work is complete.

* The rate control module 'sample' is the only currently supported
module; onoe/amrr haven't been tested and have likely bit rotted
a little. I'll follow up with some commits to make them work again
for non-11n rates, but they won't be updated to handle 11n and
aggregation. If someone wishes to do so then they're welcome to
send along patches.

* .. and "sample" doesn't really do a good job of 11n TX. Specifically,
the metrics used (packet TX time and failure/success rates) isn't as
useful for 11n. It's likely that it should be extended to take into
account the aggregate throughput possible and then choose a rate
which maximises that. Ie, it may be acceptable for a higher MCS rate
with a higher failure to be used if it gives a more acceptable
throughput/latency then a lower MCS rate @ a lower error rate.
Again, patches will be gratefully accepted.

Because of this, ATH_ENABLE_11N is still not enabled by default.

Sponsored by: Hobnob, Inc.
Obtained from: Linux, Atheros

Some cosmetic fixes to ath_rate_sample.

* Use 64 bit integer types for the sample rate statistics.
When TX'ing 11n aggregates, a 32 bit counter will overflow in a few
hours due to the high packet throughput.

* Create a default label of "" rather than defaulting to "Mb" - that way
if a rate hasn't yet been selected, it won't say "-1 Mb".

Sponsored by: Hobnob, Inc.

Modify the sample rate control algorithm to only select/sample HT
rates for HT nodes.

After discussing with Bernhard, the "right" way in net80211 to check
the channel width is ni->ni_chw, which is set to the negotiated channel
width. ni->ni_htflags is the capability, rather than the negotiated
value.

Teach both the TX path and the sample rate module about this.

Fix static ucastrate for ath_rate_sample.

* Pull out the static rix stuff into a different function
* I know this may slightly drop performance, but check if a static
rix is needed before each packet TX.

* Whilst I'm at it, add a little extra debugging to the rate
control stuff to make it easier to follow what's going on.

The sample rate module currently does the slightly wrong thing when
determining whether to use MRR or not.

It uses the 11g protection mode when calculating 11n related stuff, rather
than checking the 11n protection mode.

Furthermore, the 11n chipsets can quite happily handle multi-rate retry w/
protection; the TX path and rate control modules need to be taught about
that.

Modify the sample rate module output to be (slightly) easier to understand.

* add dot11rate_label() which returns Mb or MCS based on legacy or HT
* use it everywhere dot11rate() is used
* in the "current selection" part at the top of the debugging output,
otuput what the rate itself is rather than the rix. The rate index
(rix) has very little meaning to normal humans who don't know how
to find the PHY settings for each of the chipsets; pointing out the
rix rate and type is likely more useful.

Properly propagate whether the channel is HT40 or not when calculating
packet duration for the ath_rate_sample module.

This doesn't affect the packet TX at all; only how much time the
sample rate module attributes to a completed TX.

Add a new method to the rate control modules which extract out the
three other rates and try counts.

The 11n rate scenario path wants to take a list of rate and tries,
rather than calling setupxtxdesc().

(Mostly) teach ath_rate_sample about MCS rates.

This is just the bare minimum needed to teach ath_rate_sample to try
and handle MCS rates. It doesn't at all attempt to find the best
rate by any means - it doesn't know anything about the MCS rate
relations, TX aggregation or any of the much sexier 11n stuff
that's out there.

It's just enough to transmit 11n frames and handle TX completion.

It shouldn't affect legacy (11abg) behaviour.

Obtained from: rpaulo@

Migrate the sample rate module to the new ath_hal_gettxcompletionrates() API.

This removes the chipset-dependent TX DMA completion descriptor groveling.
It should now be (more) portable to other, later atheros chipsets when the
time comes.

purge HAL_TXSTAT_ALTRATE; you can figure this out by checking ts_finaltsi
and it cannot be used with MCS rate codes

add a debug msg for when a fixed transmit rate is not applied because
it's not found in the sta's negotiated rate set

correct fixed rate handling; the rixmap was changed a while back
to be indexed by the ieee rate code

remove module glue, it's not used any more

Switch to ath hal source code. Note this removes the ath_hal
module; the ath module now brings in the hal support. Kernel
config files are almost backwards compatible; supplying

device ath_hal

gives you the same chip support that the binary hal did but you
must also include

options AH_SUPPORT_AR5416

to enable the extended format descriptors used by 11n parts.
It is now possible to control the chip support included in a
build by specifying exactly which chips are to be supported
in the config file; consult ath_hal(4) for information.

cover up sun4v namespace pollution

Major overhaul:
o eliminate private state indexed by 802.11 rate codes; use the hal's
rate tables directly to get the same info
o calculate a mask of operational rates to optimize lookups and checks
(instead of using for loops and similar)
o optimize size bin operations
o ignore rates marked as "do not use" in the hal phy tables
o fix bug that caused upshifting to break in 11g once the rate dropped
below 11Mb/s
o add more intelligent multi-rate tx schedules
o add support for 1/2 and 1/4 width channels
o add dev.ath.X.sample_stats sysctl to dump runtime statistics to the console
(needs to go up to a user app)
o export more tuning knobs via sysctls (still a couple of magic constants)

o With the addition of HT rates the set of h/w codes has a much wider range
making the use of sc_hwmap to do direct mapping impractical. Switch to
indexing by the rate index instead of the rate code and adjust associated
state and logic appropriately. This has several benefits including
simplification of the led code.
o fix radiotap capture of HT rates
o fix conditional compilation of HT radiotap support to be based on the
hal having 5416 support; not the ABI version as hal builds may or may
not include 5416 support

move complaints about bad rate codes up a level so we can print
the h/w rate code and other useful info

remove driver-private equivalent of ni_txparms; it's now superfluous

now that the new association callback is used when joining a bss we can
eliminate the ath_rate_newassoc callback and associated code

Multi-bss (aka vap) support for 802.11 devices.

Note this includes changes to all drivers and moves some device firmware
loading to use firmware(9) and a separate module (e.g. ral). Also there
no longer are separate wlan_scan* modules; this functionality is now
bundled into the wlan module.

Supported by: Hobnob and Marvell
Reviewed by: many
Obtained from: Atheros (some bits)

revert 1.18: the negotiated rate set may not match the hal
rate tables, so using the hal's rateCodeToIndex array
will produce wrong indices for the negotiated rate set

MFC after: 3 days

convert hardware rate codes to IEEE rate codes with a lookup table
instead of a linear search

Reviewed by: sephe, avatar
Approved by: re (blanket wireless)
MFC after: 2 weeks

Update 802.11 wireless support:
o major overhaul of the way channels are handled: channels are now
fully enumerated and uniquely identify the operating characteristics;
these changes are visible to user applications which require changes
o make scanning support independent of the state machine to enable
background scanning and roaming
o move scanning support into loadable modules based on the operating
mode to enable different policies and reduce the memory footprint
on systems w/ constrained resources
o add background scanning in station mode (no support for adhoc/ibss
mode yet)
o significantly speedup sta mode scanning with a variety of techniques
o add roaming support when background scanning is supported; for now
we use a simple algorithm to trigger a roam: we threshold the rssi
and tx rate, if either drops too low we try to roam to a new ap
o add tx fragmentation support
o add first cut at 802.11n support: this code works with forthcoming
drivers but is incomplete; it's included now to establish a baseline
for other drivers to be developed and for user applications
o adjust max_linkhdr et. al. to reflect 802.11 requirements; this eliminates
prepending mbufs for traffic generated locally
o add support for Atheros protocol extensions; mainly the fast frames
encapsulation (note this can be used with any card that can tx+rx
large frames correctly)
o add sta support for ap's that beacon both WPA1+2 support
o change all data types from bsd-style to posix-style
o propagate noise floor data from drivers to net80211 and on to user apps
o correct various issues in the sta mode state machine related to handling
authentication and association failures
o enable the addition of sta mode power save support for drivers that need
net80211 support (not in this commit)
o remove old WI compatibility ioctls (wicontrol is officially dead)
o change the data structures returned for get sta info and get scan
results so future additions will not break user apps
o fixed tx rate is now maintained internally as an ieee rate and not an
index into the rate set; this needs to be extended to deal with
multi-mode operation
o add extended channel specifications to radiotap to enable 11n sniffing

Drivers:
o ath: add support for bg scanning, tx fragmentation, fast frames,
dynamic turbo (lightly tested), 11n (sniffing only and needs
new hal)
o awi: compile tested only
o ndis: lightly tested
o ipw: lightly tested
o iwi: add support for bg scanning (well tested but may have some
rough edges)
o ral, ural, rum: add suppoort for bg scanning, calibrate rssi data
o wi: lightly tested

This work is based on contributions by Atheros, kmacy, sephe, thompsa,
mlaier, kevlo, and others. Much of the scanning work was supported by
Atheros. The 11n work was supported by Marvell.

save changes for handling 5416/5418 parts

Track v0.9.20.3 hal:

o no more ds_vdata in tx/rx descriptors
o split h/w tx/rx descriptor from s/w status
o as part of the descriptor split change the rate control module api
so the ath_buf is passed in to the module so it can fetch both
descriptor and status information as needed
o add some const poisoning

Also for sample rate control algorithm:

o split debug msgs (node, rate, any)
o uniformly bounds check rate indices (and in some cases correct checks)
o move array index ops to after bounds checking
o use final tsi from the status block instead of the h/w descriptor
o replace h/w descriptor struct's with proper mask+shift defs (this
doesn't belong here; everything is known by the driver and should
just be sent down so there's no h/w-specific knowledge)

MFC after: 1 month

correct ni_txrate when using a fixed rate; fixes current rate reporting

MFC after: 3 days

fix comment and whitespace

move mgt frame tx rate responsibility from the rate control modules
to the driver; this avoids redundant logic and will be necessary
for future additions

MFC after: 2 weeks

sync with latest code in madwifi

Obtained from: madwifi
MFC after: 2 weeks

bandaid inconsistent state handling: the rate index map may be
stale when called to reset rate control state causing us to
pickup an invalid index, check for this and skip 'em (things
will eventually get fixed up so this is not harmful)

diff reduction against p4: define IEEE80211_FIXED_RATE_NONE and use
it instead of -1

fix size_to_bin

Obtained from: madwifi

nuke unintentional use of HAL_BOOL type

o fix bug where rate wouldn't lift off lowest setting when operating as
an ap in 11g with protection enabled
o correct rate selection when operating in 11g with protection when no
packets have been sent yet (from John Bicket)
o track api change to get first descriptor and use it to collect the frame
length for calculating the state bin
o add more debugging and shuffle some existing debugging to give more info
o bump version to distinguish bug fixes

sync rates for any associated stations or neighbors on state transition

fix braino introduced when converting from madwifi

version 1.1 (with cleanups)

Submitted by: John Bicket

Adding missing module dependency. This should fix the undefined symbol
error(ath_hal_computetxtime) during module loading.

Reviewed by: sam (mentor)

SampleRate rate control algorithm for the ath driver

Submitted by: John Bicket