wifi: mt76: fix race condition related to checking tx queue fill status

When drv_tx calls race against local tx scheduling, the queue fill status checks
can potentially race, leading to dma queue entries being overwritten.
Fix this by deferring packets from drv_tx calls to the tx worker, in order to
ensure that all regular queue tx comes from the same context.

Reported-by: Ryder Lee <Ryder.Lee@mediatek.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

wifi: mt76: report non-binding skb tx rate when WED is active

AQL relies on accurate tx rate reporting, so add this patch for
non-binding skb rate when WED is enabled.

Co-developed-by: Ryder Lee <ryder.lee@mediatek.com>
Signed-off-by: Ryder Lee <ryder.lee@mediatek.com>
Signed-off-by: Peter Chiu <chui-hao.chiu@mediatek.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

wifi: mt76: mt7915: rework tx bytes counting when WED is active

Concurrent binding/non-binding skbs could be handled anywhere which leads
to mixed byte counting, so switch to use PPDU TxS reporting regardless Tx
paths when WED is active.

Fixes: 43eaa3689507 ("wifi: mt76: add PPDU based TxS support for WED device")
Co-developed-by: Ryder Lee <ryder.lee@mediatek.com>
Signed-off-by: Ryder Lee <ryder.lee@mediatek.com>
Signed-off-by: Peter Chiu <chui-hao.chiu@mediatek.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

wifi: mt76: add missing locking to protect against concurrent rx/status calls

According to the documentation, ieee80211_rx_list must not run concurrently
with ieee80211_tx_status (or its variants).

Cc: stable@vger.kernel.org
Fixes: 88046b2c9f6d ("mt76: add support for reporting tx status with skb")
Reported-by: Brian Coverstone <brian@mainsequence.net>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

wifi: ieee80211: correctly mark FTM frames non-bufferable

The checks of whether or not a frame is bufferable were not
taking into account that some action frames aren't, such as
FTM. Check this, which requires some changes to the function
ieee80211_is_bufferable_mmpdu() since we need the whole skb
for the checks now.

Reviewed-by: Ilan Peer <ilan.peer@intel.com>
Signed-off-by: Johannes Berg <johannes.berg@intel.com>

wifi: mt76: handle possible mt76_rx_token_consume failures

Take into account possible error conditions of mt76_rx_token_consume
routine in mt7915_mmio_wed_init_rx_buf() and mt76_dma_add_buf()

Fixes: cd372b8c99c5 ("wifi: mt76: add WED RX support to mt76_dma_{add,get}_buf")
Fixes: 4f831d18d12d ("wifi: mt76: mt7915: enable WED RX support")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

wifi: mt76: handle possible mt76_rx_token_consume failures

Take into account possible error conditions of mt76_rx_token_consume
routine in mt7915_mmio_wed_init_rx_buf() and mt76_dma_add_buf()

Fixes: cd372b8c99c5 ("wifi: mt76: add WED RX support to mt76_dma_{add,get}_buf")
Fixes: 4f831d18d12d ("wifi: mt76: mt7915: enable WED RX support")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
(cherry picked from commit 96f134dc19645be4994e89a2f68fa89309becbee)
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20230113105848.34642-2-nbd@nbd.name

wifi: mt76: introduce rxwi and rx token utility routines

This is a preliminary patch to introduce WED RX support for mt7915.

Tested-by: Daniel Golle <daniel@makrotopia.org>
Co-developed-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Sujuan Chen <sujuan.chen@mediatek.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

wifi: mt76: fix rate reporting / throughput regression on mt7915 and newer

mt7915 and newer need to report the rate_info that's stored in wcid->rate,
since they don't fill info->status.rates.

Cc: Jonas Jelonek <jelonek.jonas@gmail.com>
Reported-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Link: https://lore.kernel.org/all/CABXGCsP0znm9pS-MiKtyxTXR7XiyFVqen0qzNpicGHDZKCzbwg@mail.gmail.com/
Fixes: 44fa75f207d8 ("mac80211: extend current rate control tx status API")
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Tested-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220927093823.6007-1-nbd@nbd.name

mt76: convert MT_TX_HW_QUEUE_EXT_PHY to MT_TX_HW_QUEUE_PHY

Report phy_indx in tx_info->hw_queue. This is a preliminary patch to add
newer chipset support

Co-developed-by: Bo Jiao <bo.jiao@mediatek.com>
Signed-off-by: Bo Jiao <bo.jiao@mediatek.com>
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: introduce phys array in mt76_dev structure

Introduce phys array in mt76_dev structure to reference mt76_phy
supported by the chipset. This is a preliminary patch to introduce
newer chipset support.

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: do not use skb_set_queue_mapping for internal purposes

Previously the code used skb_set_queue_mapping for management or non-bufferable
powersave frames that need to be sent to a different hardware queue.
This can confuse AQL, which expects the value to remain the same until the tx
status report.
The only place that currently uses the altered skb queue mapping is the txwi
write function. Change the code to pass the hardware queue id as a function
parameter instead.

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: pass original queue id from __mt76_tx_queue_skb to the driver

MT7615 and newer map multiple software tx queues to the hardware id

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mac80211: extend current rate control tx status API

This patch adds the new struct ieee80211_rate_status and replaces
'struct rate_info *rate' in ieee80211_tx_status with pointer and length
annotation.

The struct ieee80211_rate_status allows to:
(1) receive tx power status feedback for transmit power control (TPC)
per packet or packet retry
(2) dynamic mapping of wifi chip specific multi-rate retry (mrr)
chains with different lengths
(3) increase the limit of annotatable rate indices to support
IEEE802.11ac rate sets and beyond

ieee80211_tx_info, control and status buffer, and ieee80211_tx_rate
cannot be used to achieve these goals due to fixed size limitations.

Our new struct contains a struct rate_info to annotate the rate that was
used, retry count of the rate and tx power. It is intended for all
information related to RC and TPC that needs to be passed from driver to
mac80211 and its RC/TPC algorithms like Minstrel_HT. It corresponds to
one stage in an mrr. Multiple subsequent instances of this struct can be
included in struct ieee80211_tx_status via a pointer and a length variable.
Those instances can be allocated on-stack. The former reference to a single
instance of struct rate_info is replaced with our new annotation.

An extension is introduced to struct ieee80211_hw. There are two new
members called 'tx_power_levels' and 'max_txpwr_levels_idx' acting as a
tx power level table. When a wifi device is registered, the driver shall
supply all supported power levels in this list. This allows to support
several quirks like differing power steps in power level ranges or
alike. TPC can use this for algorithm and thus be designed more abstract
instead of handling all possible step widths individually.

Further mandatory changes in status.c, mt76 and ath11k drivers due to the
removal of 'struct rate_info *rate' are also included.
status.c already uses the information in ieee80211_tx_status->rate in
radiotap, this is now changed to use ieee80211_rate_status->rate_idx.
mt76 driver already uses struct rate_info to pass the tx rate to status
path. The new members of the ieee80211_tx_status are set to NULL and 0
because the previously passed rate is not relevant to rate control and
accurate information is passed via tx_info->status.rates.
For ath11k, the txrate can be passed via this struct because ath11k uses
firmware RC and thus the information does not interfere with software RC.

Compile-Tested: current wireless-next tree with all flags on
Tested-on: Xiaomi 4A Gigabit (MediaTek MT7603E, MT7612E) with OpenWrt
Linux 5.10.113

Signed-off-by: Jonas Jelonek <jelonek.jonas@gmail.com>
Link: https://lore.kernel.org/r/20220509173958.1398201-2-jelonek.jonas@gmail.com
Signed-off-by: Johannes Berg <johannes.berg@intel.com>

mt76: mt7915: add Wireless Ethernet Dispatch support

This is used to support hardware flow offloading from Ethernet to WLAN

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: make number of tokens configurable dynamically

Preparation for adding Wireless Ethernet Dispatch support

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: fix tx status related use-after-free race on station removal

There is a small race window where ongoing tx activity can lead to a skb
getting added to the status tracking idr after that idr has already been
cleaned up, which will keep the wcid linked in the status poll list.
Fix this by only adding status skbs if the wcid pointer is still assigned
in dev->wcid, which gets cleared early by mt76_sta_pre_rcu_remove

Fixes: bd1e3e7b693c ("mt76: introduce packet_id idr")
Tested-by: Ben Greear <greearb@candelatech.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: fix use-after-free by removing a non-RCU wcid pointer

Fixes an issue caught by KASAN about use-after-free in mt76_txq_schedule
by protecting mtxq->wcid with rcu_lock between mt76_txq_schedule and
sta_info_[alloc, free].

[18853.876689] ==================================================================
[18853.876751] BUG: KASAN: use-after-free in mt76_txq_schedule+0x204/0xaf8 [mt76]
[18853.876773] Read of size 8 at addr ffffffaf989a2138 by task mt76-tx phy0/883
[18853.876786]
[18853.876810] CPU: 5 PID: 883 Comm: mt76-tx phy0 Not tainted 5.10.100-fix-510-56778d365941-kasan #5 0b01fbbcf41a530f52043508fec2e31a4215

[18853.876840] Call trace:
[18853.876861] dump_backtrace+0x0/0x3ec
[18853.876878] show_stack+0x20/0x2c
[18853.876899] dump_stack+0x11c/0x1ac
[18853.876918] print_address_description+0x74/0x514
[18853.876934] kasan_report+0x134/0x174
[18853.876948] __asan_report_load8_noabort+0x44/0x50
[18853.876976] mt76_txq_schedule+0x204/0xaf8 [mt76 074e03e4640e97fe7405ee1fab547b81c4fa45d2]
[18853.877002] mt76_txq_schedule_all+0x2c/0x48 [mt76 074e03e4640e97fe7405ee1fab547b81c4fa45d2]
[18853.877030] mt7921_tx_worker+0xa0/0x1cc [mt7921_common f0875ebac9d7b4754e1010549e7db50fbd90a047]
[18853.877054] __mt76_worker_fn+0x190/0x22c [mt76 074e03e4640e97fe7405ee1fab547b81c4fa45d2]
[18853.877071] kthread+0x2f8/0x3b8
[18853.877087] ret_from_fork+0x10/0x30
[18853.877098]
[18853.877112] Allocated by task 941:
[18853.877131] kasan_save_stack+0x38/0x68
[18853.877147] __kasan_kmalloc+0xd4/0xfc
[18853.877163] kasan_kmalloc+0x10/0x1c
[18853.877177] __kmalloc+0x264/0x3c4
[18853.877294] sta_info_alloc+0x460/0xf88 [mac80211]
[18853.877410] ieee80211_prep_connection+0x204/0x1ee0 [mac80211]
[18853.877523] ieee80211_mgd_auth+0x6c4/0xa4c [mac80211]
[18853.877635] ieee80211_auth+0x20/0x2c [mac80211]
[18853.877733] rdev_auth+0x7c/0x438 [cfg80211]
[18853.877826] cfg80211_mlme_auth+0x26c/0x390 [cfg80211]
[18853.877919] nl80211_authenticate+0x6d4/0x904 [cfg80211]
[18853.877938] genl_rcv_msg+0x748/0x93c
[18853.877954] netlink_rcv_skb+0x160/0x2a8
[18853.877969] genl_rcv+0x3c/0x54
[18853.877985] netlink_unicast_kernel+0x104/0x1ec
[18853.877999] netlink_unicast+0x178/0x268
[18853.878015] netlink_sendmsg+0x3cc/0x5f0
[18853.878030] sock_sendmsg+0xb4/0xd8
[18853.878043] ____sys_sendmsg+0x2f8/0x53c
[18853.878058] ___sys_sendmsg+0xe8/0x150
[18853.878071] __sys_sendmsg+0xc4/0x1f4
[18853.878087] __arm64_compat_sys_sendmsg+0x88/0x9c
[18853.878101] el0_svc_common+0x1b4/0x390
[18853.878115] do_el0_svc_compat+0x8c/0xdc
[18853.878131] el0_svc_compat+0x10/0x1c
[18853.878146] el0_sync_compat_handler+0xa8/0xcc
[18853.878161] el0_sync_compat+0x188/0x1c0
[18853.878171]
[18853.878183] Freed by task 10927:
[18853.878200] kasan_save_stack+0x38/0x68
[18853.878215] kasan_set_track+0x28/0x3c
[18853.878228] kasan_set_free_info+0x24/0x48
[18853.878244] __kasan_slab_free+0x11c/0x154
[18853.878259] kasan_slab_free+0x14/0x24
[18853.878273] slab_free_freelist_hook+0xac/0x1b0
[18853.878287] kfree+0x104/0x390
[18853.878402] sta_info_free+0x198/0x210 [mac80211]
[18853.878515] __sta_info_destroy_part2+0x230/0x2d4 [mac80211]
[18853.878628] __sta_info_flush+0x300/0x37c [mac80211]
[18853.878740] ieee80211_set_disassoc+0x2cc/0xa7c [mac80211]
[18853.878851] ieee80211_mgd_deauth+0x4a4/0x10a0 [mac80211]
[18853.878962] ieee80211_deauth+0x20/0x2c [mac80211]
[18853.879057] rdev_deauth+0x7c/0x438 [cfg80211]
[18853.879150] cfg80211_mlme_deauth+0x274/0x414 [cfg80211]
[18853.879243] cfg80211_mlme_down+0xe4/0x118 [cfg80211]
[18853.879335] cfg80211_disconnect+0x218/0x2d8 [cfg80211]
[18853.879427] __cfg80211_leave+0x17c/0x240 [cfg80211]
[18853.879519] cfg80211_leave+0x3c/0x58 [cfg80211]
[18853.879611] wiphy_suspend+0xdc/0x200 [cfg80211]
[18853.879628] dpm_run_callback+0x58/0x408
[18853.879642] __device_suspend+0x4cc/0x864
[18853.879658] async_suspend+0x34/0xf4
[18853.879673] async_run_entry_fn+0xe0/0x37c
[18853.879689] process_one_work+0x508/0xb98
[18853.879702] worker_thread+0x7f4/0xcd4
[18853.879717] kthread+0x2f8/0x3b8
[18853.879731] ret_from_fork+0x10/0x30
[18853.879741]
[18853.879757] The buggy address belongs to the object at ffffffaf989a2000
[18853.879757] which belongs to the cache kmalloc-8k of size 8192
[18853.879774] The buggy address is located 312 bytes inside of
[18853.879774] 8192-byte region [ffffffaf989a2000, ffffffaf989a4000)
[18853.879787] The buggy address belongs to the page:
[18853.879807] page:000000004bda2a59 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x1d89a0
[18853.879823] head:000000004bda2a59 order:3 compound_mapcount:0 compound_pincount:0
[18853.879839] flags: 0x8000000000010200(slab|head)
[18853.879857] raw: 8000000000010200 ffffffffbc89e208 ffffffffb7fb5208 ffffffaec000cc80
[18853.879873] raw: 0000000000000000 0000000000010001 00000001ffffffff 0000000000000000
[18853.879885] page dumped because: kasan: bad access detected
[18853.879896]
[18853.879907] Memory state around the buggy address:
[18853.879922] ffffffaf989a2000: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[18853.879935] ffffffaf989a2080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[18853.879948] >ffffffaf989a2100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[18853.879961] ^
[18853.879973] ffffffaf989a2180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[18853.879986] ffffffaf989a2200: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[18853.879998] ==================================================================

Cc: stable@vger.kernel.org
Reported-by: Sean Wang <sean.wang@mediatek.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: reduce tx queue lock hold time

- call txq dequeue without holding txq lock (locking handled by mac80211)
- disable bh around tx queue schedule

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: fix timestamp check in tx_status

Should keep SKBs only if timeout timestamp is still after jiffies.
Otherwise, report tx status and drop it direclty.

Fixes: bd1e3e7b693c ("mt76: introduce packet_id idr")
Signed-off-by: Deren Wu <deren.wu@mediatek.com>
Acked-by: Felix Fietkau <nbd@nbd.name>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>
Link: https://lore.kernel.org/r/7e3784949c0b29a00465966b89fdb0192bd0298e.1637593492.git.deren.wu@mediatek.com

mt76: schedule status timeout at dma completion

Reduce MT_TX_STATUS_SKB_TIMEOUT to 250ms

Tested-by: mrkiko.rs@gmail.com
Co-developed-by: Ben Greear <greearb@candelatech.com>
Signed-off-by: Ben Greear <greearb@candelatech.com>
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: substitute sk_buff_head status_list with spinlock_t status_lock

Substitute sk_buff_head status_list with spinlock_t status_lock since we
just need it for locking

Tested-by: mrkiko.rs@gmail.com
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: remove mt76_wcid pointer from mt76_tx_status_check signature

Remove mt76_wcid pointer from mt76_tx_status_check signature since it is
always set to NULL

Tested-by: mrkiko.rs@gmail.com
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: introduce packet_id idr

Introduce per-wcid idr to manage packet id for txs. This allow fast idr
lookup and skb queueing at the same time.

Tested-by: mrkiko.rs@gmail.com
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: mt7915: report tx rate directly from tx status

Report tx rate from tx status packets instead of receving periodic mcu
event. This improves flexibility, accuracy and AQL performance, and
simplifies code flow for better readability.

Signed-off-by: Ryder Lee <ryder.lee@mediatek.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: move mt76_get_next_pkt_id in mt76.h

In order to remove duplicated code, move mt76_get_next_pkt_id routine
in mt76.h

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: fix possible NULL pointer dereference in mt76_tx

Even if this is not a real issue since mt76_tx is never run with wcid set
to NULL, fix a theoretical NULL pointer dereference in mt76_tx routine

Fixes: db9f11d3433f7 ("mt76: store wcid tx rate info in one u32 reduce locking")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: improve tx status codepath

Use ieee80211_tx_status_ext instead of ieee80211_free_skb and
ieee80211_tx_status. This makes it compatible with 802.3 encap offload
and improves performance by removing a redundant sta lookup

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: move mt76_token_init in mt76_alloc_device

In order to remove duplicated code, move mt76_token_init in
mt76_alloc_device routine

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: move token utilities in mt76 common module

Move token management in mt76 common module since it is shared between
mt7615, mt7915 and mt7921 drivers

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: remove MT76_STATE_PM in tx path

since tx/rx path is now relying pm ref counting, get rid of MT76_STATE_PM
check in the tx path

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: mt7663: rely on mt76_connac_pm_ref/mt76_connac_pm_unref in tx path

Introduce mt7615_tx_worker routine as mt76 tx worker callback for
mt7663.
Rely on mt76_connac_pm_ref/mt76_connac_pm_unref to check PM state and
increment/decrement wake counter

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: testmode: add support to send larger packet

Add support to send larger packet in testmode to meet requirements
of some test cases.
The limit of max packet size is determined based on tx rate mode setting.

Signed-off-by: Shayne Chen <shayne.chen@mediatek.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: check return value of mt76_txq_send_burst in mt76_txq_schedule_list

Since mt76_txq_send_burst routine can report a negative error code,
check the returned value before incrementing the number of transmitted
frames in mt76_txq_schedule_list routine.
Return -EBUSY directly if the device is in reset or in power management.

Fixes: 90fdc1717b186 ("mt76: use mac80211 txq scheduling")
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: reduce q->lock hold time

Instead of holding it for the duration of an entire station schedule run,
which can block out competing tasks for a significant amount of time,
only hold it for scheduling one batch of packets for one station.
Improves responsiveness under load

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: testmode: introduce dbdc support

Add testmode support for DBDC NICs (both MT7615D and MT7915D work).
Testmode data and parameters are moved from per-dev to per-phy
for maintaining the value of each band.

Signed-off-by: Shayne Chen <shayne.chen@mediatek.com>
Acked-by: Ryder Lee <ryder.lee@mediatek.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: remove unused variable q

Kernel test robot reported warning:

drivers/net/wireless/mediatek/mt76/tx.c: In function
'mt76_txq_schedule':
>> drivers/net/wireless/mediatek/mt76/tx.c:499:21: warning: variable 'q'
>> set but not used [-Wunused-but-set-variable]
499 | struct mt76_queue *q;
| ^

Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Souptick Joarder <jrdr.linux@gmail.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>
Link: https://lore.kernel.org/r/1607542617-4005-1-git-send-email-jrdr.linux@gmail.com

mt76: attempt to free up more room when filling the tx queue

Run dma cleanup immediately if the queue is almost full, instead of waiting
for the tx interrupt

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: improve tx queue stop/wake

Instead of stopping and waking only a single queue, handle all phy tx queues
mapped ot the same hardware queue.
Also allow the driver to block tx queues

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: move tx hw data queues in mt76_phy

Move hw data queues in mt76_phy from mt76_dev since mt7915 supports per
phy hw queues in dbdc mode

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: rely on mt76_queue in tx_queue_skb signature

Rely on mt76_queue instead of qid in tx_queue_skb signature.
This is a preliminary patch to move data queues in mt76_phy and add
dbdc support to mt7915 driver

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: mt7915: add 802.11 encap offload support

It is currently limited to 3-address mode AP and STA interfaces

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: move pad estimation out of mt76_skb_adjust_pad

Move frame pad computation out of mt76_skb_adjust_pad routine.
This is a preliminary patch to introduce sdio tx aggregation.

Tested-by: Sean Wang <sean.wang@mediatek.com>
Co-developed-by: Sean Wang <sean.wang@mediatek.com>
Signed-off-by: Sean Wang <sean.wang@mediatek.com>
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: remove retry_q from struct mt76_txq and related code

Since the switch to using AQL by default, mtxq->retry_q is never filled anymore
Remove it to get rid of a few more unnecessary cycles in the tx path

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: move txwi handling code to dma.c, since it is mmio specific

This way we can make some functions static

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: convert from tx tasklet to tx worker thread

This improves performance by allowing the scheduler to move the tx scheduling
work to idle CPUs. Since tx scheduling work is very latency sensitive and
kept short via AQL, sched_set_fifo_low is used to keep worker priority above
normal tasks

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: remove struct mt76_sw_queue

All members except for the struct mt76_queue pointer have been removed

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: rely on AQL for burst size limits on tx queueing

Now that AQL works well on all mt76 drivers, completely replace the arbitrary
burst sizing and number of bursts logic for tx scheduling.
For the short period of time in which AQL does not work well yet, limit each
stations to 16 in-flight packets that have no estimated tx time.
This should avoid filling the queue if a station connects and queues up a
large number of packets before rate control information is available, especially
with hardware rate control

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: remove qid argument to drv->tx_complete_skb

It is not needed

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: unify queue tx cleanup code

Cleanup and preparation for changing tx scheduling behavior

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: move mt76_check_agg_ssn to driver tx_prepare calls

mt7615 and newer drivers do not need this, since they use sequence number offload
Moving this code also reduces the number of callsites to make it easier to review

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: do not inject packets if MT76_STATE_PM is set

Do not tx packets in mt76_txq_send_burst() or mt76_txq_schedule_list()
if the device is in runtime-pm

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: mt76u: add mt76_skb_adjust_pad utility routine

Introduce mt76_skb_adjust_pad to reuse the code adding sdio support to
mt7615 driver and remove code duplication. Move 4B header configuration
for usb devices out of mt76_skb_adjust_pad

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: add API for testmode support

This can be used for calibration in the manufacturing process.
It supports sending a configurable number of packets with a specific rate
and configurable tx power levels / antenna settings.
It also supports receiving packets and showing some statistics, including
packet counters and detailed RSSI information.
It will only be compiled in if CONFIG_NL80211_TESTMODE is enabled

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: overwrite qid for non-bufferable mgmt frames

Overwrite hw queue id for non-bufferable management frames if the hw
support always txq (altxq) in order to be in sync with mac txwi code

Fixes: cdad4874057d ("mt76: mt7615: add dma and tx queue initialization for MT7622")
Fixes: f40ac0f3d3c0 ("mt76: mt7615: introduce mt7663e support")
Suggested-by: Felix Fietkau <nbd@nbd.name>
Tested-by: Sean Wang <sean.wang@mediatek.com>
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: overwrite qid for non-bufferable mgmt frames

Overwrite hw queue id for non-bufferable management frames if the hw
support always txq (altxq) in order to be in sync with mac txwi code

Fixes: cdad4874057d ("mt76: mt7615: add dma and tx queue initialization for MT7622")
Fixes: f40ac0f3d3c0 ("mt76: mt7615: introduce mt7663e support")
Suggested-by: Felix Fietkau <nbd@nbd.name>
Tested-by: Sean Wang <sean.wang@mediatek.com>
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: remove unnecessary annotations

Sparse report warnings at mt76_tx_status_unlock() and mt76_tx_status_lock()

warning: context imbalance in mt76_tx_status_lock() - wrong count at exit
warning: context imbalance in mt76_tx_status_unlock() - unexpected unlock

The root cause is the additional __acquire(&dev->status_list.lock)
and __release(&dev->status_list.unlock) called
from inside mt76_tx_status_lock() and mt76_tx_status_unlock().

Remove __acquire(&dev->status_list.lock) annotation
Remove __releases(&dev->status_list.unlock)
Correct &dev->status_list.unlock to &dev->status_list.lock
-unlock not defined in the sk_buff_head struct

Signed-off-by: Jules Irenge <jbi.octave@gmail.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: move state from struct mt76_dev to mt76_phy

Allows keeping per-wiphy state separate

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: add support for an extra wiphy in the tx status path

This is preparation for supporting multiple wiphys per device to support the
concurrent dual-band feature of MT7615D

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: add support for an extra wiphy in the main tx path

This is preparation for supporting multiple wiphys per device to support the
concurrent dual-band feature of MT7615D

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: remove empty flag in mt76_txq_schedule_list

Remove empty flag in mt76_txq_schedule_list and mt76_txq_send_burst
since we just need retry_q length to notify mac80211 to reschedule the
current tx queue

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: remove offchannel check in tx scheduling

tx queues are already disabled by mac80211 during scanning or other
off-channel activity. There is no need to repeat the check in mt76,
since scheduled queues are selected by mac80211 as well.

Signed-off-by: Balakrishna Bandi <b.balakrishna@globaledgesoft.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: fix some checkpatch warnings

This fixes the following checkpatch warnings:
CHECK: Alignment should match open parenthesis
CHECK: No space is necessary after a cast

Signed-off-by: Ryder Lee <ryder.lee@mediatek.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: move mt76_tx_tasklet in mt76 module

Move mt76{15,03}_tx_tasklet in mt76_alloc_device in order to be used as
default tx_tasklet initialization. Remove duplicated code in
mt7603/mt7615 drivers

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: switch to SPDX tag instead of verbose boilerplate text

No functional change intended.

Add SPDX identifiers to all remaining files in /mt76.

Signed-off-by: Ryder Lee <ryder.lee@mediatek.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: dma: introduce skb field in mt76_txwi_cache

Introduce skb field in mt76_txwi_cache. Moreover add txwi_flags
to mt76_driver_ops since new chipsets will release mt76_txwi_cache/skbs
at tx completion instead of dma one.
This is a preliminary patch to add mt7615 support

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: set txwi_size according to the driver value

Dynamically allocate txwi since new chipsets will use longer txwi
descriptors

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: only schedule txqs from the tx tasklet

Reduces lock contention from the tx path and improves performance

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: store wcid tx rate info in one u32 reduce locking

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: use mac80211 txq scheduling

Performance improvement and preparation for adding airtime fairness support

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: introduce mt76_txq_id field in mt76_queue_entry

Add mt76_txq_id field to mt76_queue_entry in order to properly
track outstanding frames for mt7615 that relies on a single hw queue

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: introduce mt76_sw_queue data structure

Introduce mt76_sw_queue data structure in order to support new
chipsets (e.g. mt7615) that have a shared hardware queue for all traffic
identifiers. mt76_sw_queue will be used to track outstanding packets

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: remove mt76_queue dependency from tx_queue_skb function pointer

Remove mt76_queue dependency from tx_queue_skb function pointer and
rely on mt76_tx_qid instead. This is a preliminary patch to introduce
mt76_sw_queue support

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: when releasing PS frames, end the service period if no frame was found

Fixes a rare corner case if the txq dequeue attempt fails, but mac80211
still has PS buffered packets

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: mt76x2: implement full device restart on watchdog reset

Restart the firmware and re-initialize the MAC to be able to recover
from more kinds of hang states

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: introduce q->stopped parameter

Introduce mt76_queue stopped parameter in order to run
ieee80211_wake_queue only when mac80211 queues have been
previously stopped and avoid to disable interrupts when
it is not necessary

Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: fix tx status timeout processing

Remove bogus check for MT_PACKET_ID_NO_ACK in mt76_tx_status_skb_get, which
is already handled in callers and turns timeout calls into no-ops
Do not clean up pending status items on reordering of tx status information
if the timeout is not reached yet

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: avoid scheduling tx queues for powersave stations

In case a tx queue wake call arrives after a client has transitioned to
powersave, make sure that the queue is not kept active until the client
has left powersave mode

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: fix tx status reporting for non-probing frames

On MT76x2, the hardware does not report tx status in the FIFO register,
if the packet id is 0.
Change the allocation of packet IDs to use 0 for no-ack packets, 1 for
non-probing packets and 2-255 for packets with tx status requested.
Fixes rate control issues

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: request tx status for powersave released EOSP packet

Allows mac80211 to keep track of the service period

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: fix potential NULL pointer dereference in mt76_stop_tx_queues

Starting from mac80211 commit adf8ed01e4fd ("mac80211: add an optional
TXQ for other PS-buffered frames") and commit 0eeb2b674f05 ("mac80211:
add an option for station management TXQ") a new per-sta queue has been
introduced for bufferable management frames.
sta->txq[IEEE80211_NUM_TIDS] is initialized just if the driver reports
the following hw flags:
- IEEE80211_HW_STA_MMPDU_TXQ
- IEEE80211_HW_BUFF_MMPDU_TXQ
This can produce a NULL pointer dereference in mt76_stop_tx_queues
since mt76 iterates on all available sta tx queues assuming they are
initialized by mac80211. This issue has been spotted analyzing the code
(it has not triggered any crash yet)

Signed-off-by: Lorenzo Bianconi <lorenzo.bianconi@redhat.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

mt76: fix race condition in station removal

If there are still pending packets in the tx queue when removing a station,
it could possibly lead to a call to further attempts to pull packets from
the mac80211 tx queue after it has already been removed from the scheduling
list.
Prevent this from happening by calling synchronize_rcu after deleting the
wcid pointer before further cleaning up the tx queues.
To be extra careful, ensure that mtxq->list is always initialized properly.

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: avoid queue/status spinlocks while passing tx status to mac80211

There is some code in the mac80211 tx status processing code that could
potentially call back into the tx codepath.
To avoid deadlocks, make sure that no tx related spinlocks are taken
during the ieee80211_tx_status call.

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: add support for reporting tx status with skb

MT76x2/MT76x0 has somewhat unreliable tx status reporting, and for that
reason the driver currently does not report per-skb tx ack status at all.
This breaks things like client idle polling, which relies on the tx ack
status of a transmitted nullfunc frame.

This patch adds code to report skb-attached tx status if requested by
mac80211 or the rate control module. Since tx status is polled from a
simple FIFO register, the code needs to account for the possibility of
tx status events getting lost.

The code keeps a list of skbs for which tx status is required and passes
them to mac80211 once tx status has been filled in and the DMA queue is
done with it.
If a tx status event is not received after one second, the status rates
are cleared, and a succesful ACK is indicated to avoid spurious disassoc
during assoc or client polling.

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: do not store aggregation sequence number for null-data frames

Fixes a rare corner case where a BlockAckReq might get the wrong
sequence number.

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: check aggregation sequence number for frames sent via drv_tx

ps-poll response frames can be sent via drv_tx. Store the frame sequence number
for such frames, in case a BlockAckReq needs to be sent

Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: unify AC to hw queue mapping

Use the same AC to hardware queue mappings for all subdrivers.
Note: this change BK and BE mappings for USB drivers.

Signed-off-by: Stanislaw Gruszka <sgruszka@redhat.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

mt76: introduce tx_queue_skb function pointer in mt76_bus_ops

Add tx_queue_skb function pointer in mt76_bus_ops since mt76x2u based
devices do not map mt76x2_txwi on dma buffers and it is not possible
to reuse mt76_dma_tx_queue_skb() routine to enqueue tx frames to
hw buffers

Signed-off-by: Lorenzo Bianconi <lorenzo.bianconi@redhat.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

mt76: rename mt76_tx_queue_skb in mt76_dma_tx_queue_skb

Move mt76_dma_tx_queue_skb routine in dma.c.
Remove static qualifier from mt76_get_txwi definition

Signed-off-by: Lorenzo Bianconi <lorenzo.bianconi@redhat.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

mt76: only stop tx queues on offchannel, not during the entire scan

During scans, mac80211 frequently switches back to the home channel to
minimize interruption of ongoing traffic. Keep regular tx queues active
during that time.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

mt76: check for pending reset before attempting to schedule tx

The check within mt76_txq_send_burst is not enough, as it happens after
a first frame has already been queued up

Signed-off-by: Felix Fietkau <nbd@nbd.name>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

mt76: add rcu locking around tx scheduling

Fixes a reported lockdep error in mac80211:

[ 179.867321] =============================
[ 179.871510] WARNING: suspicious RCU usage
[ 179.875528] 4.14.32 #0 Not tainted
[ 179.878924] -----------------------------
[ 179.882981] backports-2017-11-01/net/mac80211/tx.c:594 suspicious rcu_dereference_check() usage!
[ 179.891785]
[ 179.891785] other info that might help us debug this:
[ 179.891785]
[ 179.899824]
[ 179.899824] rcu_scheduler_active = 2, debug_locks = 1
[ 179.906343] 2 locks held by ksoftirqd/0/7:
[ 179.910479] #0: (&(&q->lock)->rlock){+.-.}, at: [<86b207a4>] mt76_dma_tx_cleanup+0x64/0x354 [mt76]
[ 179.919734] #1: (&(&fq->lock)->rlock){+.-.}, at: [<87238410>] ieee80211_tx_dequeue+0x54/0xc3c [mac80211]
[ 179.929890]
[ 179.929890] stack backtrace:
[ 179.934257] CPU: 0 PID: 7 Comm: ksoftirqd/0 Not tainted 4.14.32 #0
[ 179.940421] Stack : 00000000 00000000 00000000 00000000 80e0fce2 00000036 00000000 00000000
[ 179.948864] 87c3d24c 80696377 8061039c 00000000 00000007 00000001 87c5db78 6534689d
[ 179.957306] 00000000 00000000 80e10000 87c5da74 00000001 0000015a 00000007 00000000
[ 179.965748] 00000000 806a0000 000e4171 00000000 00000000 00000000 ffffffff 00000001
[ 179.974189] 806c0000 8692b240 86b000d0 87316fe4 00000001 802c9a68 00000000 80700000
[ 179.982632] ...
[ 179.985104] Call Trace:
[ 179.987582] [<80010a48>] show_stack+0x58/0x100
[ 179.992040] [<804c2c58>] dump_stack+0xe8/0x170
[ 179.996868] [<87234a04>] ieee80211_tx_h_select_key+0xa8/0x5b8 [mac80211]
[ 180.004299] [<87238d44>] ieee80211_tx_dequeue+0x988/0xc3c [mac80211]
[ 180.011048] [<86b230dc>] mt76_txq_schedule+0x110/0x3a4 [mt76]
[ 180.016821] [<86b209d0>] mt76_dma_tx_cleanup+0x290/0x354 [mt76]
[ 180.022777] [<86be2e60>] mt7603_tx_tasklet+0x40/0x6c [mt7603e]
[ 180.028637] [<80037058>] tasklet_action+0x110/0x1ec
[ 180.033532] [<804e1dac>] __do_softirq+0x164/0x35c
[ 180.038235] [<80037174>] run_ksoftirqd+0x40/0x84
[ 180.042870] [<800580c8>] smpboot_thread_fn+0x1a8/0x1d8
[ 180.048023] [<800542e8>] kthread+0x130/0x144
[ 180.052297] [<8000b1f8>] ret_from_kernel_thread+0x14/0x1c

Signed-off-by: Felix Fietkau <nbd@nbd.name>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

mt76: add common code shared between multiple chipsets

This will be used by drivers for MT76x2e, MT7603e and MT7628

Signed-off-by: Felix Fietkau <nbd@nbd.name>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>