wifi: rtw89: update DMA function with different generation

The register of control and polling function for TX/RX DMA is different
from different generation, so update them. Also rename polling_dma
function to polling_dma_idle to avoid misunderstanding.

Signed-off-by: Chin-Yen Lee <timlee@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://msgid.link/20240302005828.13666-4-pkshih@realtek.com

wifi: rtw89: pci: implement PCI CLK/ASPM/L1SS for WiFi 7 chips

PCI CLK/ASPM/L1SS is power management mechanism used to reduce power
consumption of PCI chip. The registers for setting of these features
in WiFi 7 Chip are different from WiFi 6 chip, so separate them
in generation information.

Signed-off-by: Chin-Yen Lee <timlee@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://msgid.link/20240222064258.59782-4-pkshih@realtek.com

wifi: rtw89: fw: read firmware secure information from efuse

To support firmware secure boot, read secure information from efuse to
know if current hardware module can support secure boot with certain
cryptography method.

This information should be prepared before downloading firmware, so read
efuse right after power on at probe stage. The secure information includes
secure cryptography method and secure key index that are used to choose
proper key material when downloading firmware.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://msgid.link/20240204012627.9647-3-pkshih@realtek.com

wifi: rtw89: pci: validate RX tag for RXQ and RPQ

PCI RX ring is a kind of read/write index ring, and DMA and ring index are
asynchronous, so suddenly driver gets newer index ahead before DMA. To
resolve this rare situation, we use a RX tag as helpers to make sure DMA
is done.

The RX tag is a 13-bit value, and range is from 1 ~ 0x1FFF, but 0 isn't
used so should be skipped.

Only enable this validation to coming WiFi 7 chips, because existing
chips use different design and don't really meet this situation.

Add missed rx_ring_eq_is_full for 8851BE by the way.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://msgid.link/20240121071826.10159-4-pkshih@realtek.com

wifi: rtw89: pci: interrupt v2 refine IMR for SER

During SER (system error recovery), expect to deal with
only ISR related to halt. So, refine IMR configuration.

Signed-off-by: Zong-Zhe Yang <kevin_yang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://msgid.link/20240121071826.10159-3-pkshih@realtek.com

wifi: rtw89: pci: use DBI function for 8852AE/8852BE/8851BE

Sometimes driver can't use kernel API pci_read/write_config_byte
to access the PCI config space of above address 0x100 due to
the negotiated PCI setting. 8852AE/8852BE/8851BE provide another
way called DBI function, which belongs to WiFi mac and could
access all PCI config space for this case.

Link: https://lore.kernel.org/linux-wireless/79fe81b7db7148b9a7da2353c16d70fb@realtek.com/T/#t
Signed-off-by: Chin-Yen Lee <timlee@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://msgid.link/20240103012346.6822-1-pkshih@realtek.com

wifi: rtw89: 8922a: dump MAC registers when SER occurs

To diagnose the reason why firmware or hardware get abnormal, add to dump
MAC registers related to counters and interrupt masks. With these values,
people can classify problems and check if registers values are unexpected,
and then correct them. However, it could possible false alarm because
firmware triggers this SER event by wrong conditions that we should
correct it at firmware or register settings.

In field, SER might happen under special conditions, and very hard to
happen again, so dump lots of registers to provide rich information to
catch the problem.

Signed-off-by: Chia-Yuan Li <leo.li@realtek.com>
Signed-off-by: Zong-Zhe Yang <kevin_yang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20231204080751.15354-5-pkshih@realtek.com

wifi: rtw89: pci: update interrupt mitigation register for 8922AE

To reduce interrupts, configure the mitigation setting of 8922AE when
bringing interface up, and then check situations to decide turning on or
off the function. With this, interrupt count decreases to 20,141 from
202,141 in period of 20 seconds.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20231110012319.12727-8-pkshih@realtek.com

wifi: rtw89: pci: correct interrupt mitigation register for 8852CE

To reduce interrupt count, configure mitigation register with thresholds
of time and packet count. We missed that 8852CE uses different register
address, so correct it. Then, interrupt counts down to 30,763 from 229,825
during stress test in 20 seconds.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20231110012319.12727-7-pkshih@realtek.com

wifi: rtw89: 8922ae: add v2 interrupt handlers for 8922AE

The handlers include three parts -- 1) configure interrupt mask;
2) enable/disable interrupt; 3) recognize (read) interrupt status.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20231110012319.12727-6-pkshih@realtek.com

wifi: rtw89: pci: generalize interrupt status bits of interrupt handlers

For WiFi 7, interrupt status registers and their definitions are changed
a lot, but the logic is still the same, so define fields to reuse the code.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20231110012319.12727-5-pkshih@realtek.com

wifi: rtw89: pci: add pre_deinit to be called after probe complete

At probe stage, we only do partial initialization to enable ability to
download firmware and read capabilities. After that, we use this pre_deinit
to disable HCI to save power.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20231110012319.12727-4-pkshih@realtek.com

wifi: rtw89: pci: stop/start DMA for level 1 recovery according to chip gen

Level 1 recovery is to recover TX/RX rings, so it needs PCI to stop/start
DMA. But, different chip gen have different implementations, either
register address/mask or function flow. So, configure callback of
stop/start DMA by chip gen.

Signed-off-by: Zong-Zhe Yang <kevin_yang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20231110012319.12727-3-pkshih@realtek.com

wifi: rtw89: pci: reset BDRAM according to chip gen

Configure callback of reset BDRAM (buffer descriptor RAM) by chip gen.
Refine the one of 802.11ax chip gen and drop a redundant duplicate of it
in 802.11ax chip gen. Then, assign right callback of rst_bdram for HCI ops
which needs to do callback according to chip gen.

Signed-off-by: Zong-Zhe Yang <kevin_yang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20231110012319.12727-2-pkshih@realtek.com

wifi: rtw89: pci: implement PCI mac_pre_init for WiFi 7 chips

Call this function when doing MAC initialization at probe stage. It does
partial initial registers only, because we only need basic ability to
download firmware. The function to clear index is the sub-function,
so set its pointer as well.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20231101072149.21997-4-pkshih@realtek.com

wifi: rtw89: pci: use gen_def pointer to configure mac_{pre,post}_init and clear PCI ring index

Use gen_def pointer to call three WiFi 6 specific functions, and add _ax
suffix to them. Then, we will implement functions for WiFi 7 chips later.

The mac_{pre,post}_init are used to initialize PCI during doing MAC
initialization, and clear PCI ring index to make index consistent between
driver, firmware and hardware.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20231101072149.21997-3-pkshih@realtek.com

wifi: rtw89: pci: add PCI generation information to pci_info for each chip

In order to reuse PCI initial and configuration flows, add struct
rtw89_pci_gen_def to abstract the differences between WiFi 6/7 generations.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20231101072149.21997-2-pkshih@realtek.com

wifi: rtw89: pci: generalize code of PCI control DMA IO for WiFi 7

The register to enable/disable PCI DMA IO has many variants, so define
and use a field to control it accordingly.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20231026120049.9187-5-pkshih@realtek.com

wifi: rtw89: pci: add new RX ring design to determine full RX ring efficiently

To make hardware efficient to determine if RX ring is full, introduce new
design that checks if reading and writing indices are equal. Comparing
to old design, initial indices of both reading and writing indices are 0
that means empty, and hardware checks full by "writing index + 1 ==
reading index". The "+1" has extra cost for hardware, so new design is
to avoid this.

Take ring size is 256 as an example, the initial reading and writing
indices are 255 and 0 respectively; the initial values mean empty. If two
indices are the same, for example 5 and 5, it means ring is full.

wp rp used_cnt state
255 0 0 initial (ring is empty)
255 1 1 receive 1st packet
255 2 2 receive 2nd packet
0 2 1 driver read 1st packet
1 2 0 driver read 2nd packet (ring is empty)
:
5 5 255 ring is full

Note: 'rp' is hardware writing index

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20231026120049.9187-4-pkshih@realtek.com

wifi: rtw89: pci: define PCI ring address for WiFi 7 chips

PCI rings are used to DMA TX/RX packets. The address of WiFi 7 chips are
different from previous ones, so add them according to hardware design.
Another difference is that driver doesn't need to configure BD (buffer
descriptor) RAM table, which is used by hardware to fetch BD ahead before
fetching whole TX data.

A TX ring contains numbers of TX BD (e.g. 512):

TX BD (buffer descriptor; continual memory)
+---+---+---+---+ +---+
| | | | | ... | |
+-|-+---+---+---+ +---+
|
| point to TX WD (WiFi descriptor; metadata of a skb data)
v
+------+
| |
| |
+-|----+
|
| point to a skb data
v
+------+
| |
| |
+------+

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20231026120049.9187-3-pkshih@realtek.com

wifi: rtw89: add chip_info::txwd_info size to generalize TX WD submit

For existing chips, size of TX WD info is 6 words, but upcoming WiFi 7
chips become 8 words, so add a chip_info to reuse the code.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20230911082049.33541-5-pkshih@realtek.com

Fix nomenclature for USB and PCI wireless devices

A mouse that uses a USB connection is called a "USB mouse" device (or
"USB mouse" for short), not a "mouse USB" device. By analogy, a WiFi
adapter that connects to the host computer via USB is a "USB wireless"
device, not a "wireless USB" device. (The latter term more properly
refers to a defunct Wireless USB specification, which described a
technology for sending USB protocol messages over an ultra wideband
radio link.)

Similarly for a WiFi adapter card that plugs into a PCIe slot: It is a
"PCIe wireless" device, not a "wireless PCIe" device.

Rephrase the text in the kernel source where the word ordering is
wrong.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/57da7c80-0e48-41b5-8427-884a02648f55@rowland.harvard.edu

wifi: rtw89: add chip_ops::query_rxdesc() and rxd_len as helpers to support newer chips

The next generation chips use different RX descriptor format, so add
a chip_ops to hook suitable handlers. Also, the length of RX descriptor is
different, so add a variable to store the length according to chip and
descriptor content dynamically. Then, the code can be more general.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20230522122513.13559-2-pkshih@realtek.com

wifi: rtw89: pci: fix interrupt enable mask for HALT C2H of RTL8851B

RTL8851B keeps almost the same interrupt flow as RTL8852A and RTL8852B.
But, it uses a different bitmask for interrupt indicator of FW HALT C2H.
So, we make a chip judgement in pci when configuring interrupt mask.

Signed-off-by: Zong-Zhe Yang <kevin_yang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20230508084335.42953-2-pkshih@realtek.com

wifi: rtw89: use chip_info::small_fifo_size to choose debug_mask

Previously, 8852B has smaller FIFO size than others, so I use chip_id to
choose debug_mask before. 8851B has similar design, so add a field to
chip_info as a general expression.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20230421024551.29994-6-pkshih@realtek.com

wifi: rtw89: pci: update PCI related settings to support 8851B

Many settings of 8851B are like 8852A or 8852B. Change them to proper
settings as hardware design.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20230330133324.19538-5-pkshih@realtek.com

wifi: rtw89: add function to wait for completion of TX skbs

Allocate a per-skb completion to track those skbs we are interested in
and wait for them to complete transmission with TX status.

Normally, the completion object is freed by wait side, but it could be
timeout result that complete side should free the object instead. Add a
owner field with RCU to determine which side should free the object.

Signed-off-by: Po-Hao Huang <phhuang@realtek.com>
Signed-off-by: Zong-Zhe Yang <kevin_yang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20230411124832.14965-3-pkshih@realtek.com

wifi: rtw89: Remove redundant pci_clear_master

Remove pci_clear_master to simplify the code,
the bus-mastering is also cleared in do_pci_disable_device,
like this:
./drivers/pci/pci.c:2197
static void do_pci_disable_device(struct pci_dev *dev)
{
u16 pci_command;

pci_read_config_word(dev, PCI_COMMAND, &pci_command);
if (pci_command & PCI_COMMAND_MASTER) {
pci_command &= ~PCI_COMMAND_MASTER;
pci_write_config_word(dev, PCI_COMMAND, pci_command);
}

pcibios_disable_device(dev);
}.
And dev->is_busmaster is set to 0 in pci_disable_device.

Signed-off-by: Cai Huoqing <cai.huoqing@linux.dev>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Acked-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20230323112613.7550-5-cai.huoqing@linux.dev

wifi: rtw89: fix potential race condition between napi_init and napi_enable

A race condition can happen if netdev is registered, but NAPI isn't
initialized yet, and meanwhile user space starts the netdev that will
enable NAPI. Then, it hits BUG_ON():

kernel BUG at net/core/dev.c:6423!
invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
CPU: 0 PID: 417 Comm: iwd Not tainted 6.2.7-slab-dirty #3 eb0f5a8a9d91
Hardware name: LENOVO 21DL/LNVNB161216, BIOS JPCN20WW(V1.06) 09/20/2022
RIP: 0010:napi_enable+0x3f/0x50
Code: 48 89 c2 48 83 e2 f6 f6 81 89 08 00 00 02 74 0d 48 83 ...
RSP: 0018:ffffada1414f3548 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffa01425802080 RCX: 0000000000000000
RDX: 00000000000002ff RSI: ffffada14e50c614 RDI: ffffa01425808dc0
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000100 R12: ffffa01425808f58
R13: 0000000000000000 R14: ffffa01423498940 R15: 0000000000000001
FS: 00007f5577c0a740(0000) GS:ffffa0169fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5577a19972 CR3: 0000000125a7a000 CR4: 0000000000750ef0
PKRU: 55555554
Call Trace:
<TASK>
rtw89_pci_ops_start+0x1c/0x70 [rtw89_pci 6cbc75429515c181cbc386478d5cfb32ffc5a0f8]
rtw89_core_start+0xbe/0x160 [rtw89_core fe07ecb874820b6d778370d4acb6ef8a37847f22]
rtw89_ops_start+0x26/0x40 [rtw89_core fe07ecb874820b6d778370d4acb6ef8a37847f22]
drv_start+0x42/0x100 [mac80211 c07fa22af8c3cf3f7d7ab3884ca990784d72e2d2]
ieee80211_do_open+0x311/0x7d0 [mac80211 c07fa22af8c3cf3f7d7ab3884ca990784d72e2d2]
ieee80211_open+0x6a/0x90 [mac80211 c07fa22af8c3cf3f7d7ab3884ca990784d72e2d2]
__dev_open+0xe0/0x180
__dev_change_flags+0x1da/0x250
dev_change_flags+0x26/0x70
do_setlink+0x37c/0x12c0
? ep_poll_callback+0x246/0x290
? __nla_validate_parse+0x61/0xd00
? __wake_up_common_lock+0x8f/0xd0

To fix this, follow Jonas' suggestion to switch the order of these
functions and move register netdev to be the last step of PCI probe.
Also, correct the error handling of rtw89_core_register_hw().

Fixes: e3ec7017f6a2 ("rtw89: add Realtek 802.11ax driver")
Cc: stable@vger.kernel.org
Reported-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Link: https://lore.kernel.org/linux-wireless/CAOiHx=n7EwK2B9CnBR07FVA=sEzFagb8TkS4XC_qBNq8OwcYUg@mail.gmail.com/T/#t
Suggested-by: Jonas Gorski <jonas.gorski@gmail.com>
Tested-by: Larry Finger<Larry.Finger@lwfinger.net>
Reviewed-by: Larry Finger<Larry.Finger@lwfinger.net>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20230323082839.20474-1-pkshih@realtek.com

wifi: rtw89: 8852be: enable CLKREQ of PCI capability

Enable CLKREQ to reduce power consumption for 8852BE.

Signed-off-by: Chin-Yen Lee <timlee@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20230126112715.5811-1-pkshih@realtek.com

wifi: rtw89: fix assignation of TX BD RAM table

TX BD's RAM table describes how HW allocates usable buffer section
for each TX channel at fetch time. The total RAM size for TX BD is
chip-dependent. For 8852BE, it has only half size (32) for TX channels
of single band. Original table arrange total size (64) for dual band.
It will overflow on 8852BE circuit and cause section conflicts between
different TX channels.

So, we do the changes below.
* add another table for single band chip and export both kind of tables
* point to the expected one in rtw89_pci_info by chip

Signed-off-by: Zong-Zhe Yang <kevin_yang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20230113090632.60957-4-pkshih@realtek.com

wifi: rtw89: add HE radiotap for monitor mode

With basic HE radiotap, we can check data rate in sniffer data. To store
the radiotap data, we reserve headroom of aligned 64 bytes, and then
update HE radiotap in monitor mode, so it doesn't affect performance in
normal mode.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20221125072416.94752-3-pkshih@realtek.com

wifi: rtw89: 8852b: change debug mask of message of no TX resource

8852B has smaller TX FIFO than others in WiFi chip, so it would be buffer
full frequently, but it doesn't affect TX performance. However, it shows
verbose debug messages with RTW89_DBG_UNEXP mask that is used to indicate
abnormal behavior, so change debug mask to RTW89_DBG_TXRX for 8852B.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20221108014230.11068-1-pkshih@realtek.com

wifi: rtw89: add WoWLAN function support

WoWLAN is a feature which allows devices to be woken up from suspend
state through WLAN events.

When user enables WoWLAN feature and then let the device enter suspend
state, WoWLAN firmware will be loaded by the driver and periodically
monitors WiFi packets. Power consumption of WiFi chip will be reduced
in this state.

We now implement WoWLAN function in rtw8852ae and rtw8852ce chip.
Currently supported WLAN events include receiving magic packet,
rekey packet and deauth packet, and disconnecting from AP.

Signed-off-by: Chin-Yen Lee <timlee@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20221027052707.14605-7-pkshih@realtek.com

wifi: rtw89: pci: set power cut closed for 8852be

Entering LPS with PCIe APHY power cut closed would cause PCIe link issue.
To avoid the combinational issue, keep PCIe APHY power cut always on.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220927062611.30484-7-pkshih@realtek.com

wifi: rtw89: pci: add to do PCI auto calibration

8852be needs this with n times calibration to correct hardware clock.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220927062611.30484-6-pkshih@realtek.com

wifi: rtw89: add DMA busy checking bits to chip info

8852B has less DMA channels, so its checking bits are different from other
chips.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220927062611.30484-4-pkshih@realtek.com

wifi: rtw89: pci: mask out unsupported TX channels

8852BE doesn't support some TX channels, so mask them out, or it access
undefined registers.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220927062611.30484-2-pkshih@realtek.com

wifi: rtw89: pci: update LTR settings

Modify PCI LTR control flow and LTR idle latency to improve power save
efficiency.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220916033811.13862-4-pkshih@realtek.com

wifi: rtw89: pci: concentrate control function of TX DMA channel

Different chips use different register and mask for
tx dma channels, so concentrate them.

Signed-off-by: Chin-Yen Lee <timlee@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220912071706.13619-4-pkshih@realtek.com

wifi: rtw89: correct enable functions of HCI/PCI DMA

Some PCI and MAC registers are changed for different
chips and correct them accordingly. And HCI MAD functions
belongs to MAC core, so move it to mac.h/.c.

Signed-off-by: Chin-Yen Lee <timlee@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220912071706.13619-3-pkshih@realtek.com

wifi: rtw89: 8852c: L1 DMA reset has offloaded to FW

For 8852C, rtw89_pci_lv1rst_stop_dma() and rtw89_pci_lv1rst_start_dma()
are offloaded to FW L1 reset flow. So, driver no longer needs to do them.

Signed-off-by: Zong-Zhe Yang <kevin_yang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220912071706.13619-2-pkshih@realtek.com

wifi: rtw89: pci: correct TX resource checking in low power mode

Number of TX resource must be minimum of TX_BD and TX_WD. Only considering
TX_BD could drop TX packets pulled from mac80211 if TX_WD is unavailable.

Fixes: 52edbb9fb78a ("rtw89: ps: access TX/RX rings via another registers in low power mode")
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220824063312.15784-2-pkshih@realtek.com

wifi: rtw89: pci: fix interrupt stuck after leaving low power mode

We turn off interrupt in ISR, and re-enable interrupt in threadfn or
napi_poll according to the mode it stays. If we are turning off interrupt,
rtwpci->running flag is unset and interrupt handler stop processing even
if it was called, so disallow to re-enable interrupt in this situation.
Or, wifi chip doesn't trigger interrupt events anymore because interrupt
status (ISR) isn't clear by interrupt handler anymore.

Fixes: c83dcd0508e2 ("rtw89: pci: add a separate interrupt handler for low power mode")
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220824063312.15784-1-pkshih@realtek.com

wifi: rtw89: pci: correct suspend/resume setting for variant chips

We find that suspend/resume tests cause 8852CE lost, because some pci
registers are changed for 8852CE. So, correct them accordingly.

Signed-off-by: Chin-Yen Lee <timlee@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220819064811.37700-6-pkshih@realtek.com

wifi: rtw89: pci: enable CLK_REQ, ASPM, L1 and L1ss for 8852c

8852CE controls CLKREQ, ASPM L1, L1ss via wifi registers
instead, so change them accordingly.

Signed-off-by: Chin-Yen Lee <timlee@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220819064811.37700-5-pkshih@realtek.com

wifi: rtw89: pci: fix PCI PHY auto adaption by using software restore

There is chance that PCI PHY auto adaption fail. When first time boot up,
software restore the right adaption value and close PHY auto adaption
mechanism.

Signed-off-by: Chia-Yuan Li <leo.li@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220819064811.37700-4-pkshih@realtek.com

rtw89: 8852c: disable dma during mac init

Without this patch, our hardware attempts to perform dma while device
cpu restarts, and leads to iommu page faults caused by invalid requests.
Some platforms show warning messages as below:
rtw89_8852ce 0000:01:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT
domain=0x000a address=0x10000000004 flags=0x0030]

Signed-off-by: Po-Hao Huang <phhuang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220704023453.19935-2-pkshih@realtek.com

wifi: rtw89: declare support for mac80211 chanctx ops by chip

Some HW features are required if we hook chanctx ops to mac80211. With it,
mac80211 would expect the HW-supported variant ops exists on some behavior,
e.g. HW scan. But, HW features may depend on chip's FW or its development.
Besides, how many chanctx can be supported also depends on chip design.
We can neither decide whether to generally support chanctx ops nor how many
chanctx can be supported.

So, support_chanctx_num is added under chip info to deal with this by chip.
For now, all chip configure support_chanctx_num as 0. We haven't really
hook chanctx ops yet. So, chip can run without mac80211 chanctx as before.

Signed-off-by: Zong-Zhe Yang <kevin_yang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220809104952.61355-13-pkshih@realtek.com

wifi: rtw89: 8852a: adjust IMR for SER L1

SER (system error recovery) L1 (level 1) has a step-by-step handshake
process with FW. These handshakes still rely on B_AX_HS0ISR_IND_INT_EN.
So, even already during recovery, we enable this bit in IMR.

Signed-off-by: Zong-Zhe Yang <kevin_yang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220721074952.19676-1-pkshih@realtek.com

wifi: rtw89: pci: fix PCI doesn't reclaim TX BD properly

TX BD (TX ring index) and TX WD (WiFi descriptor buffer) are freed
asynchronously. With burst packets, we free TX WD, but the corresponding
TX BD couldn't be freed yet. Then, TX can possibly get stuck due to no
more TX BD.

To avoid this, ignore reclaiming TX BD only if TX WD is no free space,
because at this moment TX BD must have some spaces. Otherwise, we reclaim
TX BD to resolve TX stuck issue.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220610072610.27095-11-pkshih@realtek.com

wifi: rtw89: add UNEXP debug mask to keep monitor messages unexpected to happen frequently

Some warning messages could bother users. With proper handling, these
situations don't really affect usage, but we still need to keep monitor
these messages. If they happen frequently, we must review driver or
hardware design to clarify.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220610072610.27095-8-pkshih@realtek.com

rtw89: pci: handle hardware watchdog timeout interrupt status

This watchdog timeout status bit indicates hardware gets wrong, so run SER
L2 flow that calls mac80211 to restart hardware.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220520071731.38563-4-pkshih@realtek.com

rtw89: pci: only mask out INT indicator register for disable interrupt v1

The design of INT indicator register (R_AX_PCIE_HIMR00_V1) is to reduce IO
during frequent interrupts, because it can stop chip sending interrupt to
host if we just set this indicator to 0, not all IMR(s). This indicator
register looks like a root interrupt controller of wifi chip.

However, we can't set all other IMR(s) to 0 during we are running on
interrupt service routine, or the indicator register can't reflect the
status of certain interrupt happened during this period, and then miss
some interrupts especially SER interrupt events.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220516005215.5878-7-pkshih@realtek.com

rtw89: pci: allow to process RPP prior to TX BD

RPP is to report certain skb(s) can be freed, and TX BD indicates which
TX descriptors can be freed. Normally, TX BD is happened before RPP.
In low power mode, RPP can happen ahead, so change flow to handle this
case.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220421120903.73715-8-pkshih@realtek.com

rtw89: ps: access TX/RX rings via another registers in low power mode

In low power mode, we need to pause PCI to configure IMR and PCI ring
index registers accordingly, because the regular registers are power-off
in this mode.

In the transition moment named paused in code, we can't touch ring index,
so don't kick off DMA immediately. Instead, queue them into pending queue,
and kick off after the moment.

There are three low power modes, which are RF off/clock gate/power gate,
but PCI enter low power mode in later two modes only. So, add a mask
to achieve this.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220421120903.73715-7-pkshih@realtek.com

rtw89: ser: re-enable interrupt in threadfn if under_recovery

Normally, we re-enable interrupt by napi_poll, but for this special
situation, we must turn it on immediately because napi_poll isn't
scheduled.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220421120903.73715-6-pkshih@realtek.com

rtw89: pci: add a separate interrupt handler for low power mode

In lower power mode, there are very low amount of RX, and it must process
in a separated function instead of schedule_napi(), because the existing
napi_poll does many things to optimize performance, but not all registers
can access in low power mode. The simple way is to use threadfn to process
the simple thing.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220421120903.73715-5-pkshih@realtek.com

rtw89: pci: reclaim TX BD only if it really need

To reclaim TX BD, we need to read hardware reading index to determine if
any DMA is complete. Since this IO spends time, do this thing only if we
really need it when TX BD has no free buffer corresponding to target skb.

The experimental result shows that reading counter decreases from 26,000
to 130 per second.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220421120903.73715-4-pkshih@realtek.com

rtw89: pci: add variant RPWM/CPWM to enter low power mode

RPWM/CPWM are registers that can set and check low power mode. Since chips
use different address, add a field to access them in common flow.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220421120903.73715-3-pkshih@realtek.com

rtw89: pci: add variant IMR/ISR and configure functions

8852CE uses different but similar IMR/ISR registers, and its masks are also
different in various states, so add config_intr_mask ops to configure masks
according to under_recovery or low_power states.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220421120903.73715-2-pkshih@realtek.com

rtw89: pci: correct return value handling of rtw89_write16_mdio_mask()

Fix wrong checking statement. Fortunately, this wrong code doesn't affect
existing chip.

Fixes: 740c431c22fe ("rtw89: pci: add register definition to rtw89_pci_info to generalize pci code")
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220413010804.8941-5-pkshih@realtek.com

rtw89: pci: add L1 settings

Configure L1 settings of enter and exit.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220325060055.58482-8-pkshih@realtek.com

rtw89: pci: add deglitch setting

Add setting to support 8852ce.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220325060055.58482-7-pkshih@realtek.com

rtw89: pci: set address info registers depends on chips

Address info registers are used to configure size of DMA address info to
point skb->data. With different size, it can support different number of
scatters.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220325060055.58482-6-pkshih@realtek.com

rtw89: pci: add LTR setting for v1 chip

Add LTR handle to PCI deinit as well.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220325060055.58482-5-pkshih@realtek.com

rtw89: pci: refine pci pre_init function

The pre_init is used to initialize partial PCI function during PCI probe.
It doesn't need to initialize all functions, so probe can be faster.

Signed-off-by: Chia-Yuan Li <leo.li@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220325060055.58482-4-pkshih@realtek.com

rtw89: pci: add pci attributes to configure operating mode

Refine operating mode function to support variant chips.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220325060055.58482-3-pkshih@realtek.com

rtw89: pci: add register definition to rtw89_pci_info to generalize pci code

The PCI code of 8852AE and 8852CE are different, but the flow and register
names are similar. To reuse the code, add a struct to define register or
value accordingly. We also use chip id to control the slightly different
flow.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220325060055.58482-2-pkshih@realtek.com

rtw89: support variant of fill_txdesc

The txdesc is descriptor related to skb->data. The v1 version contains
8 dwords txwd_body and 6 dwords txwd_info, and the format is also different
from original one.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220318023214.32411-10-pkshih@realtek.com

rtw89: pci: support variant of fill_txaddr_info

The txaddr_info is used to fill the DMA address of skb->data. The v1
version can support up to 10 entries, but the maximum size of each entry
is 2047, so it fill more than one entry for large packet, like 3000 bytes.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220318023214.32411-9-pkshih@realtek.com

rtw89: add chip_info::h2c_desc_size/fill_txdesc_fwcmd to support new chips

8852A and 8852C use different H2C header and size, so add h2c_desc_size
to allocate different header size and fill content by fill_txdesc_fwcmd.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220318023214.32411-8-pkshih@realtek.com

rtw89: ser: control hci interrupts on/off by state

While SER (system error recover) is processing, it's supposed to mean
something is under recovery. So, disable interrupts (excluding the one
of halt which could be used during SER) to avoid unexpected behavior.
And then, enable interrupts after SER is done.

Signed-off-by: Zong-Zhe Yang <kevin_yang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220314071250.40292-5-pkshih@realtek.com

rtw89: pci: use a struct to describe all registers address related to DMA channel

We have had a struct rtw89_pci_ch_dma_addr to describe register address,
so use it as regular. Since the addresses should be changed dynamically
according to operating mode, I don't change it to be constant.

These changes don't affect the logic, so I put them in this separated
patch.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220307060457.56789-5-pkshih@realtek.com

rtw89: pci: add V1 of PCI channel address

8852CE use V1 address, and flow is totally shared with 8852AE.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220307060457.56789-4-pkshih@realtek.com

rtw89: pci: add struct rtw89_pci_info

Use this struct to implement chip::ops related to PCI interface.

Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220307060457.56789-3-pkshih@realtek.com

rtw89: use pci_read/write_config instead of dbi read/write

In the past we use dbi function of wifi mac to read/write
pci config space, but the function will be remove in new
chip. So use kernel api pci_read/write_config_byte instead.

Signed-off-by: Chin-Yen Lee <timlee@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20220121075555.12457-2-pkshih@realtek.com

rtw89: extract modules by chipset

We are planning to support more chipsets, e.g. 8852C. Before that, we
consider architecutre to handle multiple kinds of chipsets. Obviosuly,
based on original design, rtw89_core module will have large size even
if there is only one chipset under running. It is because all chipset
related things are put in rtw89_core now. To reduce such overhead, we
extract modules of rtw89 and adjust dependencies between modules.

The following assumes that 8852AE, 8852AU, and 8852CE are all supported,
we describe the difference before and after extraction.

[Before extraction]
-------------
|------------------------------------ | rtw89_usb |
V -------------
--------------------------------------- -------------
| rtw89_core (including 8852A, 8852C) | <--- | rtw89_pci |
--------------------------------------- -------------
The data of 8852A and 8852C are built in rtw89_core.
And rtw89_pci is the entry of 8852AE and 8852CE.
And rtw89_usb is the entry of 8852AU.

[After extraction]
------------- ----------------
|----------- | rtw89_usb | <-------- | rtw89_8852au |
| ------------- ----------------
V --------------- |
-------------- | | <---------------
| rtw89_core | <--- | rtw89_8852a |
-------------- | | <---------------
^ ^ --------------- |
| | ------------- ----------------
| | | | <-------- | rtw89_8852ae |
| |----------- | rtw89_pci | ----------------
| | | <-----------------
| ------------- |
| --------------- ----------------
|--------------- | rtw89_8852c | <------ | rtw89_8852ce |
--------------- ----------------
The data of 8852A/8852C is extracted to rtw89_8852a/rtw89_8852c.
And rtw89_pci/rtw89_usb handles only common flow of pci/usb bus.
Finally, 8852AE, 8852AU, and 8852CE have individual entry modules,
i.e. rtw89_8852ae, rtw89_8852au, and rtw89_8852ce correspondingly.

Signed-off-by: Zong-Zhe Yang <kevin_yang@realtek.com>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://lore.kernel.org/r/20211221025828.25092-1-pkshih@realtek.com

rtw89: remove unneeded semicolon

Eliminate the following coccicheck warning:
./drivers/net/wireless/realtek/rtw89/pci.c:1348:2-3: Unneeded semicolon

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Fixes: e3ec7017f6a2 ("rtw89: add Realtek 802.11ax driver")
Signed-off-by: Yang Li <yang.lee@linux.alibaba.com>
Acked-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>
Link: https://lore.kernel.org/r/1634630094-1156-1-git-send-email-yang.lee@linux.alibaba.com

rtw89: add Realtek 802.11ax driver

This driver named rtw89, which is the next generation of rtw88, supports
Realtek 8852AE 802.11ax 2x2 chip whose new features are OFDMA, DBCC,
Spatial reuse, TWT and BSS coloring; now some of them aren't implemented
though.

The chip architecture is entirely different from the chips supported by
rtw88 like RTL8822CE 802.11ac chip. First of all, register address ranges
are totally redefined, so it's impossible to reuse register definition. To
communicate with firmware, new H2C/C2H format is proposed. In order to have
better utilization, TX DMA flow is changed to two stages DMA. To provide
rich RX status information, additional RX PPDU packets are added.

Since there are so many differences mentioned above, we decide to propose
a new driver. It has many authors, they are listed in alphabetic order:

Chin-Yen Lee <timlee@realtek.com>
Ping-Ke Shih <pkshih@realtek.com>
Po Hao Huang <phhuang@realtek.com>
Tzu-En Huang <tehuang@realtek.com>
Vincent Fann <vincent_fann@realtek.com>
Yan-Hsuan Chuang <tony0620emma@gmail.com>
Zong-Zhe Yang <kevin_yang@realtek.com>

Tested-by: Aaron Ma <aaron.ma@canonical.com>
Tested-by: Brian Norris <briannorris@chromium.org>
Signed-off-by: Ping-Ke Shih <pkshih@realtek.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>
Link: https://lore.kernel.org/r/20211008035627.19463-1-pkshih@realtek.com