wifi: iwlwifi: pcie: don't synchronize IRQs from IRQ

On older devices (before unified image!) we can end up calling
stop_device from an rfkill interrupt. However, in stop_device
we attempt to synchronize IRQs, which then of course deadlocks.

Avoid this by checking the context, if running from the IRQ
thread then don't synchronize. This wouldn't be correct on a
new device since RSS is supported, but older devices only have
a single interrupt/queue.

Fixes: 37fb29bd1f90 ("wifi: iwlwifi: pcie: synchronize IRQs before NAPI")
Reviewed-by: Miri Korenblit <miriam.rachel.korenblit@intel.com>
Reviewed-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Kalle Valo <kvalo@kernel.org>
Link: https://msgid.link/20231215111335.59aab00baed7.Iadfe154d6248e7f9dfd69522e5429dbbd72925d7@changeid

wifi: iwlwifi: pcie: fix RB status reading

On newer hardware, a queue's RB status / write pointer
can be bigger than 4095 (0xFFF), so we cannot mask the
value by 0xFFF unconditionally. Since anyway that's
only necessary on older hardware, move the masking to
the helper function and apply it only for older HW.
This also moves the endian conversion in to handle it
more easily.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Gregory Greenman <gregory.greenman@intel.com>
Link: https://lore.kernel.org/r/20230830112059.7be2a3fff6f4.I94f11dee314a4f7c1941d2d223936b1fa8aa9ee4@changeid
Signed-off-by: Johannes Berg <johannes.berg@intel.com>

wifi: iwlwifi: pcie: fix kernel-doc issues

Fix various missing kernel-doc annotations etc.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Gregory Greenman <gregory.greenman@intel.com>
Link: https://lore.kernel.org/r/20230830112059.a11b39f9a07e.Ia7b189f003db8f6ccaf0a547e71c80e00b85fb5a@changeid
Signed-off-by: Johannes Berg <johannes.berg@intel.com>

wifi: iwlwifi: remove 'def_rx_queue' struct member

This is only ever initialized to zero, use a new define
for the default RX queue instead.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Gregory Greenman <gregory.greenman@intel.com>
Link: https://lore.kernel.org/r/20230816104355.e0c6fa57c162.I907bbb428cf99725f06a348c8dbce5d3dd877136@changeid
Signed-off-by: Johannes Berg <johannes.berg@intel.com>

wifi: iwlwifi: Enable loading of reduce-power tables into several segments

Replace the field reduce_power_dram with a struct that holds data about
the reduced-power tables drams regions. Generalize load_payloads_segments()
to work for both pnvm tables and reduction power tables.
Make required adjustments in the data structures.

Signed-off-by: Alon Giladi <alon.giladi@intel.com>
Signed-off-by: Gregory Greenman <gregory.greenman@intel.com>
Link: https://lore.kernel.org/r/20230606103519.6fe66958f049.I85d80682229fc02fe354462cc9da40937558f30c@changeid
Signed-off-by: Johannes Berg <johannes.berg@intel.com>

wifi: iwlwifi: Implement loading and setting of fragmented pnvm image

Save the pnvm payloads in several DRAM segments (not only in one as
used to). In addition, allocate a FW structure in DRAM that holds the
segments' addresses and forward its address to the FW. It's done when
FW has the capability to handle pnvm images this way (helps to process
large pnvm images).

Signed-off-by: Alon Giladi <alon.giladi@intel.com>
Signed-off-by: Gregory Greenman <gregory.greenman@intel.com>
Link: https://lore.kernel.org/r/20230606103519.dbdad8995ce1.I986213527982637042532de3851a1bd8a11be87a@changeid
Signed-off-by: Johannes Berg <johannes.berg@intel.com>

wifi: iwlwifi: Allow trans_pcie track more than 1 pnvm DRAM region

Change the field pnvm_dram to an array that describes many regions
and add a counter to the number of pnvm regions that were allocated
in DRAM.

Signed-off-by: Alon Giladi <alon.giladi@intel.com>
Signed-off-by: Gregory Greenman <gregory.greenman@intel.com>
Link: https://lore.kernel.org/r/20230606103519.bb206d71bf45.I627640701757bb2f234f8e18a3afbd6af1206658@changeid
Signed-off-by: Johannes Berg <johannes.berg@intel.com>

wifi: iwlwifi: call napi_synchronize() before freeing rx/tx queues

When rx/tx queues are being freed, on a different CPU there could be
still rx flow running. Call napi_synchronize() to prevent such a race.

Signed-off-by: Gregory Greenman <gregory.greenman@intel.com>
Co-developed-by: Benjamin Berg <benjamin.berg@intel.com>
Signed-off-by: Benjamin Berg <benjamin.berg@intel.com>
Link: https://lore.kernel.org/r/20230416154301.5171ee44dcc1.Iff18718540da412e084e7d8266447d40730600ed@changeid
Signed-off-by: Johannes Berg <johannes.berg@intel.com>

iwlwifi: pcie: adjust to Bz completion descriptor

The Bz devices got a new completion descriptor again since
we only ever really used 4 out of 32 bytes anyway. Adjust
the code to deal with that. Note that the intention was to
reduce the size, but the hardware was implemented wrongly.

While at it, do some cleanups and remove the union to simplify
the code, clean up iwl_pcie_free_bd_size() to no longer need
an argument and add iwl_pcie_used_bd_size() with the logic to
selct completion descriptor size.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20220204122220.bef461a04110.I90c8885550fa54eb0aaa4363d322f50e301175a6@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: yoyo: add IMR DRAM dump support

Support debug collection of the platform IMR memory region,
where data is copied by FW during d3 state

Signed-off-by: Mukesh Sisodiya <mukesh.sisodiya@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20220129105618.715f04ecc635.Ib89a6caa06c1324c1c0dd3f9f4cf7407f2857155@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: avoid void pointer arithmetic

Avoid void pointer arithmetic since it's technically
undefined and causes warnings in some places that use
our code.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20220128153014.e349104ecd94.Iadc937f475158b9437becdfefb361a97e7eaa934@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: remove spaces from queue names

If we use spaces in the queue names, we get files with spaces in their
name in procfs, which is ugly. Remove the spaces.

Reported-by: Joe Perches <joe@perches.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210802172232.0ef8aedd4f9b.If527b9ae5bf8de8c6877d4b6a21ed8d81b0f877e@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: dump error on FW reset handshake failures

If the firmware crashes while we're waiting for the reset
handshake then it cannot possibly make progress anymore,
and we will just time out the wait. That's pointless, so
just stop waiting at that point.

Additionally, if it never acknowledges the reset handshake,
something went wrong.

Dump an error in both of these cases, but we need to do it
synchronously here since the device will be turned off.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210802170640.8b6a33544b4b.I55f97f70f8efa64db064a9207177a094c60ac8f1@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: optimise struct iwl_rx_mem_buffer layout

On 64-bit machines, struct iwl_rx_mem_buffer has a lot of
padding due to the use of pointers after the small items.
Move the list entry before them, and while at it also add
documentation for it.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210802170640.6a62255b3df0.I47bb36530a3c2cdbd73454c796ce608ee2a32a6c@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: support loading the reduced power table from UEFI

This new feature allows OEMs to set a special reduced power table in a
UEFI variable, which we use to tell the firmware to change the TX
power tables.

Read the variable and store it in a dram block to pass it to the
firmware. We do this as part of the PNVM loading flow.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210621103449.259a33ba5074.I2e0bb142d2a9c412547cba89b62dd077b328fdc4@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: free IML DMA memory allocation

In the case of gen3 devices with image loader (IML) support,
we were leaking the IML DMA allocation and never freeing it.
Fix that.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210618105614.07e117dbedb7.I7bb9ebbe0617656986c2a598ea5e827b533bd3b9@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: remove TR/CR tail allocations

The TR/CR tail data are meant to be per-queue-arrays, however,
we allocate them completely wrong (we have a separate allocation
per queue).

Looking at this more closely, it turns out that the hardware
never uses these - we have a separate free list per RX queue
and maintain a write pointer for that in a register, and the
RX itself is indicated in the RB status (rb_stts) DMA region.

Despite nothing using the tail pointers, the hardware will
unconditionally access them to write updates, even when we aren't
using CRs/TRs.

Give it dummy values that we never use/update so it can do that
without causing trouble.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210617110647.5f5764e04c46.I4d5de1929be048085767f1234a1e07b517ab6a2d@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: fix some kernel-doc comments

"ubd" is really called "used_bd", fix that.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210617100544.2d4b46c656bb.Iff9ee6a7e65d439169202911dad2cbea626fb887@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: identify the RF module

Identify and print out the RF module to be able to identify
(from logs and through debugfs) which one (and version) is
present on the system.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210617100544.cd1ef97b2c04.Iad42a59902a87a50b45b9ce88705863686a83b54@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: don't enable BHs with IRQs disabled

After the fix from Jiri that disabled local IRQs instead of
just BHs (necessary to fix an issue with submitting a command
with IRQs already disabled), there was still a situation in
which we could deep in there enable BHs, if the device config
sets the apmg_wake_up_wa configuration, which is true on all
7000 series devices.

To fix that, but not require reverting commit 1ed08f6fb5ae
("iwlwifi: remove flags argument for nic_access"), split up
nic access into a version with BH manipulation to use most
of the time, and without it for this specific case where the
local IRQs are already disabled.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>
Link: https://lore.kernel.org/r/iwlwifi.20210415164821.d0f2edda1651.I75f762e0bed38914d1300ea198b86dd449b4b206@changeid

iwlwifi: tx: move handing sync/async host command to trans

Handling host commands in a sync way is not directly related to PCIe
transport, and can serve as common logic for any transport, so move
it to trans layer.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210117164916.fde99af4e0f7.I4cab95919eb35cc5bfb26d32dcf5e15419d0e0ef@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwl-trans: iwlwifi: move sync NMI logic to trans

The code is not directly related to PCIe transport, and it will help
moving sync/async commands logic out of PCIe in the next patches.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210117130510.271f59887fd1.I8ff41236f4e11a25df83d76c982a2a30ba2b9903@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: properly implement NAPI

Instead of pretending to have NAPI and then relying entirely on
interrupts anyway, properly implement NAPI and schedule the poll
when we get an interrupt, re-enabling the interrupt only after
the poll completed.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210117130510.a5951ac4fc06.I9c84a147288fcfb1b019572c6758f2d92949f5d7@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: support firmware reset handshake

There are some races in the hardware that can possibly lead to
a bus lockup later during a restart when we manage to kill the
firmware at a bad time (while it's accessing the bus).

To work around this, add support for a new handshake between
firmware and driver to ensure that the firmware is in a well-
known state before we kill it.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20201209231352.7756fcc9865c.I13de65e0ffcb4186dd4c1a465f66df2e98c9a947@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: use SPDX tags

Use SPDX tags instead of the long copyright notices. Also cleanup
some duplicate copyright notices and combine the years where possible.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20201210000603.481bcb512a6f.I8146abe5a637079e7336209f23cb26af98b12b31@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: remove sw_csum_tx

This was a hack done to test the data path of devices that
didn't support well CSUM offload in Tx. This is not needed
anymore.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20201209231352.6c9fc9fb48d5.I2aaebf90e6fe81860105d049a8d35746fa8d86c2@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: move reclaim flows to the queue file

Reclaim flows are bus-independent TX functions so we move
it to the common place handling bus-independent tx operations

used spatch rule

@@
@@
(
-iwl_trans_pcie_freeze_txq_timer
+iwl_trans_txq_freeze_timer
|
-iwl_trans_pcie_set_q_ptrs
+iwl_trans_txq_set_q_ptrs
|
-iwl_pcie_txq_free_tfd
+iwl_txq_free_tfd
|
-iwl_pcie_txq_progress
+iwl_txq_progress
|
-iwl_trans_pcie_reclaim
+iwl_trans_txq_reclaim
)

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20201209231352.40723e92b6bf.I83cf71d9c6d989ec42f52b353f1d33f32540db59@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: implement set_pnvm op

Implement the set_pnvm op to store the PNVM settings to the context
info and the corresponding code to free the DRAM block when the
context is freed.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>
Link: https://lore.kernel.org/r/iwlwifi.20201008181047.85847cfb0972.I202d90e99779f722df14b2d4102d3e466343a6f6@changeid

iwlwifi: pcie: make iwl_pcie_txq_update_byte_cnt_tbl bus independent

To avoid duplicating code we need to call iwl_pcie_txq_update_byte_cnt_tbl
function from non bus independent code so make it bus independent.

Used spatch rule

@r1@
struct iwl_trans_pcie *trans_pcie;
@@
(
-trans_pcie->scd_bc_tbls
+trans->txqs.scd_bc_tbls
|
-iwl_pcie_txq_update_byte_cnt_tbl
+iwl_txq_gen1_update_byte_cnt_tbl
|
-iwl_pcie_txq_inval_byte_cnt_tbl
+iwl_txq_gen1_inval_byte_cnt_tbl
|
-iwl_pcie_tfd_unmap
+iwl_txq_gen1_tfd_unmap
|
-iwl_pcie_tfd_tb_get_addr
+iwl_txq_gen1_tfd_tb_get_addr
|
-iwl_pcie_tfd_tb_get_len
+iwl_txq_gen1_tfd_tb_get_len
|
-iwl_pcie_tfd_get_num_tbs
+iwl_txq_gen1_tfd_get_num_tbs
)

/* clean all new unused variables */
@ depends on r1@
type T;
identifier i;
expression E;
@@
- T i = E;
... when != i

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200930191738.8d33e791ec8c.Ica35125ed640aa3aa1ecc38fb5e8f1600caa8df6@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: move all bus-independent TX functions to common code

After moving out all Tx fields not related to pcie-bus
it's time to move the code to a common place.

We also rename all pcie functions name to txq.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200930161256.3947a5276003.I3fe1bec2b25a965a49532df288f47b8b59eb1500@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwl-trans: move dev_cmd_offs, page_offs to a common trans header

dev_cmd_offs, page_offs field is not directly related to the PCIe
transport, so move the structures it uses to the common iwl-trans.h header.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200926002540.83b41765961f.Icd12bfb2a736ccf4cbe080973c746fb70a3c4a50@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: move bc_table_dword to a common trans header

The bc_table_dword code is not directly related to the PCIe transport,
so move the structures it uses to the common iwl-trans.h header.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200924162105.1e3027abd0b5.Ie9f672983796586671a4374bf6dcda846875b690@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: iwl-trans: move tfd to trans layer

The tfd_* code is not directly related to the PCIe transport,
so move the structures it uses to the common iwl-trans.h header.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200924162105.a1a2ce2bce38.Ieada0fe087ebcc9bc84e22fc5abc9dba8914dd9f@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: move bc_pool to a common trans header

The bc_pool code is not directly related to the PCIe transport,
so move the structures it uses to the common iwl-trans.h header.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200924162105.f6469c62665d.Id920f1ae8f70886b66a98956a6412726d29a9481@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: move txq-specific from trans_pcie to common trans

We don't want to have txq code in the PCIe transport code, so move all
the relevant elements to a new iwl_txq structure and store it in
iwl_trans.

spatch

@ replace_pcie @
struct iwl_trans_pcie *trans_pcie;
@@

(
-trans_pcie->queue_stopped
+trans->txqs.queue_stopped
|
-trans_pcie->queue_used
+trans->txqs.queue_used
|
-trans_pcie->txq
+trans->txqs.txq
|
-trans_pcie->txq
+trans->txqs.txq
|
-trans_pcie->cmd_queue
+trans->txqs.cmd.q_id
|
-trans_pcie->cmd_fifo
+trans->txqs.cmd.fifo
|
-trans_pcie->cmd_q_wdg_timeout
+trans->txqs.cmd.wdg_timeout
)

// clean all new unused variables
@ depends on replace_pcie @
type T;
identifier i;
expression E;
@@
- T i = E;
... when != i

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200529092401.a428d3c9d66f.Ie04ae55f33954636a39c98e7ae1e739c0507435b@changeid

iwlwifi: move iwl_txq and substructures to a common trans header

The txq code is not directly related to the PCIe transport, so move the
structures it uses to the common iwl-trans.h header.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200529092401.d9d0082b8369.I8298f6e83804c1ea99217a79d95d23ef68b184d4@changeid

iwlwifi: pcie: keep trans instead of trans_pcie in iwl_txq

We used both the trans and the trans_pcie structures in
iwl_txq, so we can keep the trans structure instead. This
helps with the refactoring of txq code out of pcie.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200529092401.1f826d34339e.I23182a59bfbe089a1f659742d6fee6f64d2ed08c@changeid

iwlwifi: pcie: skip fragmented receive buffers

We don't really expect fragmented RBs, and don't seem to be seeing
them in practice since that would've caused a crash. Nevertheless,
we should be expecting the hardware to send them.

Parse the flag indicating a fragmented buffer, but then discard it
and any fragments thereof, at least for now. We need to do more
work in the higher layers to properly deal with this, since we may
not get "normal" firmware notifications that are fragmented, only
RX, and then we need to put it back together and add the necessary
API to report a chain of things to the higher layers, this doesn't
fit into the struct iwl_rx_cmd_buffer today.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200425130140.e78a59f70b1d.Ica656a98a4e4220d73edc97600edd680cbc97241@changeid

iwlwifi: pcie: gen2: use DMA pool for byte-count tables

Since the recent patch in this area, we no longer allocate 64k
for a single queue, but only 1k, which still means a full page.
Use a DMA pool to reduce this further, since we will have a lot
of queues in a typical system that can share pages.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200425130140.6e84c79aea30.Ie9a417132812d110ec1cc87852f101477c01cfcb@changeid

iwlwifi: pcie: move iwl_pcie_ctxt_info_alloc_dma() to user

There's no need for this to be an inline in the header file,
only the context-info.c file ever uses it. Move it there.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200418110539.818a06457888.Ib4f55280cd14d7edab37f2992b381c9b6ca4cd7a@changeid

iwlwifi: pcie: make iwl_pcie_cmdq_reclaim static

There's no need for this to be exposed outside of the tx.c
file, make it static.

Change-Id: I41d40008311b108d0578bd2ec73c5477e700a839
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: use partial pages if applicable

If we have only 2k RBs like on the latest (AX210) hardware, then
even on x86 where PAGE_SIZE is 4k we currently waste half of the
memory.

If this is the case, return partial pages from the allocator and
track the offset in each RBD (to be able to find the data in them
and remap them later.)

This might also address other platforms with larger PAGE_SIZE by
putting more RBs into a single large page.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: map only used part of RX buffers

We don't need to map *everything* of the RX buffers, we won't use
that much, map only the part we're going to use. This save some
IOMMU space (if applicable and it can deal with that) and also
prepares a bit for mapping partial pages for 2K buffers later.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: allocate more receive buffers for HE devices

For HE-capable devices, we need to allocate more receive buffers as
there could be 256 frames aggregated into a single A-MPDU, and then
they might contain A-MSDUs as well. Until 22000 family, the devices
are able to put multiple frames into a single RB and the default RB
size is 4k, but starting from AX210 family this is no longer true.
On the other hand, those newer devices only use 2k receive buffers
(by default).

Modify the code and configuration to allocate an appropriate number
of RBs depending on the device capabilities:

* 4096 for AX210 HE devices, which use 2k buffers by default,
* 2048 for 22000 family devices which use 4k buffers by default,
* 512 for existing 9000 family devices, which doesn't really
change anything since that's the default before this patch,
* 512 also for AX210/22000 family devices that don't do HE.

Theoretically, for devices lower than AX210, we wouldn't have to
allocate that many RBs if the RB size was manually increased, but
to support that the code got more complex, and it didn't really
seem necessary as that's a use case for monitor mode only, where
hopefully the wasted memory isn't really much of a concern.

Note that AX210 devices actually support bigger than 12-bit VID,
which is required here as we want to allocate 4096 buffers plus
some for quick recycling, so adjust the code for that as well.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: extend hardware workaround to context-info

After more investigation on the hardware side, it appears that the
hardware bug regarding 2^32 boundary reaching/crossing also affects
other uses of the DMA engine, in particular the ones triggered by
the context-info (image loader) mechanism.

It also turns out that the bug only affects devices with gen2 TX
hardware engine, so we don't need to change context info for gen3.
The TX path workarounds are simpler to still keep for both though.

Add the workaround to that code as well; this is a lot simpler as
we have just a single way to allocate DMA memory there.

I made the algorithm recursive (with a small limit) since it's
actually (almost) impossible to hit this today - dma_alloc_coherent
is currently documented to always return 32-bit addressable memory
regardless of the DMA mask for it, and so we could only get REALLY
unlucky to get the very last page in that area.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: allocate smaller dev_cmd for TX headers

As noted in the previous commit, due to the way we allocate the
dev_cmd headers with 324 byte size, and 4/8 byte alignment, the
part we use of them (bytes 20..40-68) could still cross a page
and thus 2^32 boundary.

Address this by using alignment to ensure that the allocation
cannot cross a page boundary, on hardware that's affected. To
make that not cause more memory consumption, reduce the size of
the allocations to the necessary size - we go from 324 bytes in
each allocation to 60/68 on gen2 depending on family, and ~120
or so on gen1 (so on gen1 it's a pure reduction in size, since
we don't need alignment there).

To avoid size and clearing issues, add a new structure that's
just the header, and use kmem_cache_zalloc().

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: move page tracking into get_page_hdr()

Move the tracking that records the page in the SKB for later
free (refcount decrement) into the get_page_hdr() function
for better code reuse.

While at it, also add an assertion that this doesn't overwrite
any existing page pointer in the skb.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: make some RX functions static

These aren't used outside the rx.c file, so make them static.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: remove IWL_DEVICE_22560/IWL_DEVICE_FAMILY_22560

This is dead code, nothing uses the IWL_DEVICE_22560 macro and
thus nothing every uses IWL_DEVICE_FAMILY_22560. Remove it all.

While at it, remove some code and definitions used only in this
case, and clean up some comments/names that still refer to it.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

iwlwifi: pcie: make iwl_pcie_gen2_update_byte_tbl static

It is called within tx-gen2.c only.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

iwlwifi: add support for suspend-resume flow for new device generation

The new device generation has a slightly different suspend resume flow
Currently, the way the driver instruct the device to move to D3 is by
sending D3_CONFIG_CMD.
Instead of using the host command the indication is by writing to the
doorbell interrupt.
The FW will respond with interrupt to indicate transition completion.

Signed-off-by: Haim Dreyfuss <haim.dreyfuss@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: add a pointer to the trans_cfg directly in trans

Add a pointer to the iwl_trans structure and point it to the trans
part of the cfg. This is the first step in disassociating the trans
configuration from the rest of the configuration.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pass the iwl_config_trans_params when needed

Instead of accessing the iwl_config_trans_params from the cfg that is
stored in the trans struct, pass this structure directly to functions
that need it during trans_alloc. This will be useful to isolate the
elements needed during allocation and pass them separately before the
actual cfg struct is known.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: separate elements from cfg that are needed by trans_alloc

In order to be able to select the cfg depending on the HW revision or
on the RF ID, we need to set up the trans before selecting the cfg.
To do so, move the elements from cfg that are needed by
iwl_trans_alloc() to a separate struct at the top of the cfg, so it
can be used by other cfg types as well, before selecting the rest of
the configuration.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: dbg_ini: use function to check if ini dbg mode is on

use iwl_trans_dbg_ini_valid function instead of a boolean value check if
dbg_ini mode is on. It is needed for a future patch.

Signed-off-by: Shahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: remove pm_runtime completely

This means:
1) stop calling pm_runtime_resume when starting the hardware
2) removing the unneeded low_power parameter to start / stop hw / fw
transport ops
3) squashing transport functions that are now the same
_iwl_trans_pcie_start_hw / iwl_trans_pcie_start_hw

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: remove some more d0i3 code from the transport

CMD_SEND_IN_IDLE, CMD_MAKE_TRANS_IDLE and CMD_WAKE_UP_TRANS
are not used. Remove them.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: remove the opmode's d0i3 handlers

Remove the now unneeded functions that called those from the
transport layer.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: Set w-pointer upon resume according to SN

During D3 state, FW may send packets.
As a result, "write" queue pointer will be incremented by FW.
Upon resume from D3, driver should adjust its shadows of "write" and "read"
pointers to the value reported by FW.

1. Keep TID used during wowlan configuration.
2. Upon resume, set driver's "write" and "read" queue pointers
to the value reported by FW.

Signed-off-by: Alex Malamud <alex.malamud@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: fix RF-Kill interrupt while FW load for gen2 devices

Newest devices have a new firmware load mechanism. This
mechanism is called the context info. It means that the
driver doesn't need to load the sections of the firmware.
The driver rather prepares a place in DRAM, with pointers
to the relevant sections of the firmware, and the firmware
loads itself.
At the end of the process, the firmware sends the ALIVE
interrupt. This is different from the previous scheme in
which the driver expected the FH_TX interrupt after each
section being transferred over the DMA.

In order to support this new flow, we enabled all the
interrupts. This broke the assumption that we have in the
code that the RF-Kill interrupt can't interrupt the firmware
load flow.

Change the context info flow to enable only the ALIVE
interrupt, and re-enable all the other interrupts only
after the firmware is alive. Then, we won't see the RF-Kill
interrupt until then. Getting the RF-Kill interrupt while
loading the firmware made us kill the firmware while it is
loading and we ended up dumping garbage instead of the firmware
state.

Re-enable the ALIVE | RX interrupts from the ISR when we
get the ALIVE interrupt to be able to get the RX interrupt
that comes immediately afterwards for the ALIVE
notification. This is needed for non MSI-X only.

Cc: stable@vger.kernel.org
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: dbg: move trans debug fields to a separate struct

Unite iwl_trans debug related fields under iwl_trans_debug struct to
increase readability and keep iwl_trans clean.

Signed-off-by: Shahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: fix load in rfkill flow for unified firmware

When we have a single image (same firmware image for INIT and
OPERATIONAL), we couldn't load the driver and register to the
stack if we had hardware RF-Kill asserted.

Fix this. This required a few changes:

1) Run the firmware as part of the INIT phase even if its
ucode_type is not IWL_UCODE_INIT.
2) Send the commands that are sent to the unified image in
INIT flow even in RF-Kill.
3) Don't ask the transport to stop the hardware upon RF-Kill
interrupt if the RF-Kill is asserted.
4) Allow the RF-Kill interrupt to take us out of L1A so that
the RF-Kill interrupt will be received by the host (to
enable the radio).

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

iwlwifi: pcie: initialize debug_rfkill to -1

This will let us introduce a mechanism to start with rfkill
faked, and put 0 here to override it.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: unite macros with same meaning

TFD_*_SLOTS and IWL_*_QUEUE_SIZE both define the TX queue
size (number of TFDs).
Get rid of TFD_*_SLOTS and use only IWL_*_QUEUE_SIZE.

Signed-off-by: Shaul Triebitz <shaul.triebitz@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: no need to check return value of debugfs_create functions

When calling debugfs functions, there is no need to ever check the
return value. The function can work or not, but the code logic should
never do something different based on this.

Cc: Johannes Berg <johannes.berg@intel.com>
Cc: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Cc: Luca Coelho <luciano.coelho@intel.com>
Cc: Intel Linux Wireless <linuxwifi@intel.com>
Cc: Kalle Valo <kvalo@codeaurora.org>
Cc: linux-wireless@vger.kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: switch to correct RBD/CD layout for 22560

The layout of the RBD (receive buffer descriptor) isn't quite right,
the hardware ended up being implemented differently. Switch to the
correct RBD layout. While at it, remove the now useless extra defines.

Also, switch the CD (completion descriptor) to the right format, which
is basically just a code cleanup because the only field we really used
(rbid) is still in the same place. We may need fragmentation later if
we ever want to use it.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: add sync_nmi to trans ops

Allow modules from outside pcie to call sync_nmi.

Signed-off-by: Shahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: allocate rb_stts's for all queues in one place

AX210 devices assume that the (DRAM) addresses of the rb_stts's for
the different queues are continuous.
So allocate the rb_stts's for all the Rx queues in one place.

Signed-off-by: Shaul Triebitz <shaul.triebitz@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: fix TX while flushing

When flushing TX queues no new TX should go into the system.
However, in the following scenario we get TX:
1. Queues are stopped and there are packets in overflow queue
2. Station is removed and flush begins
3. Flush empties space, and reclaim path TXes SKB from overflow
queue.

Note that the fact the queues are stopped during the process
doesn't matter - the packet will be TXed since the TX path
doesn't care if TX queues are stopped or not, just if there is
space in the queue, which there is, since we just freed a
packet.

A fix here is rather complicated, since the flow is very racy.

Change code not to warn if we are TXing from overflow TX.
In case there is TX from both overflow TX and TX path we will
miss a warning we optimally had, but we can live with that.

Make sure we don't return before overflow queue is empty, otherwise
we will think queues are empty, but they will be refilled, resulting
with assert.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Fixes: 3955525d5d17 ("iwlwifi: pcie: buffer packets to avoid overflowing Tx queues")
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: fix send hcmd timeout recovery flow

Both iwl_trans_fw_error and iwl_force_nmi initiate async recovery flow.
Calling them both is redundant and causing a race.

Solve this by removing the call to iwl_trans_fw_error.

Signed-off-by: Shahar S Matityahu <shahar.s.matityahu@intel.com>
Fixes: cfadc3ffccd5 ("iwlwifi: pcie: stop the firmware when we restart it")
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: dbg_ini: implement paging memory dump

Implement paging memory dump in the new dump mechanism.
To support this change, moved iwl_self_init_dram strcut from trans_pcie
to trans so that it will accessible via fw_runtime.

Signed-off-by: Shahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: align licensing to dual GPL/BSD

These files have a long history of code changes, but analysing
the remaining code leads to having only a few changes that are
not already owned by Intel, notably from
- Andy Lutomirski <luto@amacapital.net>
- Joonwoo Park <joonwpark81@gmail.com>
- Kirtika Ruchandani <kirtika@chromium.org>
- Rajat Jain <rajatja@google.com>
- Stanislaw Gruszka <sgruszka@redhat.com>
remaining in the code today.

Note that
- I myself was working for Intel and for any possibly code
that might be before my employment there give permission
- Wizery employees were working for Intel

More specifically, we identified the following commits that
(partially may) remain today:

25c03d8e8c13 Joonwoo Park <joonwpark81@gmail.com> ("iwlwifi: do not schedule tasklet when rcv unused irq")
f36d04abe684 Stanislaw Gruszka <sgruszka@redhat.com> ("iwlwifi: use dma_alloc_coherent")
387f3381f732 Stanislaw Gruszka <sgruszka@redhat.com> ("iwlwifi: fix dma mappings and skbs leak")
2624e96ce16b Stanislaw Gruszka <sgruszka@redhat.com> ("iwlwifi: fix possible data overwrite in hcmd callback")
bfe4b80e9f73 Stanislaw Gruszka <sgruszka@redhat.com> ("iwlwifi: always check if got h/w access before write")
d536c32b45d2 Andy Lutomirski <luto@amacapital.net> ("iwlwifi: pcie: log when waking the NIC for hcmd submission fails")
a6d24fad00d9 Rajat Jain <rajatja@google.com> ("iwlwifi: pcie: dump registers when HW becomes inaccessible")
fb12777ab59b Kirtika Ruchandani <kirtika@chromium.org> ("iwlwifi: Add more call-sites for pcie reg dumper")
3a73a30049f2 Stanislaw Gruszka <sgruszka@redhat.com> ("iwlwifi: cleanup/fix memory barriers")
aa5affbacb24 Stanislaw Gruszka <sgruszka@redhat.com> ("iwlwifi: dump stack when fail to gain access to the device")

Align the licenses with their permission to clean up and to
make it all identical.

CC: Joonwoo Park <joonwpark81@gmail.com>
CC: Stanislaw Gruszka <sgruszka@redhat.com>
CC: Andy Lutomirski <luto@amacapital.net>
CC: Rajat Jain <rajatja@google.com>
CC: Kirtika Ruchandani <kirtika@chromium.org>
Acked-by: Johannes Berg <johannes@sipsolutions.net>
Acked-by: Kirtika Ruchandani <kirtika@chromium.org>
Acked-by: Stanislaw Gruszka <sgruszka@redhat.com>
Acked-by: Joonwoo Park <joonwpark81@gmail.com>
Acked-by: Rajat Jain <rajatja@google.com>
Acked-by: Andy Lutomirski <luto@kernel.org>
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: remove unnecessary iwl_pcie_enable_rx_wake() function

This function was only used by 9000 A-step devices, which we don't
support anymore, so it can be removed.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: dbg: dump data according to the new ini TLVs

When ini TLVs are loaded, dump data according to the
stored configuration.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: add debugfs file to read fw debug data recording

FW debug data will oneshot read all data available in DRAM
and fill the supplied user buffer. In case the read request
is greater than the new data in DRAM, the driver will write
all data it has and return the buffer immediately.

Signed-off-by: Shahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: Lior Cohen <lior2.cohen@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: add infrastructure for multiple debug buffers

In future devices we will have more than one debug buffer.
Prepare the infrastructure for allocation and release of
multiple debug buffers by grouping the variables in an array
of structures and moving it to trans section, where they will
be visible to opmode and FW.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: refactor txq_alloc for supporting more command type

Support more txq_alloc command types by moving the command declaration
to the gen specific area. While at it, move some of the code segments
to a common place for re-use.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: change monitor DMA to be coherent

Allow access to the memory by the host and the device simultaneously.
This will be needed in some future patches.

Signed-off-by: Shahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: remove all occurrences of the FSF address paragraph

The Free Software Foundation address is superfluous and causes
checkpatch to issue a warning when present. Remove all paragraphs
with FSF's address to prevent that.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: improve the flow when a NIC is disconnected

When the NIC is disconnected, we just can't do anything
besides seeking for help from the bus driver. Dumping the
device's memory is not necessary and just bloats the logs
with unusable data. Moreover, asking mac80211 to restart
the hardware is also useless. Bypass all this.

Also, use the STATUS_TRANS_DEAD status bit instead of a
bool inside the transport layer. The advantage of this is
that now, the transport and the op_mode can know what is the
situation and bypass the useless recovery steps mentioned
above.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: make gen2 of apm_init non-static

This will allow using the same init in future generations.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: store the default rxq number

Store the default rxq number in a variable, so we won't need
to use the actual number in the code.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: remove dump_regs() from transport ops

This is used only within PCIe, and there's no reason to go through
the transport methods for a function call within PCIe itself.
Remove the dump_regs() method and call the function directly.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: make non-static hcmd and rx code

Allow other device generations to use the utilities that
are used to send and reclaim host commands and to allocate
rx, by making it non-static.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: allow using tx init for other queues but the command queue

We would like to allow using tx init code for other queues but
the command queue - for newer devices.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: drop packets with bad status in CD

We need to drop packets with errors (such as replay,
MIC, ICV, conversion, duplicate and so on).

Drop invalid packets, put the status bits in the metadata and
move the enum definition to the correct place (FW API header).

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: move some msix and rx functions to a common place

We would like to allow other utlities to init msix and rx.
Put their declarations in a place accessible to other utilities.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: unionize used bd and completion descriptor

This allows less "dummy" declarations and casting.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: support rx structures for 22560 devices

The rfh for 22560 devices has changed so it supports now
the same arch of using used and free lists, but different
structures to support the last.
Use the new structures, hw dependent, to manage the lists.

bd, the free list, uses the iwl_rx_transfer_desc,
in which the vid is stored in the structs' rbid
field, and the page address in the addr field.

used_bd, the used list, uses the iwl_rx_completion_desc
struct, in which the vid is stored in the structs' rbid
field.

rb_stts, the hw "write" pointer of rx is stored in a
__le16 array, in which each entry represents the write
pointer per queue.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: introduce new rx structures used by 22560 RFH

22560 devices RFH uses different structures, which act similar
to the legacy rxq management lists - free and used list.

The iwl_rx_transfer_desc struct is part of the free list,
and consists of pointers to the empty rb's the driver wants to
pass to the fw.

The iwl_rx_completion_desc struct is part of the used list,
and consists of pointers to the buffer the fw filled up
with new rx, both commands and data, for the host.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: support multiple tfd queue max sizes for different devices

22560 devices tfd queue max size is 2^16. Allow a configurable
max size in the driver for supporting different devices.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: cancel the injective function between hw pointers to tfd entry index

Nowadays, the tfd queue max size is 2^8, and the reserved size in the
command header sequence field for the tfd entry index is 8 bits,
allowing an injective function from the hw pointers to the tfd entry index
in the sequence field.

In 22560 devices the tfd queue max size is 2^16, meaning that
the hw pointers are 16 bit long (allowing to point to each entry
in the tfd queue). However, the reserved space in the sequence field for
the tfd entry doesn't change, and we are limited to 8 bit.
This requires cancelling the injective function from hw pointer to
tfd entry in the sequence number.

Use iwl_pcie_get_cmd_index to wrap the hw pointer's to the n_window
size, which is maximum 256 in tx queues, and so, keep the injective
function between the window wrapped hw pointers to tfd entry index in
the sequence.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: update gen3 interrupts - sw error and image response

In 22560 devices the ROM sendis an interrupt to the host
once the IML reading is done.
Handle this interrupt, and indicate sw error in case the
value is fail.

Additionally, the cause for sw error in 22560 devices
have been changed, so update the cause list.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: start early debug for 22560 devices

In 22560 devices we can start debug using context info gen3. Configure
the fw to start collecting logs to the dram before init.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: support context information for 22560 devices

Context information structure was added to 22000 devices for
firmware self init.

In the next generation of devices the context information
changes significantly, and the original context information
is divided roughly to three data structures: context information gen3,
prph information and prph scratch.

In addition, the init flow changes so the firmware is loaded
by the IML, and so we must allocate the IML on the DRAM and
give the ROM the IML's address before kicking the firmware's
self init.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: allocate and free rx cr's and tr's tails

The hw now refers to two new blocks:
* rx tr tail - The Tail index on the free buffers queue TR,
which is update by the device after reading the free buffer
from the tr.
* rx cr tail - Updated by the driver when completing
processing a new completion descriptor in the cr.

Add these two new struct to the rxq, allocate and free them
when needed.

In addition, the register for rx write pointer had been changed
to HBUS_TARG_WRPTR. The way to differentiate tx from rx is the
queue number. TX range is 0-511, and RX's is 512-527.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: remove non-responsive device

If we fail to to grab NIC access because the device is not responding
(i.e. CSR_GP_CNTRL returns 0xFFFFFFFF), remove the device from the PCI
bus, to avoid any further damage, and to let the user space rescan.

In order to inform the userspace that a rescan is needed, we send a
kobject uevent with "INACCESSIBLE".

This functionality is disabled by default, but can be enabled via a
new module parameter called "remove_when_gone". In the future we may
change this module parameter to include 3 modes instead: do nothing;
auto-rescan or; send uevent.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Rajat Jain <rajatja@google.com>

iwlwifi: add TX queue size parameter to TX queue allocation

As preparation for dynamic queue sizing, add a parameter
of the TX queue size to the dynamic queue allocation op
mode API.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: Revert "iwlwifi: pcie: dynamic Tx command queue size"

This reverts commit dd05f9aab4426ff178b12d601e50d19d336eba30.

Shorter TX queues support was added eventually without the
need for the parameters this patch added.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: fix DMA memory mapping / unmapping

22000 devices (previously referenced as A000) can support
short transmit queues. This means that we have less DMA
descriptors (TFD) for those shorter queues.
Previous devices must still have 256 TFDs for each queue
even if those 256 TFDs point to fewer buffers.

When I introduced support for the short queues for 22000
I broke older devices by assuming that they can also have
less TFDs in their queues. This led to several problems:

1) the payload of the commands weren't unmapped properly
which caused the SWIOTLB to complain at some point.
2) the hardware could get confused and we get hardware
crashes.

The corresponding bugzilla entries are:

https://bugzilla.kernel.org/show_bug.cgi?id=198201
https://bugzilla.kernel.org/show_bug.cgi?id=198265

Cc: stable@vger.kernel.org # 4.14+
Fixes: 4ecab5616023 ("iwlwifi: pcie: support short Tx queues for A000 device family")
Reviewed-by: Sharon, Sara <sara.sharon@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

iwlwifi: avoid duplicate sw reset executions in the code

Most of the sw resets in the code are done by one function,
which writes to the relevant CSR.

Use the common function to perform the only reset which was
done separately, redundant to the common code.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: dump registers when HW becomes inaccessible

We conclude the HW became inaccessible when we timeout waiting for
a bit to be set in a memory mapped register (CSR_GP_CNTRL). This
conclusion may not be true because the bit may not get set due to:
- a firmware issue
- a driver issue
- a PCI bus issue
- a platform issue
There are a lot of such reports with really no good debug information
beyond this message to help us.

Add some debug information and attempt to dump the different register
spaces at such a failure:

* Dump some configuration space of device - this will tell us if
something very basic is broken in the PCIe bus (so that configuration
accesses are failing). If this works, the PCIe bus seems OK. If this
does not work, it is definitely an PCIe issue.

* Dump some memory mapped registers - if we're reading some sane'ish
values, this will tell us that the PCIe bus is OK, but may be a firmware
/ driver issue. If this does not work, it may be a PCI configuration
issue or a driver/firmware issue.

* Dump parent and device's AER registers, will give us some straws to
chew on.

This is the sample output:
[ 13.082651] ------------[ cut here ]------------
[ 13.086791] iwlwifi 0000:01:00.0: iwlwifi transaction failed, dumping registers
[ 13.086793] iwlwifi 0000:01:00.0: iwlwifi device config registers:
[ 13.086893] iwlwifi 0000:01:00.0: 00000000: 095a8086 00100406 02800059 00000000 00000004 00000000 00000000 00000000
[ 13.086895] iwlwifi 0000:01:00.0: 00000020: 00000000 00000000 00000000 50108086 00000000 000000c8 00000000 00000100
[ 13.086901] iwlwifi 0000:01:00.0: iwlwifi device memory mapped registers:
[ 13.086989] iwlwifi 0000:01:00.0: 00000000: ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
[ 13.086991] iwlwifi 0000:01:00.0: 00000020: ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
[ 13.086999] iwlwifi 0000:01:00.0: iwlwifi device AER capability structure:
[ 13.087033] iwlwifi 0000:01:00.0: 00000000: 14010001 00100000 00000000 00462031 00002000 00002000 00000014 40000001
[ 13.087034] iwlwifi 0000:01:00.0: 00000020: 0000000f d140000c 00000000
[ 13.087036] iwlwifi 0000:01:00.0: iwlwifi parent port (0000:00:1c.0) config registers:
[ 13.087074] iwlwifi 0000:00:1c.0: 00000000: 9d108086 00100506 060400f1 00810010 00000000 00000000 00010100 200000f0
[ 13.087075] iwlwifi 0000:00:1c.0: 00000020: d140d140 0001fff1 00000000 00000000 00000000 00000040 00000000 0006010b
[ 13.087087] ------------[ cut here ]------------
[ 13.087095] WARNING: CPU: 0 PID: 1759 at drivers/net/wireless/iwl7000/iwlwifi/pcie/trans.c:2082 iwl_trans_pcie_reclaim+0x1ee4/0x2b9a [iwlwifi]()
[ 13.087096] Timeout waiting for hardware access (CSR_GP_CNTRL 0xffffffff)

Signed-off-by: Rajat Jain <rajatja@google.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: dynamic Tx command queue size

Devices in the A000 family can use a different size for the command queue.
To allow this, make the command queue size configurable and set the size
for A000 devices to 32.

Signed-off-by: Shahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: move rx workqueue initialization to iwl_trans_pcie_alloc()

Work queues cannot be allocated when a mutex is held because the mutex
may be in use and that would make it sleep. Doing so generates the
following splat with 4.13+:

[ 19.513298] ======================================================
[ 19.513429] WARNING: possible circular locking dependency detected
[ 19.513557] 4.13.0-rc5+ #6 Not tainted
[ 19.513638] ------------------------------------------------------
[ 19.513767] cpuhp/0/12 is trying to acquire lock:
[ 19.513867] (&tz->lock){+.+.+.}, at: [<ffffffff924afebb>] thermal_zone_get_temp+0x5b/0xb0
[ 19.514047]
[ 19.514047] but task is already holding lock:
[ 19.514166] (cpuhp_state){+.+.+.}, at: [<ffffffff91cc4baa>] cpuhp_thread_fun+0x3a/0x210
[ 19.514338]
[ 19.514338] which lock already depends on the new lock.

This lock dependency already existed with previous kernel versions,
but it was not detected until commit 49dfe2a67797 ("cpuhotplug: Link
lock stacks for hotplug callbacks") was introduced.

Reported-by: David Weinehall <david.weinehall@intel.com>
Reported-by: Jiri Kosina <jikos@kernel.org>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

iwlwifi: pcie: support short Tx queues for A000 device family

This allows to modify TFD_TX_CMD_SLOTS to a power of 2
which is smaller than 256.
Note that we still need to set values to wrap at 256
into the scheduler's write pointer, but all the rest of
the code can use shorter transmit queues.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: free the TSO page when a Tx queue is unmapped on A000 devices

When we unmap a non-empty Tx queue, we need to free the
pages that we allocated for the headers in TSO flows.
This code existed for the 9000 device family, but somehow
it got left out when the new Tx path for the A000 devices
was written.

Fixes: 2b0c5946d9ed ("iwlwifi: pcie: introduce a000 TX queues management")
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: rename iwl_trans_check_hw_rf_kill() to pcie

Rename this function to the more appropriate iwl_pcie_check_hw_rf_kill()
since it's only a function in the pcie code and cannot be called from
any other place.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: wait longer after device reset

The newest devices need a longer time to reset because of
their more complex hardware. Wait 5ms after device reset.
Consolidate all the places that reset the device in the
PCIe transport to avoid future bugs.

While at it, unify the flow to use set_bit instead of full
write as requested by the hardware designers.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: fix 9000-series RF-kill interrupt propagation

A hardware issue on 9000 series devices sometimes causes RF-kill
interrupts to not be propagated to the host properly if ASPM is
enabled. Work around this by setting the right hardware bit to
allow it to interrupt the host for this reason (rfkill).

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: make iwl_pcie_apm_stop_master() return void

Nothing ever checks the return value of iwl_pcie_apm_stop_master(),
so there's no point in it having one - make it return void.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: add a W/A for a scheduler hardware bug

In case we need to move the scheduler write pointer by
steps of 0x40, 0x80 or 0xc0, the scheduler gets stuck.
This leads to hardware error interrupts with status:
0x5A5A5A5A or alike.

In order to work around this, detect in the transport
layer that we are going to hit this case and tell iwlmvm
to increment the sequence number of the packets. This
allows to keep the requirement that the WiFi sequence
number is in sync with the index in the scheduler Tx queue
and it also allows to avoid the problematic sequence.
This means that from time to time, we will start a queue
from ssn + 1, but that shouldn't be a problem since we
don't switch to new queues for AMPDU now that we have
DQA which allows to keep the same queue while toggling
the AMPDU state.

This bug has been fixed on 9000 devices and up.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: don't report RF-kill enabled while shutting down

When toggling the RF-kill pin quickly in succession, the driver can
get rather confused because it might be in the process of shutting
down, expecting all commands to go through quickly due to rfkill,
but the transport already thinks the device is accessible again,
even though it previously shut it down. This leads to bugs, and I
even observed a kernel panic.

Avoid this by making the PCIe code only report that the radio is
enabled again after the higher layers actually decided to shut it
off.

This also pulls out this common RF-kill checking code into a common
function called by both transport generations and also moves it to
the direct method - in the internal helper we don't really care
about the RF-kill status anymore since we won't report it up until
the stop anyway.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: add fake RF-kill to debugfs

In order to debug "hardware" RF-kill flows, add a low-level hook to
allow changing the "hardware" RF-kill from debugfs.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: add AMSDU to gen2

This is essentially the same code as gen1, except that it uses
gen2 functions and SW checksum is not included.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: move to 512 queues

Avoid using the old define since it will enlarge necessary
structs for previous HW.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: support debug applying on a000 hw

Allow configuring debug destination on a000 HW.

Signed-off-by: Liad Kaufman <liad.kaufman@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: get rid of txq id assignment

In TVQM mode the queue ID is assigned after enablement.
Get rid of assuming pre-defined TX queue ID in functions
that will be used by TVQM allocation path.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: alloc queues dynamically

Change queue allocation to be dynamic. On transport init only
the command queue is being allocated. Other queues are allocated
on demand.
This is due to the huge amount of queues we will soon enable (512)
and as a preparation for TX Virtual Queue Manager feature (TVQM),
where firmware will assign the actual queue number on demand.
This includes also allocation of the byte count table per queue
and not as a contiguous chunk of memory.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: introduce new stop_device

This function is basically the same as gen1, except for clean
ups of old devices configuration that are never used in a000
configuration.
It will also help with refactoring rf_kill later on.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: prepare for dynamic queue allocation

In a000 transport we will allocate queues dynamically.
Right now queue are allocated as one big chunk of memory
and accessed as such.
The dynamic allocation of the queues will require accessing
the queues as pointers.
In order to keep simplicity of pre-a000 tx queues handling,
keep allocating and freeing the memory in the same style,
but move to access the queues in the various functions as
individual pointers.
Dynamic allocation for the a000 devices will be in a separate
patch.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: support host commands in new transport

Code is basically the same, with a cleanups of old narrow host
command, ampg workarounds, some cosmetic stuff, and usage of
TFH functions when accessing TFD queues.
This enables also the cleanup of iwl_pcie_tfd_set_tb() since
now it won't be called anywhere in the a000 data path

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: support new TX command

Move to use the correct structure.
Remove code referring to old command.
Update DMA locations.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: cleanup old transport code from gen2

Cleanup code that is irrelevant for a000 devices.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: copy TX functions to new transport

This is just a copy-paste in order to make changes tracking
easier.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: introduce a000 TX queues management

In a000 devices the TX handling is different in a few ways:
* Queues are allocated dynamically
* DQA is enabled by default
* Driver shouldn't access TFH registers - ucode configures it
all in SCD_QUEUE_CFG command

Support all this in a new API with op mode, where op mode sends
the command, transport will allocate the queue dynamically, fill
in DMA properties, send the command to FW and get the ID back.
Current implementation only sets the new transport API and fills
the DMA properties.
Future patches will complete the other parts.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: add context information support

Context information structure is going to be used in a000
devices for firmware self init.

The self init includes firmware self loading from DRAM by
ROM.
This means the TFH relevant firmware loading can be cleaned up.

The firmware loading includes the paging memory as well, so op
mode can stop initializing the paging and sending the DRAM_BLOCK_CMD.

Firmware is doing RFH, TFH and SCD configuration, while driver
only fills the required configurations and addresses in the
context information structure.

The only remaining access to RFH is the write pointer, which
is updated upon alive interrupt after FW configured the RFH.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: remove the active field in struct iwl_txq

We already have queue_used in the transport - we can
use it instead.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

Revert "iwlwifi: introduce trans API to get byte count table"

This reverts commit 8aacf4b73fe8 ("iwlwifi: introduce trans API
to get byte count table").
The commit is not needed as a better approach will be taken.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: don't increment / decrement a bool

David reported that the code I added uses the decrement
and increment operator on a boolean variable.

Fix that.

Fixes: 0cd58eaab148 ("iwlwifi: pcie: allow the op_mode to block the tx queues")
Reported-by: David Binderman <dcb314@hotmail.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: fix another RF-kill race

When resuming, it's possible for the following scenario to occur:

* iwl_pci_resume() enables the RF-kill interrupt
* iwl_pci_resume() reads the RF-kill state (e.g. to 'radio enabled')
* RF_KILL interrupt triggers, and iwl_pcie_irq_handler() reads the
state, now 'radio disabled', and acquires the &trans_pcie->mutex.
* iwl_pcie_irq_handler() further calls iwl_trans_pcie_rf_kill() to
indicate to the higher layers that the radio is now disabled (and
stops the device while at it)
* iwl_pcie_irq_handler() drops the mutex
* iwl_pci_resume() continues, acquires the mutex and calls the higher
layers to indicate that the radio is enabled.

At this point, the device is stopped but the higher layers think it's
available, and can call deeply into the driver to try to enable it.
However, this will fail since the device is actually disabled.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: avoid variable shadowing in TFD helpers

The various TFD/TB helpers have two code paths depending on the
type of TFD supported, with variable shadowing due to the new if
branches. Move the fall-through code into else branches to avoid
variable shadowing. While doing so, rename some of the variables
and do some other cleanups (like removing void * casts of void *
pointers.)

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: move to wide ID for all commands

Due to firmware design considerations, move to wide ID for
all commands.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: Set affinity mask for rx interrupt vectors per cpu

In order to utilize the host's CPUs in the most efficient way
we bind each rx interrupt vector to each CPU on the host.
Each rx interrupt is prioritized to execute only on the designated CPU
rather than any CPU.
Processor affinity takes advantage of the fact that some remnants of
a process that was run on a given processor may remain in that
processor's memory state for example, data in the CPU cache after
another process is run on that CPU. Scheduling that process to execute
on the same processor could result in an efficient use of process by
reducing performance-degrading situations such as cache misses
and parallel processing.

Signed-off-by: Haim Dreyfuss <haim.dreyfuss@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: Configure shared interrupt vector in MSIX mode

In case the OS provides fewer interrupts than requested, different
causes will share the same interrupt vector as follow:
1.One interrupt less: non rx causes shared with FBQ.
2.Two interrupts less: non rx causes shared with FBQ and RSS.
3.More than two interrupts: we will use fewer RSS queues.

Also make the request depend on the number of online CPUs
instead of possible CPUs.

Signed-off-by: Haim Dreyfuss <haim.dreyfuss@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: merge iwl_queue and iwl_txq

The original intent was to have the general iwl_queue shared
between RX and TX queues, but it is not the actual status.
Since it is not shared with any struct but iwl_txq, it adds
unnecessary complexity. Merge those structs.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: assign and access a000 TFD & TBs

Previous patch introduced the new formats. This patch
allocates the new structures and adjusts code accordingly.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: introduce trans API to get byte count table

In future HW the byte count table address will be configured
by ucode per queue. Add API to expose the byte count table to
the opmode

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: introduce new tfd and tb formats

New hardware supports bigger TFDs and TBs.
Introduce the new formats and adjust defines and code
relying on old format.
Changing the actual TFD allocation is trickier and
deferred to the next patch.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: move iwl_drv to be shared across transports

All transports has this structure. By moving it to be
shared, we can get rid of casting to the specific transport
in probe and remove.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: centralize SCD status logging

Centralize the logging of SCD status. The motivation is
that for a000 devices we will have new SCD HW, but this
code was duplicate anyway, so it is a proper cleanup.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: generalize and increase the size of scratchbuf

Currently the scratch buffer is set to 16 bytes and indicates
the size of the bi-directional DMA.
However, next HW generation will perform additional offloading,
and will write the result in the key location of the TX command,
so the size of the bi-directional consistent memory should grow
accordingly - increase it to 40.
Generalize the code to get rid of now irrelevant scratch references.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: track rxb status

In MQ environment and new architecture in early stages
we may encounter DMA issues. Track RXB status and bail
out in case we receive index to an RXB that was not
mapped and handed over to HW.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: fix a race in firmware loading flow

Upon firmware load interrupt (FH_TX), the ISR re-enables the
firmware load interrupt only to avoid races with other
flows as described in the commit below. When the firmware
is completely loaded, the thread that is loading the
firmware will enable all the interrupts to make sure that
the driver gets the ALIVE interrupt.
The problem with that is that the thread that is loading
the firmware is actually racing against the ISR and we can
get to the following situation:

CPU0 CPU1
iwl_pcie_load_given_ucode
...
iwl_pcie_load_firmware_chunk
wait_for_interrupt
<interrupt>
ISR handles CSR_INT_BIT_FH_TX
ISR wakes up the thread on CPU0
/* enable all the interrupts
* to get the ALIVE interrupt
*/
iwl_enable_interrupts
ISR re-enables CSR_INT_BIT_FH_TX only
/* start the firmware */
iwl_write32(trans, CSR_RESET, 0);

BUG! ALIVE interrupt will never arrive since it has been
masked by CPU1.

In order to fix that, change the ISR to first check if
STATUS_INT_ENABLED is set. If so, re-enable all the
interrupts. If STATUS_INT_ENABLED is clear, then we can
check what specific interrupt happened and re-enable only
that specific interrupt (RFKILL or FH_TX).

All the credit for the analysis goes to Kirtika who did the
actual debugging work.

Cc: <stable@vger.kernel.org> [4.5+]
Fixes: a6bd005fe92 ("iwlwifi: pcie: fix RF-Kill vs. firmware load race")
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: decouple PCIe transport from mac80211

The PCIe transport needs to store two pointers in each TX SKB, and
currently assumes mac80211's ieee80211_tx_info is present in the CB
to do that.

In order to remove that assumption, have the opmodes pass in the
offset to where the pointers can be stored in the CB and use the
offset in the PCIe code.

To make the disentanglement complete, remove mac80211.h includes
from everywhere in the generic iwlwifi code. This required adding
an include of cfg80211.h in one place.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: mvm: support dqa queue sharing

Support DQA queue sharing when no free queue exists for
allocation to a STA that already exists. This means that
a single queue will serve more than a single TID (although
the RA will be the same for all TIDs served).

We try to choose the lowest AC possible, to ensure the
shared queues have the lowest possible combined AC
requirements. The queue to share is chosen only from the
same RA's DATA queues as follows (in descending priority):
1. An AC_BE queue
2. Same AC queue
3. Highest AC queue that is lower than new AC
4. Any existing AC (there always is at least 1 DATA queue)

If any aggregations existed for any of the TIDs of the
shared queue - they are stopped (the FW is notified), but
no delBA is sent.

Signed-off-by: Liad Kaufman <liad.kaufman@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: workaround HW shadow registers bug

Integrated 9000 devices have a bug with shadow registers
value retention.
If driver writes RBD registers while MAC is asleep the
values are stored in shadow registers to be copied whenever
MAC wakes up.
However, in 9000 devices a MAC wakeup is not triggered
and when the bus powers down due to inactivity the shadow
values and dirty bits are lost.
Turn on the chicken-bits that cause MAC wakeup for RX-related
values as well when the device is in D0.
When the device is in low power mode turn the RX wakeup chicken
bits off since driver is idle and this W/A is not needed.
Remove previous W/A which was ineffective.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: trans: don't call the trans-specific ref/unref directly

It's cleaner to always call the iwl_trans_ref/unref() functions
instead of sometimes calling the trans-specific ops directly. This
also prepares for moving some of the code from the trans-specific ops
to the common trans code.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: refcounting is not necessary anymore

We don't use the refcount value anymore, all the refcounting is done
in the runtime PM usage_count value. Remove it.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: pcie: fix global table size

My patch resized the pool size, but neglected to resize
the global table, which is obviously wrong since the global
table maps the pool's rxb to vid one to one. This results
in a panic in 9000 devices.
Add a build bug to avoid such a case in the future.

Fixes: 7b5424361ec9 ("iwlwifi: pcie: fine tune number of rxbs")
Reported-by: Haim Dreyfuss <haim.dreyfuss@intel.com>
Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: pcie: fine tune number of rxbs

We kick the allocator when we have 2 RBDs that don't have
attached RBs, and the allocator allocates 8 RBs meaning
that it needs another 6 RBDs to attach the RBs to.
The design is that allocator should always have enough RBDs
to fulfill requests, so we give in advance 6 RBDs to the
allocator so that when it is kicked, it gets additional 2 RBDs
and has enough RBDs.
These RBDs were taken from the Rx queue itself, meaning
that each Rx queue didn't have the maximal number of
RBDs, but MAX - 6.
Change initial number of RBDs in the system to include both
queue size and allocator reserves.
Note the multi-queue is always 511 instead of 512 to avoid a
full queue since we cannot detect this state easily enough in
the 9000 arch.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: pcie: Add new configuration to enable MSIX

Working with MSIX requires prior configuration.
This includes requesting interrupt vectors from the OS,
registering the vectors and mapping the optional causes to the
relevant interrupt. In addition add new interrupt handler
to handle MSIX interrupt.

Signed-off-by: Haim Dreyfuss <haim.dreyfuss@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: pcie: fix RF-Kill vs. firmware load race

When we load the firmware, we hold trans_pcie->mutex to
avoid nested flows. We also rely on the ISR to wake up the
thread when the DMA has finished copying a chunk. During
this flow, we enable the RF-Kill interrupt.

The problem is that the RF-Kill interrupt handler can take
the mutex and bring the device down. This means that if
we load the firmware while the RF-Kill switch is enabled
(which will happen when we load the INIT firmware to read
the device's capabilities and register to mac80211), we
may get an RF-Kill interrupt immediately and the ISR will
be waiting for the mutex held by the thread that is
currently loading the firmware. At this stage, the ISR
won't be able to service the DMA's interrupt needed to
wake up the thread that load the firmware. We are in a
deadlock situation which ends when the thread that loads
the firmware fails on timeout and releases the mutex.

To fix this, take the mutex later in the flow, disable
the interrupts and synchronize_irq() to give a chance to
the RF-Kill interrupt to run and complete.
After that, mask all the interrupts besides the DMA
interrupt and proceed with firmware load. Make sure to
check that there was no RF-Kill interrupt when the
interrupts were disabled.

This fixes https://bugzilla.kernel.org/show_bug.cgi?id=111361

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: pcie: enable multi-queue rx path

Previous patches enabled new 9000 hardware DMA for one queue
only.
Enable the actual multi-queue path and configuration now.
This requires also per-queue NAPI struct.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: pcie: add RTPM support when wifi is enabled

Enable runtime power management (RTPM) for PCIe devices and implement
the corresponding functions to enable D0i3 mode when the device is
idle.

Additionally, remove some unnecessary #ifdef's because the RTPM code
will not be called if runtime PM is not configured.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: pcie: add 9000 series multi queue rx DMA support

The 9000 series introduces several changes in the device
DMA operation.
As the device now supports multi-queue rx, several DMA channels
should be configured.
The flows of providing the device with the allocated RBDs now
changes as well - the device maintains a separate table of used
and free table.

The hardware may use the free table to feed RBDs to any queue.
This requires maintaing a shared table to map returned RBDs to
the original RXB - for that purpose the VID is introduced - an
internal identifier of the RB placed in the lower 12 bits and
returned by HW in the used data.

Another change is the support of 64 bit DMA address.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: pcie: add infrastructure for multi-queue rx

The 9000 series devices will support multi rx queues.
Current code has one static rx queue - change it to allocate
a number of queues per the device capability (pre-9000 devices
have the number of rx queues set to one).

Subsequent generalizations are:

Change the code to access an explicit numbered rx queue only
when the queue number is known - when handling interrupt, when
accessing the default queue and when iterating the queues.
The rest of the functions will receive the rx queue as a pointer.

Generalize the warning in allocation failure to consider the
allocator status instead of a single rx queue status.

Move the rx initial pool of memory buffers to be shared among
all the queues and allocated to the default queue on init.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: pcie: buffer packets to avoid overflowing Tx queues

When the Tx queues are full above a threshold, we
immediately stop the mac80211's queue to stop getting new
packets. This worked until TSO was enabled.
With TSO, one single packet from mac80211 can use many
descriptors since a large send needs to be split into
several segments.
This means that stopping mac80211's queues is not enough
and we also need to ensure that we don't overflow the Tx
queues with one single packet from mac80211.
Add code to transport layer to do just that. Stop
mac80211's queue as soon as the queue is full above the
same threshold as before, and keep pushing the current
packet along with its segments on the queue, but check
that we don't overflow. If that would happen, buffer the
segments, and send them when there is room in the Tx queue
again. Of course, we first need to send the buffered
segments and only then, wake up mac80211's queues.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: pcie: build an A-MSDU using TSO core

When the op_mode sends an skb whose payload is bigger than
MSS, PCIe will create an A-MSDU out of it. PCIe assumes
that the skb that is coming from the op_mode can fit in one
A-MSDU. It is the op_mode's responsibility to make sure
that this guarantee holds.

Additional headers need to be built for the subframes.
The TSO core code takes care of the IP / TCP headers and
the driver takes care of the 802.11 subframe headers.

These headers are stored on a per-cpu page that is re-used
for all the packets handled on that same CPU. Each skb
holds a reference to that page and releases the page when
it is reclaimed. When the page gets full, it is released
and a new one is allocated.

Since any SKB that doesn't go through the fast-xmit path
of mac80211 will be segmented, we can assume here that the
packet is not WEP / TKIP and has a proper SNAP header.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: pcie: allow to pretend to have Tx CSUM for debug

Allow to configure the driver to pretend to have TX CSUM
offload support. This will be useful to test the TSO flows
that will come in further patches.
This configuration is disabled by default.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: change the Intel Wireless email address

ilw@linux.intel.com is not available anymore.
linuxwifi@intel.com should be used instead.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: update host command messages to new format

Host commands now have a group id, express this in printed messages.

Signed-off-by: Sharon Dvir <sharon.dvir@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: pcie: allow the op_mode to block the tx queues

In certain flows (see next patches), the op_mode may need to
block the Tx queues for a short period. Provide an API for
that. The transport is in charge of counting the number of
times the queues are blocked since the op_mode may block the
queues several times in a row before unblocking them.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: clean up transport debugfs handling

Transport code currently calls itself through the transport ops,
which is quite pointless. Clean up all of this. While at it,
remove the unnecessary dir argument and the redundant IDI code.

In slave transports, call both the common slave debugfs and the
transport's own. SDIO has no files, so remove it all there.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: trans: make various conversion macros inlines

Make the various conversion functions typesafe, so we don't
accidentally try to call them with the wrong pointers and
cast them to something that will crash.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: add support for 12K Receive Buffers

802.11ac allows A-MSDU that can be up to 12KB long. Since
an entire A-MSDU needs to fit into one single Receive
Buffer (RB), add support for big RBs.
Since this adds lots of pressure to the memory manager and
significantly increase the true_size of the RX buffers,
don't enable this by default.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>

iwlwifi: move under intel vendor directory

Part of reorganising wireless drivers directory and Kconfig.

Signed-off-by: Kalle Valo <kvalo@codeaurora.org>