wifi: iwlwifi: remove async command callback

There's only one user of this code, which is STA unblock
during sleep for uAPSD on really old devices. Instead of
having this all through the API with calls up and down,
just implemented a special-case CMD_BLOCK_TXQS flag for
this, it's only needed in the old gen1 transport.

While at it, fix a complain that lockdep would have, as
we lock the cmd queue and then the TXQs in the reclaim
by using spin_lock_nested(). We no longer need to disable
BHs in iwl_trans_pcie_block_txq_ptrs() since it's called
with them disabled already.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Miri Korenblit <miriam.rachel.korenblit@intel.com>
Link: https://msgid.link/20231207044813.2bd95e0570fc.I16486dbc82570d2f73a585872f5394698627310d@changeid
Signed-off-by: Johannes Berg <johannes.berg@intel.com>

iwlwifi: de-const properly where needed

In order to de-const variables simply casting through (void *) is
not enough: "cast from 'const .. *' to 'void *' drops const qualifier".
Cast through (uintptr_t) as well [1] to make this compile on systems
with more strict requirements.
In addition passing const void *data to dma_map_single() also
drops the (const) qualifier. De-constify on variable on assignment
which may be overwritten later. In either case the (void *) cast
to dma_map_single() is not needed (anymore) either.

[1] See __DECONST() in sys/sys/cdefs.h in FreeBSD

Sponsored by: The FreeBSD Foundation
Signed-off-by: Bjoern A. Zeeb <bz@FreeBSD.ORG>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20220128153014.eb696eb56bf6.Ide1dd041f9b908c5154a600286a7453750b0704a@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: Fix softirq/hardirq disabling in iwl_pcie_gen2_enqueue_hcmd()

Analogically to what we did in 2800aadc18a6 ("iwlwifi: Fix softirq/hardirq
disabling in iwl_pcie_enqueue_hcmd()"), we must apply the same fix to
iwl_pcie_gen2_enqueue_hcmd(), as it's being called from exactly the same
contexts.

Reported-by: Heiner Kallweit <hkallweit1@gmail.com
Signed-off-by: Jiri Kosina <jkosina@suse.cz>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>
Link: https://lore.kernel.org/r/nycvar.YFH.7.76.2104171112390.18270@cbobk.fhfr.pm

iwlwifi: pcie: don't disable interrupts for reg_lock

The only thing we do touching the device in hard interrupt context
is, at most, writing an interrupt ACK register, which isn't racing
in with anything protected by the reg_lock.

Thus, avoid disabling interrupts here for potentially long periods
of time, particularly long periods have been observed with dumping
of firmware memory (leading to lockup warnings on some devices.)

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210210135352.da916ab91298.I064c3e7823b616647293ed97da98edefb9ce9435@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: tx: move handing sync/async host command to trans

Handling host commands in a sync way is not directly related to PCIe
transport, and can serve as common logic for any transport, so move
it to trans layer.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210117164916.fde99af4e0f7.I4cab95919eb35cc5bfb26d32dcf5e15419d0e0ef@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: mvm: don't send commands during suspend\resume transition

D3_CONFIG_CMD and D0I3_END_CMD should be the last\first
command upon suspend\resume correspondingly, otherwise,
FW will raise an assert (0x342).

There are firmware notifications that cause the driver to
send a command back to the firmware. If such a notification
is sent to the driver while the the driver prepares the
firmware for D3, operation, what is likely to happen is that
the handling of the notification will try to get the mutex
and will wait unil the driver finished configuring the
firmware for D3. Then the handling notification will get
the mutex and handle the notification which will lead to
the aforementioned ASSERT 342.

To avoid this, we need to prevent any command to be sent to
the firmware between the D3_CONFIG_CMD and the D0I3_END_CMD.
Check this in the utility layer that sends the host commands
and in the transport layer as well.
Flag the D3_CONFIG_CMD and the D0I3_END_CMD commands as
commands that must be sent even if the firmware has already
been configured for D3 operation.

Signed-off-by: Haim Dreyfuss <haim.dreyfuss@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210117164916.1935a993b471.I3192c93c030576ca16773c01b009c4d93610d6ea@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwl-trans: iwlwifi: move sync NMI logic to trans

The code is not directly related to PCIe transport, and it will help
moving sync/async commands logic out of PCIe in the next patches.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20210117130510.271f59887fd1.I8ff41236f4e11a25df83d76c982a2a30ba2b9903@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: use SPDX tags

Use SPDX tags instead of the long copyright notices. Also cleanup
some duplicate copyright notices and combine the years where possible.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20201210000603.481bcb512a6f.I8146abe5a637079e7336209f23cb26af98b12b31@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: move all bus-independent TX functions to common code

After moving out all Tx fields not related to pcie-bus
it's time to move the code to a common place.

We also rename all pcie functions name to txq.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200930161256.3947a5276003.I3fe1bec2b25a965a49532df288f47b8b59eb1500@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwl-trans: move dev_cmd_offs, page_offs to a common trans header

dev_cmd_offs, page_offs field is not directly related to the PCIe
transport, so move the structures it uses to the common iwl-trans.h header.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200926002540.83b41765961f.Icd12bfb2a736ccf4cbe080973c746fb70a3c4a50@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: move bc_table_dword to a common trans header

The bc_table_dword code is not directly related to the PCIe transport,
so move the structures it uses to the common iwl-trans.h header.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200924162105.1e3027abd0b5.Ie9f672983796586671a4374bf6dcda846875b690@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: iwl-trans: move tfd to trans layer

The tfd_* code is not directly related to the PCIe transport,
so move the structures it uses to the common iwl-trans.h header.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200924162105.a1a2ce2bce38.Ieada0fe087ebcc9bc84e22fc5abc9dba8914dd9f@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: move bc_pool to a common trans header

The bc_pool code is not directly related to the PCIe transport,
so move the structures it uses to the common iwl-trans.h header.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200924162105.f6469c62665d.Id920f1ae8f70886b66a98956a6412726d29a9481@changeid
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

mm, treewide: rename kzfree() to kfree_sensitive()

As said by Linus:

A symmetric naming is only helpful if it implies symmetries in use.
Otherwise it's actively misleading.

In "kzalloc()", the z is meaningful and an important part of what the
caller wants.

In "kzfree()", the z is actively detrimental, because maybe in the
future we really _might_ want to use that "memfill(0xdeadbeef)" or
something. The "zero" part of the interface isn't even _relevant_.

The main reason that kzfree() exists is to clear sensitive information
that should not be leaked to other future users of the same memory
objects.

Rename kzfree() to kfree_sensitive() to follow the example of the recently
added kvfree_sensitive() and make the intention of the API more explicit.
In addition, memzero_explicit() is used to clear the memory to make sure
that it won't get optimized away by the compiler.

The renaming is done by using the command sequence:

git grep -w --name-only kzfree |\
xargs sed -i 's/kzfree/kfree_sensitive/'

followed by some editing of the kfree_sensitive() kerneldoc and adding
a kzfree backward compatibility macro in slab.h.

[akpm@linux-foundation.org: fs/crypto/inline_crypt.c needs linux/slab.h]
[akpm@linux-foundation.org: fix fs/crypto/inline_crypt.c some more]

Suggested-by: Joe Perches <joe@perches.com>
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: David Howells <dhowells@redhat.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
Cc: James Morris <jmorris@namei.org>
Cc: "Serge E. Hallyn" <serge@hallyn.com>
Cc: Joe Perches <joe@perches.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Dan Carpenter <dan.carpenter@oracle.com>
Cc: "Jason A . Donenfeld" <Jason@zx2c4.com>
Link: http://lkml.kernel.org/r/20200616154311.12314-3-longman@redhat.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

iwlwifi: move txq-specific from trans_pcie to common trans

We don't want to have txq code in the PCIe transport code, so move all
the relevant elements to a new iwl_txq structure and store it in
iwl_trans.

spatch

@ replace_pcie @
struct iwl_trans_pcie *trans_pcie;
@@

(
-trans_pcie->queue_stopped
+trans->txqs.queue_stopped
|
-trans_pcie->queue_used
+trans->txqs.queue_used
|
-trans_pcie->txq
+trans->txqs.txq
|
-trans_pcie->txq
+trans->txqs.txq
|
-trans_pcie->cmd_queue
+trans->txqs.cmd.q_id
|
-trans_pcie->cmd_fifo
+trans->txqs.cmd.fifo
|
-trans_pcie->cmd_q_wdg_timeout
+trans->txqs.cmd.wdg_timeout
)

// clean all new unused variables
@ depends on replace_pcie @
type T;
identifier i;
expression E;
@@
- T i = E;
... when != i

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200529092401.a428d3c9d66f.Ie04ae55f33954636a39c98e7ae1e739c0507435b@changeid

iwlwifi: pcie: gen2: use DMA pool for byte-count tables

Since the recent patch in this area, we no longer allocate 64k
for a single queue, but only 1k, which still means a full page.
Use a DMA pool to reduce this further, since we will have a lot
of queues in a typical system that can share pages.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200425130140.6e84c79aea30.Ie9a417132812d110ec1cc87852f101477c01cfcb@changeid

iwlwifi: pcie: allocate much smaller byte-count table

The hardware needs a byte-count table with the size of each frame
on the queue to build A-MPDUs, but:
* newer generation no longer have the duplicated space at the end,
they can deal with the wrap properly - and we don't even fill
the dup anyway
* we have a maximum queue size of 512 right now and don't use the
theoretical hardware maximum of 65536.

Together, this reduces the byte count table DMA allocation from
64KiB (65536*2 + 64*2 rounded up) to 1 KiB (though that might be
rounded up to a full 4 KiB page by the allocator, not sure it can
share the allocations.)

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200425130140.c263b787b5ab.I059507a9760b1ce1d45d84dcaa91629a5cfb58e0@changeid

iwlwifi: pcie: gen2: minor code cleanups in byte table update

One line should be indented less, otherwise it looks like it
belongs into the parentheses, which clearly it doesn't; also
some variables can move into their respective if branches.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200417131727.a4858aa0441b.I0e70e4a5493fe6b8db6390f9349ff0e7888ab240@changeid

iwlwifi: pcie: actually release queue memory in TVQM

The iwl_trans_pcie_dyn_txq_free() function only releases the frames
that may be left on the queue by calling iwl_pcie_gen2_txq_unmap(),
but doesn't actually free the DMA ring or byte-count tables for the
queue. This leads to pretty large memory leaks (at least before my
queue size improvements), in particular in monitor/sniffer mode on
channel hopping since this happens on every channel change.

This was also now more evident after the move to a DMA pool for the
byte count tables, showing messages such as

BUG iwlwifi:bc (...): Objects remaining in iwlwifi:bc on __kmem_cache_shutdown()

This fixes https://bugzilla.kernel.org/show_bug.cgi?id=206811.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Fixes: 6b35ff91572f ("iwlwifi: pcie: introduce a000 TX queues management")
Cc: stable@vger.kernel.org # v4.14+
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>
Link: https://lore.kernel.org/r/iwlwifi.20200417100405.f5f4c4193ec1.Id5feebc9b4318041913a9c89fc1378bb5454292c@changeid

iwlwifi: pcie: validate queue ID before array deref/bit ops

Validate that the queue ID is in range before trying to use it as
an index or for test_bit() - the previous bug showed that this has
in fact happened, and it was lucky that we caught it there, had the
bit been set then we'd have actually used the value despite being
far out of range.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: extend hardware workaround to context-info

After more investigation on the hardware side, it appears that the
hardware bug regarding 2^32 boundary reaching/crossing also affects
other uses of the DMA engine, in particular the ones triggered by
the context-info (image loader) mechanism.

It also turns out that the bug only affects devices with gen2 TX
hardware engine, so we don't need to change context info for gen3.
The TX path workarounds are simpler to still keep for both though.

Add the workaround to that code as well; this is a lot simpler as
we have just a single way to allocate DMA memory there.

I made the algorithm recursive (with a small limit) since it's
actually (almost) impossible to hit this today - dma_alloc_coherent
is currently documented to always return 32-bit addressable memory
regardless of the DMA mask for it, and so we could only get REALLY
unlucky to get the very last page in that area.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: allocate smaller dev_cmd for TX headers

As noted in the previous commit, due to the way we allocate the
dev_cmd headers with 324 byte size, and 4/8 byte alignment, the
part we use of them (bytes 20..40-68) could still cross a page
and thus 2^32 boundary.

Address this by using alignment to ensure that the allocation
cannot cross a page boundary, on hardware that's affected. To
make that not cause more memory consumption, reduce the size of
the allocations to the necessary size - we go from 324 bytes in
each allocation to 60/68 on gen2 depending on family, and ~120
or so on gen1 (so on gen1 it's a pure reduction in size, since
we don't need alignment there).

To avoid size and clearing issues, add a new structure that's
just the header, and use kmem_cache_zalloc().

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: detect the DMA bug and warn if it happens

Warn if the DMA bug is going to happen. We don't have a good
way of actually aborting in this case and we have workarounds
in place for the cases where it happens, but in order to not
be surprised add a safety-check and warn.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: work around DMA hardware bug

There's a hardware bug in the flow handler (DMA engine), if the
address + len of some TB wraps around a 2^32 boundary, the carry
bit is then carried over into the next TB.

Work around this by copying the data to a new page when we find
this situation, and then copy it in a way that we cannot hit the
very end of the page.

To be able to free the new page again later we need to chain it
to the TSO page, use the last pointer there to make sure we can
never use the page fully for DMA, and thus cannot cause the same
overflow situation on this page.

This leaves a few potential places (where we didn't observe the
problem) unaddressed:
* The second TB could reach or cross the end of a page (and thus
2^32) due to the way we allocate the dev_cmd for the header
* For host commands, a similar thing could happen since they're
just kmalloc().
We'll address these in further commits.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: move page tracking into get_page_hdr()

Move the tracking that records the page in the SKB for later
free (refcount decrement) into the get_page_hdr() function
for better code reuse.

While at it, also add an assertion that this doesn't overwrite
any existing page pointer in the skb.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: trace IOVA for iwlwifi_dev_tx_tb

We trace the whole TFD with all TBs when in iwlwifi_dev_tx,
but sometimes we add TBs to it later and then we don't have
any of this data. Trace the I/O virtual address (IOVA) (it
can be the physical address, or as returned by the IOMMU)
here to aid debugging the DMA flows.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: fix support for transmitting SKBs with fraglist

When the implementation of SKBs with fraglist was sent upstream, a
merge-damage occurred and half the patch was not applied.

This causes problems in high-throughput situations with AX200 devices,
including low throughput and FW crashes.

Introduce the part that was missing from the original patch.

Fixes: 0044f1716c4d ("iwlwifi: pcie: support transmitting SKBs with fraglist")
Cc: stable@vger.kernel.org # 4.20+
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
[ This patch was created by me, but the original author of this code
is Johannes, so his s-o-b is here and he's marked as the author of
the patch. ]
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: remove IWL_DEVICE_22560/IWL_DEVICE_FAMILY_22560

This is dead code, nothing uses the IWL_DEVICE_22560 macro and
thus nothing every uses IWL_DEVICE_FAMILY_22560. Remove it all.

While at it, remove some code and definitions used only in this
case, and clean up some comments/names that still refer to it.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

iwlwifi: pcie: make iwl_pcie_gen2_update_byte_tbl static

It is called within tx-gen2.c only.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

iwlwifi: pcie: don't consider IV len in A-MSDU

From gen2 PN is totally offloaded to hardware (also the space for the
IV isn't part of the skb). As you can see in mvm/mac80211.c:3545, the
MAC for cipher types CCMP/GCMP doesn't set
IEEE80211_KEY_FLAG_PUT_IV_SPACE for gen2 NICs.

This causes all the AMSDU data to be corrupted with cipher enabled.

Signed-off-by: Mordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

iwlwifi: add a pointer to the trans_cfg directly in trans

Add a pointer to the iwl_trans structure and point it to the trans
part of the cfg. This is the first step in disassociating the trans
configuration from the rest of the configuration.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: separate elements from cfg that are needed by trans_alloc

In order to be able to select the cfg depending on the HW revision or
on the RF ID, we need to set up the trans before selecting the cfg.
To do so, move the elements from cfg that are needed by
iwl_trans_alloc() to a separate struct at the top of the cfg, so it
can be used by other cfg types as well, before selecting the rest of
the configuration.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: remove pm_runtime completely

This means:
1) stop calling pm_runtime_resume when starting the hardware
2) removing the unneeded low_power parameter to start / stop hw / fw
transport ops
3) squashing transport functions that are now the same
_iwl_trans_pcie_start_hw / iwl_trans_pcie_start_hw

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: remove some more d0i3 code from the transport

CMD_SEND_IN_IDLE, CMD_MAKE_TRANS_IDLE and CMD_WAKE_UP_TRANS
are not used. Remove them.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: remove the refs / unrefs from the transport

This code is now stale

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: fix the byte count table format for 22560 devices

Starting from 22560, the byte count is expected to be in
bytes and we have now 14 bits. Ajust the code to this.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

iwlwifi: add sync_nmi to trans ops

Allow modules from outside pcie to call sync_nmi.

Signed-off-by: Shahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: fix send hcmd timeout recovery flow

Both iwl_trans_fw_error and iwl_force_nmi initiate async recovery flow.
Calling them both is redundant and causing a race.

Solve this by removing the call to iwl_trans_fw_error.

Signed-off-by: Shahar S Matityahu <shahar.s.matityahu@intel.com>
Fixes: cfadc3ffccd5 ("iwlwifi: pcie: stop the firmware when we restart it")
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: calculate pointers from out_cmd instead of out_cmd->hdr

The out_cmd structure starts with a header, so there's no need to use
&out_cmd->hdr, out_cmd alone is enough. We use this when calculating
other addresses and klocwork gets confused with that because it thinks
we are trying to access hdr (as an array) beyond its size.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: fix the use of a wrong define

The code checks that we haven't exceeded the maximum number of
TBs by comparing to a define of gen1 instead of gen2, fix it.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: memcpy from dev_cmd and not dev_cmd->hdr

Klocwork complains about copying from dev_cmd->hdr if
copying more than 4 bytes since it means part of the
copy is from the next field. This isn't a real bug,
but for not failing Klocwork next time - fix this.

Signed-off-by: Liad Kaufman <liad.kaufman@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: tighten boundary checks

The driver assumes certain sizes and lengths aren't crossed in some
places. Make sure this indeed happens.

Found by Klocwork.

Signed-off-by: Liad Kaufman <liad.kaufman@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: use iwl_tx_cmd_gen2 and not iwl_tx_cmd

When building AMSDU for gen2, code uses iwl_tx_cmd. The only
updated field is len, which is in the same location, so it
is not a bug. However, it is a bit confusing and error prone,
so change it.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

iwlwifi: pcie: set cmd_len in the correct place

command len is set too early in the code, since when building
AMSDU, the size changes. This causes the byte count table to
have the wrong size.

Fixes: a0ec0169b7a9 ("iwlwifi: support new tx api")
Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: don't reset TXQ write pointer

Currently code sets the write pointer when getting the TX queue
allocate response. This causes a redundant interrupt with any actual
change in the pointer. Remove this write altogether.

Fixes: 310181ec34e2 ("iwlwifi: move to TVQM mode")
Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: trace: change trace to trace one TB at a time

Split TX tracing to be per TB. This is needed now that
AMSDUs can be sent and skb can be larger than trace
limit.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: don't pad AMSDU packets

When we TX AMSDU, we shouldn't pad the packet. In the past,
we were building AMSDU only in transport layer, and gen2
functions are built based on this. However, now that op mode
may build AMSDUs, we need to take care of padding also in
gen2 "non-pcie-amsdu" path.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie gen2: check iwl_pcie_gen2_set_tb() return value

If we use the iwl_pcie_gen2_set_tb() return value for BIT(),
we should validate that it's not going to be negative, so do
the check and bail out if we hit an error. We shouldn't, as
we check if it'll fit beforehand, but better be safe.

Fixes: ab6c644539e9 ("iwlwifi: pcie: copy TX functions to new transport")
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: gen2: build A-MSDU only for GSO

If the incoming frame should be an A-MSDU, it may already be one,
for example in the case of NAN multicast being encapsulated in an
A-MSDU. Thus, use the GSO algorithm to build A-MSDU only if the
skb actually contains GSO data.

Fixes: 6ffe5de35b05 ("iwlwifi: pcie: add AMSDU to gen2")
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: gen2: pull adding frags to helper routine

Move the skb fragment loop into a helper routine to be able
to reuse it later.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: refactor txq_alloc for supporting more command type

Support more txq_alloc command types by moving the command declaration
to the gen specific area. While at it, move some of the code segments
to a common place for re-use.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: make non-static hcmd and rx code

Allow other device generations to use the utilities that
are used to send and reclaim host commands and to allocate
rx, by making it non-static.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: allow using tx init for other queues but the command queue

We would like to allow using tx init code for other queues but
the command queue - for newer devices.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: split tx to amsdu and non amsdu

The code is different enough to justify a split.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: include tcp.h implicitly

If CONFIG_IPV6 is not enabled in the kernel, tcp.h is not included
implicitly from other header files, causing compilation errors. To
solve that, explicitly include it in tx-gen2.c.

Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: support new tx api

22560 devices use a new tx cmd api. Update the code to use
the new api.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: support multiple tfd queue max sizes for different devices

22560 devices tfd queue max size is 2^16. Allow a configurable
max size in the driver for supporting different devices.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: update bytes in the byte count table

For devices which use the image loader image, the length of the frame
must be updated in the byte count in bytes, and not dwords as today.
Avoid dividing the input length by 4.

Signed-off-by: Golan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: gen2: fix race in cmd fifo write ptr

Avoid a race where two (or more) commands get the
same index:

1. T1 calls enqueue_hcmd and the local TFD index is assigned to
txq->write_ptr;
2. Context switch 'before incrementing txq->write_ptr';
3. T2 calls enqueue_hcmd and the local TFD index is assigned to
txq->write_ptr;
4. Now the index is set to the same value for both commands of T1 and
T2.

To prevent this from happening, set the local TFD index inside the
critical section (the index is set by global txq write pointer).

Signed-off-by: Shaul Triebitz <shaul.triebitz@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: use the queue size as sent by opmode

Op mode will begin tp use varying size of TX queue.
All the infra is in place, allow it.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: add TX queue size parameter to TX queue allocation

As preparation for dynamic queue sizing, add a parameter
of the TX queue size to the dynamic queue allocation op
mode API.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: Revert "iwlwifi: pcie: dynamic Tx command queue size"

This reverts commit dd05f9aab4426ff178b12d601e50d19d336eba30.

Shorter TX queues support was added eventually without the
need for the parameters this patch added.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: allocate shorter TX queues for 22000 devices

When support for shorter TX queues was introduced, it
didn't include the actual allocation of shorter queue,
which is the main motive for the change.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: implement the overlow queue for Gen2 devices

When we enable TSO, we can have a lot of packets in the
operation mode that will be pushed to the transport
no matter what is the queue's fullness state.

To cope with that the transport can buffer those packets
and add them to the ring later when there is more room.
This implementation was missing in the Gen2 devices'
code.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: don't warn if we use all the transmit pointers

Our Transmit Frame Descriptor (TFD) is a DMA descriptor that
includes several pointers to be able to transmit a packet
which is not physically contiguous.

Depending on the hardware being use, we can have 20 or 25
pointers in a single TFD. In both cases, it is more than
enough and it is quite hard to hit this limit.
It has been reported that when using specific applications
(Ktorrent), we can actually use all the pointers and then
a long standing bug showed up.

When we free the TFD, we check its number of valid pointers
and make sure it doesn't exceed the number of pointers the
hardware support.
This check had an off by one bug: it is perfectly valid to
free the 20 pointers if the TFD has 20 pointers.

Fix that.

https://bugzilla.kernel.org/show_bug.cgi?id=197981

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: fix DMA memory mapping / unmapping

22000 devices (previously referenced as A000) can support
short transmit queues. This means that we have less DMA
descriptors (TFD) for those shorter queues.
Previous devices must still have 256 TFDs for each queue
even if those 256 TFDs point to fewer buffers.

When I introduced support for the short queues for 22000
I broke older devices by assuming that they can also have
less TFDs in their queues. This led to several problems:

1) the payload of the commands weren't unmapped properly
which caused the SWIOTLB to complain at some point.
2) the hardware could get confused and we get hardware
crashes.

The corresponding bugzilla entries are:

https://bugzilla.kernel.org/show_bug.cgi?id=198201
https://bugzilla.kernel.org/show_bug.cgi?id=198265

Cc: stable@vger.kernel.org # 4.14+
Fixes: 4ecab5616023 ("iwlwifi: pcie: support short Tx queues for A000 device family")
Reviewed-by: Sharon, Sara <sara.sharon@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Kalle Valo <kvalo@codeaurora.org>

iwlwifi: pcie: remove set but not used variable tcph

This variable is never used, so remove the code to set it.
After this, the variable 'iph' also has the same fate.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: dynamic Tx command queue size

Devices in the A000 family can use a different size for the command queue.
To allow this, make the command queue size configurable and set the size
for A000 devices to 32.

Signed-off-by: Shahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: support short Tx queues for A000 device family

This allows to modify TFD_TX_CMD_SLOTS to a power of 2
which is smaller than 256.
Note that we still need to set values to wrap at 256
into the scheduler's write pointer, but all the rest of
the code can use shorter transmit queues.

Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: free the TSO page when a Tx queue is unmapped on A000 devices

When we unmap a non-empty Tx queue, we need to free the
pages that we allocated for the headers in TSO flows.
This code existed for the 9000 device family, but somehow
it got left out when the new Tx path for the A000 devices
was written.

Fixes: 2b0c5946d9ed ("iwlwifi: pcie: introduce a000 TX queues management")
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: don't init a Tx queue with an SSN > size of the queue

The TVQM tells us the initial write pointer for a queue,
but that write pointer is in WiFi sequence number unit
and not in TFD index unit. Which means that the write
pointer in the TVQM's response can be bigger than the
Tx queue ring size.

Fix that by modulo'ing the write pointer from the TVQM
with the Tx queue size.

Fixes: 66128fa08806 ("iwlwifi: move to TVQM mode")
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: fix A-MSDU on gen2 devices

The return status check of iwl_pcie_gen2_build_amsdu
was buggy. Fix it.

Fixes: 6ffe5de35b05 ("iwlwifi: pcie: add AMSDU to gen2")
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: reorganize firmware API

Apart from DVM, all firmware uses the same base API, and there's
code outside iwlmvm that needs to interact with it. Reflect this
in the source better and reorganize the firmware API to a new
fw/api/ directory.

While at it, split the already pretty large fw-api.h file into a
number of smaller files, going from almost 3k lines in there to
a maximum number of lines less than 1k.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: fix TX tracing for non-linear SKBs

When sending non-linear SKBs that should be included in the regular
TX tracing completely (and not be pushed into the tx_data tracing),
the (tracing) code didn't correctly take the fact that they were
non-linear into account and added only the skb head portion.

This probably never really triggered, since those frames we want
traced fully are most likely linear anyway, but the code gets easier
to understand and we lose an argument to the tracing function, so
overall fixing this is better.

Fixes: 206eea783385 ("iwlwifi: pcie: support frag SKBs")
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: simplify data tracepoint

There's no need to calculate the data_len outside of the tracepoint,
since it's always skb->len - hdr_len, which are both available inside.
Simplify the callers and move the calculation in.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: don't report RF-kill enabled while shutting down

When toggling the RF-kill pin quickly in succession, the driver can
get rather confused because it might be in the process of shutting
down, expecting all commands to go through quickly due to rfkill,
but the transport already thinks the device is accessible again,
even though it previously shut it down. This leads to bugs, and I
even observed a kernel panic.

Avoid this by making the PCIe code only report that the radio is
enabled again after the higher layers actually decided to shut it
off.

This also pulls out this common RF-kill checking code into a common
function called by both transport generations and also moves it to
the direct method - in the internal helper we don't really care
about the RF-kill status anymore since we won't report it up until
the stop anyway.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: fix TVQM queue ID range check

The queue ID should never be 512 either, so correct the check
to be >= instead of just >.

Fixes: 310181ec34e2 ("iwlwifi: move to TVQM mode")
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: add AMSDU to gen2

This is essentially the same code as gen1, except that it uses
gen2 functions and SW checksum is not included.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: fix host command memory leaks

Sending host command with CMD_WANT_SKB flag demands the release of the
response buffer with iwl_free_resp function.
The patch adds the memory release in all the relevant places

Signed-off-by: Shahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: mvm: support getting nvm data from firmware

This API replaces the complex NVM parsing of the iwlwifi module.
Instead, we get all needed data from firmware.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: move to TVQM mode

In TVQM firmware returns the value of the queue ID and code
should accept it.
The TX queue config API was changed. Move to new API.
This has to be done in parallel in mvm and pcie.
Do not move yet to 512 queues since there are some opens
with enabling it.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: get rid of txq id assignment

In TVQM mode the queue ID is assigned after enablement.
Get rid of assuming pre-defined TX queue ID in functions
that will be used by TVQM allocation path.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: alloc queues dynamically

Change queue allocation to be dynamic. On transport init only
the command queue is being allocated. Other queues are allocated
on demand.
This is due to the huge amount of queues we will soon enable (512)
and as a preparation for TX Virtual Queue Manager feature (TVQM),
where firmware will assign the actual queue number on demand.
This includes also allocation of the byte count table per queue
and not as a contiguous chunk of memory.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: prepare for dynamic queue allocation

In a000 transport we will allocate queues dynamically.
Right now queue are allocated as one big chunk of memory
and accessed as such.
The dynamic allocation of the queues will require accessing
the queues as pointers.
In order to keep simplicity of pre-a000 tx queues handling,
keep allocating and freeing the memory in the same style,
but move to access the queues in the various functions as
individual pointers.
Dynamic allocation for the a000 devices will be in a separate
patch.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: remove block and freeze operations from new transport

New transport will be used only by op modes that supports
buffer station offload - hence those will never be called.
Clean it up.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: support new write pointer width

In a000 devices we have 16 bytes for the TFD index and 16 for the
queue, in order to support 512 queues.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: support host commands in new transport

Code is basically the same, with a cleanups of old narrow host
command, ampg workarounds, some cosmetic stuff, and usage of
TFH functions when accessing TFD queues.
This enables also the cleanup of iwl_pcie_tfd_set_tb() since
now it won't be called anywhere in the a000 data path

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: rewrite TFD creation

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: support new TX command

Move to use the correct structure.
Remove code referring to old command.
Update DMA locations.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: cleanup old transport code from gen2

Cleanup code that is irrelevant for a000 devices.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: copy TX functions to new transport

This is just a copy-paste in order to make changes tracking
easier.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>

iwlwifi: pcie: introduce a000 TX queues management

In a000 devices the TX handling is different in a few ways:
* Queues are allocated dynamically
* DQA is enabled by default
* Driver shouldn't access TFH registers - ucode configures it
all in SCD_QUEUE_CFG command

Support all this in a new API with op mode, where op mode sends
the command, transport will allocate the queue dynamically, fill
in DMA properties, send the command to FW and get the ID back.
Current implementation only sets the new transport API and fills
the DMA properties.
Future patches will complete the other parts.

Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>