crypto: safexcel - Remove cfb and ofb

Remove the unused CFB/OFB implementation.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure/safexcel - Convert to platform remove callback returning void

The .remove() callback for a platform driver returns an int which makes
many driver authors wrongly assume it's possible to do error handling by
returning an error code. However the value returned is ignored (apart
from emitting a warning) and this typically results in resource leaks.

To improve here there is a quest to make the remove callback return
void. In the first step of this quest all drivers are converted to
.remove_new(), which already returns void. Eventually after all drivers
are converted, .remove_new() will be renamed to .remove().

Trivially convert this driver from always returning zero in the remove
callback to the void returning variant.

Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: safexcel - Cleanup ring IRQ workqueues on load failure

A failure loading the safexcel driver results in the following warning
on boot, because the IRQ affinity has not been correctly cleaned up.
Ensure we clean up the affinity and workqueues on a failure to load the
driver.

crypto-safexcel: probe of f2800000.crypto failed with error -2
------------[ cut here ]------------
WARNING: CPU: 1 PID: 232 at kernel/irq/manage.c:1913 free_irq+0x300/0x340
Modules linked in: hwmon mdio_i2c crypto_safexcel(+) md5 sha256_generic libsha256 authenc libdes omap_rng rng_core nft_masq nft_nat nft_chain_nat nf_nat nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables libcrc32c nfnetlink fuse autofs4
CPU: 1 PID: 232 Comm: systemd-udevd Tainted: G W 6.1.6-00002-g9d4898824677 #3
Hardware name: MikroTik RB5009 (DT)
pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : free_irq+0x300/0x340
lr : free_irq+0x2e0/0x340
sp : ffff800008fa3890
x29: ffff800008fa3890 x28: 0000000000000000 x27: 0000000000000000
x26: ffff8000008e6dc0 x25: ffff000009034cac x24: ffff000009034d50
x23: 0000000000000000 x22: 000000000000004a x21: ffff0000093e0d80
x20: ffff000009034c00 x19: ffff00000615fc00 x18: 0000000000000000
x17: 0000000000000000 x16: 0000000000000000 x15: 000075f5c1584c5e
x14: 0000000000000017 x13: 0000000000000000 x12: 0000000000000040
x11: ffff000000579b60 x10: ffff000000579b62 x9 : ffff800008bbe370
x8 : ffff000000579dd0 x7 : 0000000000000000 x6 : ffff000000579e18
x5 : ffff000000579da8 x4 : ffff800008ca0000 x3 : ffff800008ca0188
x2 : 0000000013033204 x1 : ffff000009034c00 x0 : ffff8000087eadf0
Call trace:
free_irq+0x300/0x340
devm_irq_release+0x14/0x20
devres_release_all+0xa0/0x100
device_unbind_cleanup+0x14/0x60
really_probe+0x198/0x2d4
__driver_probe_device+0x74/0xdc
driver_probe_device+0x3c/0x110
__driver_attach+0x8c/0x190
bus_for_each_dev+0x6c/0xc0
driver_attach+0x20/0x30
bus_add_driver+0x148/0x1fc
driver_register+0x74/0x120
__platform_driver_register+0x24/0x30
safexcel_init+0x48/0x1000 [crypto_safexcel]
do_one_initcall+0x4c/0x1b0
do_init_module+0x44/0x1cc
load_module+0x1724/0x1be4
__do_sys_finit_module+0xbc/0x110
__arm64_sys_finit_module+0x1c/0x24
invoke_syscall+0x44/0x110
el0_svc_common.constprop.0+0xc0/0xe0
do_el0_svc+0x20/0x80
el0_svc+0x14/0x4c
el0t_64_sync_handler+0xb0/0xb4
el0t_64_sync+0x148/0x14c
---[ end trace 0000000000000000 ]---

Fixes: 1b44c5a60c13 ("inside-secure - add SafeXcel EIP197 crypto engine driver")
Signed-off-by: Jonathan McDowell <noodles@earth.li>
Cc: stable@vger.kernel.org
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: safexcel - Raise firmware load failure message to error

At the moment if there is no firmware available for the safexcel driver
it will fail to load with a cryptic:

crypto-safexcel f2800000.crypto: TRC init: 15360d,80a (48r,256h)
crypto-safexcel f2800000.crypto: HW init failed (-2)

Raise the logging level of the firmware load failure to err rather than
dbg so that it's obvious what the reason for the HW init failure is.

Signed-off-by: Jonathan McDowell <noodles@earth.li>
Reviewed-by: Antoine Tenart <atenart@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: safexcel - Use request_complete helpers

Use the request_complete helpers instead of calling the completion
function directly.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: safexcel - Use crypto_wait_req

This patch replaces the custom crypto completion function with
crypto_req_done.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add MaxLinear platform

This is to add MaxLinear platform into compatible id.
Firmware endianness option is added since MaxLinear
firmware is in little endian format.

Signed-off-by: Peter Harliman Liem <pliem@maxlinear.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add fw_little_endian option

This is to add fw_little_endian option, which can
be used for platform which firmware is using little-endian
(instead of big-endian).

Signed-off-by: Peter Harliman Liem <pliem@maxlinear.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Expand soc data structure

Currently platform data is assigned directly to
version string(instead of struct). To make it more
scalable, we move it to use data struct instead.
This allows customization for individual platforms other
than version string.

Signed-off-by: Peter Harliman Liem <pliem@maxlinear.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add missing MODULE_DEVICE_TABLE for of

Without MODULE_DEVICE_TABLE, crypto_safexcel.ko module is not automatically
loaded on platforms where inside-secure crypto HW is specified in device
tree (e.g. Armada 3720). So add missing MODULE_DEVICE_TABLE for of.

Fixes: 1b44c5a60c13 ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Signed-off-by: Pali Rohár <pali@kernel.org>
Acked-by: Marek Behún <kabel@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add MODULE_FIRMWARE macros

The safexcel module loads firmware so add MODULE_FIRMWARE macros to
provide that information via modinfo.

Signed-off-by: Juerg Haefliger <juergh@protonmail.com>
Acked-by: Antoine Tenart <atenart@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Minor typo fix in the file safexcel.c

s/procesing/processing/

Signed-off-by: Bhaskar Chowdhury <unixbhaskar@gmail.com>
Acked-by: Antoine Tenart <atenart@kernel.org>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - fix platform_get_irq.cocci warnings

Remove dev_err() messages after platform_get_irq*() failures.
drivers/crypto/inside-secure/safexcel.c: line 1161 is redundant
because platform_get_irq() already prints an error

Generated by: scripts/coccinelle/api/platform_get_irq.cocci

Signed-off-by: Tian Tao <tiantao6@hisilicon.com>
Acked-by: Antoine Tenart <atenart@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: remove cipher routines from public crypto API

The cipher routines in the crypto API are mostly intended for templates
implementing skcipher modes generically in software, and shouldn't be
used outside of the crypto subsystem. So move the prototypes and all
related definitions to a new header file under include/crypto/internal.
Also, let's use the new module namespace feature to move the symbol
exports into a new namespace CRYPTO_INTERNAL.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Acked-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Fix sizeof() mismatch

An incorrect sizeof() is being used, sizeof(priv->ring[i].rdr_req) is
not correct, it should be sizeof(*priv->ring[i].rdr_req). Note that
since the size of ** is the same size as * this is not causing any
issues.

Addresses-Coverity: ("Sizeof not portable (SIZEOF_MISMATCH)")
Fixes: 9744fec95f06 ("crypto: inside-secure - remove request list to improve performance")
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Acked-by: Antoine Tenart <atenart@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add support for EIP197 with output classifier

This patch adds support for EIP197 instances that include the output
classifier (OCE) option, as used by one of our biggest customers.
The OCE normally requires initialization and dedicated firmware, but
for the simple operations supported by this driver, we just bypass it
completely for now (using what is formally a debug feature).

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@rambus.com>
Acked-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - irq balance

Balance the irqs of the inside secure driver over all
available cpus.
Currently all interrupts are handled by the first CPU.

From my testing with IPSec AES-GCM 256
on my MCbin with 4 Cores I get a 50% speed increase:

Before the patch: 99.73 Kpps
With the patch: 151.25 Kpps

Signed-off-by: Sven Auhagen <sven.auhagen@voleatech.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Fix Unable to fit even 1 command desc error w/ EIP97

Due to the additions of support for modes like AES-CCM and AES-GCM, which
require large command tokens, the size of the descriptor has grown such that
it now does not fit into the descriptor cache of a standard EIP97 anymore.
This means that the driver no longer works on the Marvell Armada 3700LP chip
(as used on e.g. Espressobin) that it has always supported.
Additionally, performance on EIP197's like Marvell A8K may also degrade
due to being able to fit less descriptors in the on-chip cache.
Putting these tokens into the descriptor was really a hack and not how the
design was supposed to be used - resource allocation did not account for it.

So what this patch does, is move the command token out of the descriptor.
To avoid having to allocate buffers on the fly for these command tokens,
they are stuffed in a "shadow ring", which is a circular buffer of fixed
size blocks that runs in lock-step with the descriptor ring. i.e. there is
one token block per descriptor. The descriptor ring itself is then pre-
populated with the pointers to these token blocks so these do not need to
be filled in when building the descriptors later.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@rambus.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add missed clk_disable_unprepare

safexcel_remove misses disabling priv->reg_clk like what is done when
probe fails.
Add the missed call to fix it.

Signed-off-by: Chuhong Yuan <hslester96@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Fixed warnings on inconsistent byte order handling

This fixes a bunch of endianness related sparse warnings reported by the
kbuild test robot as well as Ben Dooks.

Credits for the fix to safexcel.c go to Ben Dooks.

Reported-by: kbuild test robot <lkp@intel.com>
Reported-by: Ben Dooks <ben.dooks@codethink.co.uk>
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Made locally used safexcel_pci_remove() static

safexcel_pci_remove() is only used locally in the module and not exported,
so added a static function specifier.
This fixes a sparse issue reported by Ben Dooks.

Fixes: 625f269a5a7a ("crypto: inside-secure - add support for...")
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Remove #ifdef checks

When both PCI and OF are disabled, no drivers are registered, and
we get some unused-function warnings:

drivers/crypto/inside-secure/safexcel.c:1221:13: error: unused function 'safexcel_unregister_algorithms' [-Werror,-Wunused-function]
static void safexcel_unregister_algorithms(struct safexcel_crypto_priv *priv)
drivers/crypto/inside-secure/safexcel.c:1307:12: error: unused function 'safexcel_probe_generic' [-Werror,-Wunused-function]
static int safexcel_probe_generic(void *pdev,
drivers/crypto/inside-secure/safexcel.c:1531:13: error: unused function 'safexcel_hw_reset_rings' [-Werror,-Wunused-function]
static void safexcel_hw_reset_rings(struct safexcel_crypto_priv *priv)

It's better to make the compiler see what is going on and remove
such ifdef checks completely. In case of PCI, this is trivial since
pci_register_driver() is defined to an empty function that makes the
compiler subsequently drop all unused code silently.

The global pcireg_rc/ofreg_rc variables are not actually needed here
since the driver registration does not fail in ways that would make
it helpful.

For CONFIG_OF, an IS_ENABLED() check is still required, since platform
drivers can exist both with and without it.

A little change to linux/pci.h is needed to ensure that
pcim_enable_device() is visible to the driver. Moving the declaration
outside of ifdef would be sufficient here, but for consistency with the
rest of the file, adding an inline helper is probably best.

Fixes: 212ef6f29e5b ("crypto: inside-secure - Fix unused variable warning when CONFIG_PCI=n")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Bjorn Helgaas <bhelgaas@google.com> # pci.h
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Fix a maybe-uninitialized warning

A previous fixup avoided an unused variable warning but replaced
it with a slightly scarier warning:

drivers/crypto/inside-secure/safexcel.c:1100:6: error: variable 'irq' is used uninitialized whenever 'if' condition is false [-Werror,-Wsometimes-uninitialized]

This is harmless as it is impossible to get into this case, but
the compiler has no way of knowing that. Add an explicit error
handling case to make it obvious to both compilers and humans
reading the source.

Fixes: 212ef6f29e5b ("crypto: inside-secure - Fix unused variable warning when CONFIG_PCI=n")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add support for the EIP196

This patch adds support for the EIP196, which is an EIP197 derivative
that has no classification hardware and a simplified record cache.

The patch has been tested with the eip196b-ie and eip197c-iewxkbc
configurations on the Xilinx VCU118 development board as well as on the
Macchiatobin board (Marvell A8K - EIP197b-ieswx), including the crypto
extra tests.

Note that this patchset applies on top of the earlier submitted
"Add support for eip197f_iewc" series.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add support for HW with less ring AIC's than rings

The current driver assumes one dedicated ring interrupt controller per
ring. However, some existing EIP(1)97 HW has less ring AIC's than rings.
This patch allows the driver to work with such HW by detecting how many
ring AIC's are present and restricting the number of rings it *uses* by
the number of ring AIC's present. This allows it to at least function.
(optimization for the future: add ring dispatch functionality in the
interrupt service routine such that multiple rings can be supported from
one ring AIC, allowing all rings to be used)

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add support for 256 bit wide internal bus

This patch adds support for large EIP197's with a 256 bit wide internal
bus, which affects the format of the result descriptor due to internal
alignment requirements.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for the rfc4309(ccm(aes)) AEAD

This patch adds support for rfc4309(ccm(aes)) for use with IPsec ESP

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for the rfc4543(gcm(aes)) "AEAD"

This patch adds support for rfc4543(gcm(aes)) - i.e. AES-GMAC - for use
with IPsec ESP

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for the rfc4106(gcm(aes)) AEAD

This patch adds support for rfc4106(gcm(aes)) for use with IPsec ESP

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Fixed corner case TRC admin RAM probing issue

This patch fixed a corner case admin RAM probing issue witnessed on the
Xilinx VCU118 FPGA development board with an EIP197 configuration with
4096 words of admin RAM, of which only 2050 were recognised.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Fix stability issue with Macchiatobin

This patch corrects an error in the Transform Record Cache initialization
code that was causing intermittent stability problems on the Macchiatobin
board.

Unfortunately, due to HW platform specifics, the problem could not happen
on the main development platform, being the VCU118 Xilinx development
board. And since it was a problem with hash table access, it was very
dependent on the actual physical context record DMA buffers being used,
i.e. with some (bad) luck it could seemingly work quit stable for a while.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for authenc HMAC-SHA2/DES-CBC

This patch adds support for the authenc(hmac(sha224),cbc(des)),
authenc(hmac(sha256),cbc(des)), authenc(hmac(sha384),cbc(des))
and authenc(hmac(sha512),cbc(des)) aead's

changes since v1:
- nothing

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for authenc HMAC-SHA2/3DES-CBC

This patch adds support for the authenc(hmac(sha224),cbc(des3_ede)),
authenc(hmac(sha256),cbc(des3_ede)), authenc(hmac(sha384),cbc(des3_ede))
and authenc(hmac(sha512),cbc(des3_ede)) aead's

changes since v1:
- nothing

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for authenc HMAC-SHA1/DES-CBC

This patch adds support for the authenc(hmac(sha1),cbc(des)) aead

changes since v1:
- rebased on top of DES changes made to cryptodev/master

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add HMAC-SHA3 family of authentication algorithms

This patch adds support for hmac(sha3-224), hmac(sha3-256), hmac(sha3-384)
and hmac(sha3-512) authentication algorithms.

The patch has been tested with the eip197c_iewxkbc configuration on the
Xilinx VCU118 development board, including the testmgr extra tests.

changes since v1:
- nothing

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add SHA3 family of basic hash algorithms

This patch adds support for sha3-224, sha3-256, sha3-384 and sha3-512
basic hashes.

The patch has been tested with the eip197c_iewxkbc configuration on the
Xilinx VCU118 development board, including the testmgr extra tests.

changes since v1:
- nothing

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add SM4 based authenc AEAD ciphersuites

This patch adds support for the authenc(hmac(sha1),cbc(sm4)),
authenc(hmac(sm3),cbc(sm4)), authenc(hmac(sha1),rfc3686(ctr(sm4))),
and authenc(hmac(sm3),rfc3686(ctr(sm4))) aead ciphersuites.
These are necessary to support IPsec according to the Chinese standard
GM/T 022-1014 - IPsec VPN specification.

Note that there are no testvectors present in testmgr for these
ciphersuites. However, considering all building blocks have already been
verified elsewhere, it is fair to assume the generic implementation to be
correct-by-construction.
The hardware implementation has been fuzzed against this generic
implementation by means of a locally modified testmgr. The intention is
to upstream these testmgr changes but this is pending other testmgr changes
being made by Eric Biggers.

The patch has been tested with the eip197c_iewxkbc configuration on the
Xilinx VCU118 development board, using the abovementioned modified testmgr

This patch applies on top of "Add support for SM4 ciphers" and needs to
be applied before "Add (HMAC) SHA3 support".

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add support for the rfc3685(ctr(sm4)) skcipher

This patch adds support for SM4 in (32 bit) CTR mode, i.e. skcipher
rfc3686(ctr(sm4)).

changes since v1:
- nothing

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add support for the cfb(sm4) skcipher

This patch adds support for SM4 in CFB mode, i.e. skcipher cfb(sm4).

changes since v1:
- nothing

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add support for the ofb(sm4) skcipher

This patch adds support for SM4 in OFB mode, i.e. skcipher ofb(sm4).

changes since v1:
- nothing

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add support for the cbc(sm4) skcipher

This patch adds support for SM4 in CBC mode, i.e. skcipher cbc(sm4).

changes since v1:
- nothing

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add support for the ecb(sm4) skcipher

This patch adds support for SM4 in ECB mode, i.e. skcipher ecb(sm4).

changes since v1:
- make SAFEXCEL_SM4 case entry explit, using the proper SM4_BLOCK_SIZE
instead of "borrowing" the AES code which "coincidentally" works

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for HMAC-SM3 ahash

Added support for the hmac(sm3) ahash authentication algorithm

changes since v1:
- added Acked-by tag below, no changes to the source

changes since v2:
- nothing

Acked-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for basic SM3 ahash

Added support for the SM3 ahash algorithm

changes since v1:
- moved definition of CONTEXT_CONTROL_CRYPTO_ALG_SM3 (0x7) up above 0xf

changes since v2:
- allow compilation if CONFIG_CRYPTO_SM3 is not set

Acked-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add support for the Chacha20-Poly1305 AEAD

This patch adds support for the Chacha20-Poly1305 cipher suite.
It adds both the basic rfc7539(chacha20,poly1305) as well as the
rfc7539esp(chacha20,poly1305) variant for IPsec ESP acceleration.

changes since v1:
- rebased on top of DES library changes done on cryptodev/master
- fixed crypto/Kconfig so that generic fallback is compiled as well

changes since v2:
- nothing

changes since v3:
- Fixed a problem where the tcrypt performance test would run fully on the
fallback cipher instead of the HW due to using an AAD length of 8 for
rfc7539esp. While this is not actually legal ESP (which includes SPI and
sequence number in the AAD as well), it is both inconvenient and not
necessary to run these vectors on the fallback cipher.
- Due to above, also realised that for plain (non-ESP) rfc7539, you
probably want to be able to run vectors with less than 8 bytes of AAD
on the HW, and this is actually possible as long as cryptlen is large
enough, so made that possible as well.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for the CHACHA20 skcipher

Added support for the CHACHA20 skcipher algorithm.
Tested on an eip197c-iesb configuration in the Xilinx VCU118 devboard,
passes all testmgr vectors plus the extra fuzzing tests.

changes since v1:
- rebased on top of DES library changes done on cryptodev/master
- fixed crypto/Kconfig so that generic fallback is compiled as well

changes since v2:
- made switch entry SAFEXCEL_AES explit and added empty default, as
requested by Antoine Tenart. Also needed to make SM4 patches apply.

changes since v3:
- nothing

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for the AES-CMAC ahash

This patch adds support for the AES-CMAC authentication algorithm.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for the AES XCBC ahash

This patch adds support for the AES XCBC authentication algorithm

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for the AES CBCMAC ahash

This patch adds support for the AES-CBCMAC authentication algorithm.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for CRC32

This patch adds support for the CRC32 "hash" algorithm

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Fix unused variable warning when CONFIG_PCI=n

This patch fixes an unused variable warning from the compiler when the
driver is being compiled without PCI support in the kernel.

Fixes: 625f269a5a7a ("crypto: inside-secure - add support for...")
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Probe transform record cache RAM sizes

This patch actually probes the transform record cache data and
administration RAM sizes, instead of making assumptions, and then
configures the TRC based on the actually probed values.
This allows the driver to work with EIP197 HW that has TRC RAM
sizes different from those of the Marvell EIP197B/D variants.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Base RD fetchcount on actual RD FIFO size

This patch derives the result descriptor fetch count from the actual
FIFO size advertised by the hardware. Fetching result descriptors
one at a time is a performance bottleneck for small blocks, especially
on hardware with multiple pipes. Even moreso if the HW has few rings.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Base CD fetchcount on actual CD FIFO size

This patch derives the command descriptor fetch count from the actual
FIFO size advertised by the hardware. Fetching command descriptors
one at a time is a performance bottleneck for small blocks, especially
on hardware with multiple pipes. Even moreso if the HW has few rings.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Enable extended algorithms on newer HW

This patch enables algorithms that did not fit the original 32 bit
FUNCTION_EN register anymore via the FUNCTION2_EN extension reg.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure: Corrected configuration of EIP96_TOKEN_CTRL

This patch corrects the configuration of the EIP197_PE_EIP96_TOKEN_CTRL
register. Previous value was wrong and potentially dangerous.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add EIP97/EIP197 and endianness detection

This patch adds automatic EIP97/EIP197 detection, so it does not need to
rely on any static value from the device table anymore. In particular,
the static value from the table won't work for PCI devboards that cannot
be further identified save from this direct hardware probing.

The patch also adds automatic host xs endianness detection & correction.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for basic AES-CCM

This patch adds support for the basic AES-CCM AEAD cipher suite.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added AES-OFB support

This patch adds support for AES in output feedback mode (AES-OFB).

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added AES-CFB support

This patch adds support for AES in 128 bit cipher feedback mode (AES-CFB).

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Added support for basic AES-GCM

This patch adds support for the basic AES-GCM AEAD cipher suite.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Only enable algorithms advertised by the hardware

This patch probes the supported algorithms from the hardware and only
registers the ones that the hardware actually supports. This is necessary
because this is a generic driver supposed to run on a wide variety of
engines, which may or may not implement certain algorithms.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Add support for the AES-XTS algorithm

This patch adds support for the AES-XTS skcipher algorithm.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - add support for using the EIP197 without vendor firmware

Until now, the inside-secure driver required a set of firmware images
supplied by the silicon vendor, typically under NDA, to be present in
/lib/firmware/inside-secure in order to be able to function.
This patch removes the dependence on this official vendor firmware by
falling back to generic "mini" FW - developed specifically for this
driver - that can be provided under GPL 2.0 through linux-firmwares.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Acked-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - add support for PCI based FPGA development board

This patch adds support for a PCIE development board with FPGA from Xilinx,
to facilitate pre-silicon driver development by both Inside Secure and its
IP customers. Since Inside Secure neither produces nor has access to actual
silicon, this is required functionality to allow us to contribute.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Acked-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - Remove redundant algo to engine mapping code

This removes some code determine which engine has which algorithms which
was effectively redundant (may have been forward-looking?) due to always
enabling all algorithms for all currently supported engines.
A future patch will use a different, more scalable approach to achieve
this. This is removed now because otherwise the next patch will add new
hardware which would otherwise have to be added to all algorithms, so
now is a convenient time to just get rid of this.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Acked-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - use devm_platform_ioremap_resource() to simplify code

Use devm_platform_ioremap_resource() to simplify the code a bit.
This is detected by coccinelle.

Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Acked-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - add support for authenc(hmac(sha*),rfc3686(ctr(aes))) suites

This patch adds support for the following AEAD ciphersuites:
- authenc(hmac(sha1),rfc3686(ctr(aes)))
- authenc(hmac(sha224),rfc3686(ctr(aes)))
- authenc(hmac(sha256),rfc3686(ctr(aes)))
- authenc(hmac(sha384),rfc3686(ctr(aes)))
- authenc(hmac(sha512),rfc3686(ctr(aes)))

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - added support for rfc3686(ctr(aes))

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - add support for authenc(hmac(sha1),cbc(des3_ede))

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - silently return -EINVAL for input error cases

Driver was printing an error message for certain input error cases that
should just return -EINVAL, which caused the related testmgr extra tests
to flood the kernel message log. Ensured those cases remain silent while
making some other device-specific errors a bit more verbose.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - enable context reuse

The context given to the crypto engine can be reused over time. While
the driver was designed to allow this, the feature wasn't enabled in the
hardware engine. This patch enables it.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - change returned error when a descriptor reports an error

This patch changes the error reported by the Inside Secure SafeXcel
driver when a result descriptor reports an error, from -EIO to -EINVAL,
as this is what the crypto framework expects. This was found while
running the crypto extra tests.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - improve the result error format when displayed

The result descriptors contain errors, which are represented as a
bitmap. This patch updates the error message to not treat the error as a
decimal value, but as an hexadecimal one. This helps in knowing the
value does not have a direct meaning (the set bits themselves have).

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - fix coding style for a condition

This cosmetic patch fixes a cosmetic issue with if brackets.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

treewide: Replace more open-coded allocation size multiplications

As done treewide earlier, this catches several more open-coded
allocation size calculations that were added to the kernel during the
merge window. This performs the following mechanical transformations
using Coccinelle:

kvmalloc(a * b, ...) -> kvmalloc_array(a, b, ...)
kvzalloc(a * b, ...) -> kvcalloc(a, b, ...)
devm_kzalloc(..., a * b, ...) -> devm_kcalloc(..., a, b, ...)

Signed-off-by: Kees Cook <keescook@chromium.org>

crypto: inside-secure - switch to SPDX identifiers

Use the appropriate SPDX license identifiers and drop the license text.
This patch is only cosmetic.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - remove request list to improve performance

This patch main goal is to improve driver performance by moving the
crypto request from a list to a RDR ring shadow.

This is possible since there is one producer and one consume for this
RDR request shadow and one ring descriptor is left unused.
Doing this change eliminates the use of spinlock when accessing the
descriptor ring and the need to dynamicaly allocate memory per crypto
request.

The crypto request is placed in the first RDR shadow descriptor only
if there are enough descriptors, when the result handler is invoked,
it fetches the first result descriptor from RDR shadow.

Signed-off-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - ecb(des3_ede) and cbc(des3_ede) support

This patch adds support for two new algorithms in the Inside Secure
SafeXcel cryptographic engine driver: ecb(des3_ede) and cbc(des3_ede).

Signed-off-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - ecb(des) and cbc(des) support

This patch adds support for two algorithms in the Inside Secure SafeXcel
cryptographic engine driver: ecb(des) and cbc(des).

Signed-off-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - hmac(md5) support

This patch adds support for the hmac(md5) algorithm in the Inside Secure
SafeXcel cryptographic engine driver.

Signed-off-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - md5 support

This patch adds the MD5 algorithm support to the Inside Secure SafeXcel
cryptographic engine driver.

Signed-off-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - set tx_max_cmd_queue to 32

The ORO bridge (connected to the EIP197 write channel) does not
generate back pressure towards the EIP197 when its internal FIFO is
full. It assumes that the EIP will not drive more write transactions
than the maximal supported outstanding (32).

Hence tx_max_cmd_queue must be configured to 5 (or less).

Signed-off-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - reset CDR and RDR rings on module removal

This patch adds extra steps in the module removal path, to reset the
command and result rings. The corresponding interrupts are cleared, and
the ring address configuration is reset.

Signed-off-by: Ofer Heifetz <oferh@marvell.com>
[Antoine: small reworks, commit message]
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - adjust the TRC configuration for EIP197D

This patch updates the TRC configuration so that the version of the
EIP197 engine being used is taken into account, as the configuration
differs between the EIP197B and the EIP197D.

Signed-off-by: Ofer Heifetz <oferh@marvell.com>
[Antoine: commit message]
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - eip197d support

This patch adds support for the eip197d engine to the Inside Secure
SafeXcel cryptographic driver. This new engine is similar to the eip197b
and reuse most of its code.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - add multiple processing engine support

So far a single processing engine (PE) was configured and used in the
Inside Secure SafeXcel cryptographic engine driver. Some versions have
more than a single PE. This patch rework the driver's initialization to
take this into account and to allow configuring more than one PE.

Signed-off-by: Ofer Heifetz <oferh@marvell.com>
[Antoine: some reworks and commit message.]
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - dynamic ring configuration allocation

The Inside Secure SafeXcel driver currently uses 4 rings, but the
eip197d engines has 8 of them. This patch updates the driver so that
rings are allocated dynamically based on the number of available rings
supported by a given engine.

Signed-off-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - add an invalidation flag

Add a flags field in the private structure, and a first flag for engines
needing context invalidation (currently only the eip197b). The
invalidation is needed when the engine includes a TRC cache, which will
also be true for the upcoming addition of the eip197d engine.

Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - filter out the algorithms by engine

EIP engines do not support the same set of algorithms. So far the
supported engines in the Inside Secure SafeXcel driver support the same
set of algorithms, but that won't be true for all engines. This patch
adds an 'engines' field in the algorithm definitions so that they only
are registered when using a compatible cryptographic engine.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - use precise compatibles

At first we used two compatibles in the SafeXcel driver, named after the
engine revision: eip97 and eip197. However this family of engines has
more precise versions and in fact we're supporting the eip97ies and
eip197b. More versions will be supported in the future, such as the
eip197d, and we'll need to differentiate them.

This patch fixes the compatibles used in the driver, to now use precise
ones. The two historical compatibles are kept for backward
compatibility.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - move the firmware to a better location

This patch moves the firmware loaded by the Inside Secure SafeXcel
driver from /lib/firmware/ to /lib/firmware/inside-secure/eip197b/. This
prepares the driver for future patches which will support other
revisions of the EIP197 crypto engine as they'll have their own
firmwares.

To keep the compatibility of what was done, the old path is still
supported as a fallback for the EIP197b (currently the only one
supported by the driver that loads a firmware).

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - authenc(hmac(sha384), cbc(aes)) support

This patch adds the authenc(hmac(sha384),cbc(aes)) algorithm support to
the Inside Secure SafeXcel driver.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - hmac(sha384) support

This patch adds the hmac(sha384) algorithm support to the Inside Secure
SafeXcel driver.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - sha384 support

This patch adds the sha384 algorithm support to the Inside Secure
SafeXcel driver.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - authenc(hmac(sha512), cbc(aes)) support

This patch adds the authenc(hmac(sha512),cbc(aes)) algorithm support to
the Inside Secure SafeXcel driver.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - hmac(sha512) support

This patch adds the hmac(sha512) algorithm support to the Inside Secure
SafeXcel driver.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - sha512 support

This patch adds the sha512 algorithm support to the Inside Secure
SafeXcel driver.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - increase minimum transfer size

The token size was increased for AEAD support. Occasional authentication
fails arise since the result descriptor overflows. This is because the
token size and the engine minimal thresholds must be in sync.

Signed-off-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - authenc(hmac(sha1), cbc(aes)) support

This patch adds the authenc(hmac(sha1),cbc(aes)) AEAD algorithm
support to the Inside Secure SafeXcel driver.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - authenc(hmac(sha224), cbc(aes)) support

This patch adds the authenc(hmac(sha224),cbc(aes)) AEAD algorithm
support to the Inside Secure SafeXcel driver.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - authenc(hmac(sha256), cbc(aes)) support

This patch adds support for the first AEAD algorithm in the Inside
Secure SafeXcel driver, authenc(hmac(sha256),cbc(aes)). As this is the
first AEAD algorithm added to this driver, common AEAD functions are
added as well.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - improve error reporting

This patch improves the error reporting from the Inside Secure driver to
the upper layers and crypto consumers. All errors reported by the engine
aren't fatal, and some may be genuine.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - do not use memset on MMIO

This patch fixes the Inside Secure driver which uses a memtset() call to
set an MMIO area from the cryptographic engine to 0. This is wrong as
memset() isn't guaranteed to work on MMIO for many reasons. This led to
kernel paging request panics in certain cases. Use memset_io() instead.

Fixes: 1b44c5a60c13 ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Reported-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - hmac(sha224) support

This patch adds the hmac(sha224) support to the Inside Secure
cryptographic engine driver.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - hmac(sha256) support

This patch adds the hmac(sha256) support to the Inside Secure
cryptographic engine driver.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - fix a typo in a register name

This patch fixes a typo in the EIP197_HIA_xDR_WR_CTRL_BUG register name,
as it should be EIP197_HIA_xDR_WR_CTRL_BUF. This is a cosmetic only
change.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - fix typo s/allways/always/ in a define

Small cosmetic patch fixing one typo in the
EIP197_HIA_DSE_CFG_ALLWAYS_BUFFERABLE macro, it should be _ALWAYS_.

Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - fix clock resource by adding a register clock

On Armada 7K/8K we need to explicitly enable the register clock. This
clock is optional because not all the SoCs using this IP need it but at
least for Armada 7K/8K it is actually mandatory.

The binding documentation is updated accordingly.

Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - improve clock initialization

The clock is optional, but if it is present we should managed it. If
there is an error while trying getting it, we should exit and report this
error.

So instead of returning an error only in the -EPROBE case, turn it in an
other way and ignore the clock only if it is not present (-ENOENT case).

Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - fix clock management

In this driver the clock is got but never put when the driver is removed
or if there is an error in the probe.

Using the managed version of clk_get() allows to let the kernel take care
of it.

Fixes: 1b44c5a60c13 ("crypto: inside-secure - add SafeXcel EIP197 crypto
engine driver")
cc: stable@vger.kernel.org
Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - move cache result dma mapping to request

In heavy traffic the DMA mapping is overwritten by multiple requests as
the DMA address is stored in a global context. This patch moves this
information to the per-hash request context so that it can't be
overwritten.

Fixes: 1b44c5a60c13 ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - move hash result dma mapping to request

In heavy traffic the DMA mapping is overwritten by multiple requests as
the DMA address is stored in a global context. This patch moves this
information to the per-hash request context so that it can't be
overwritten.

Fixes: 1b44c5a60c13 ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Signed-off-by: Ofer Heifetz <oferh@marvell.com>
[Antoine: rebased the patch, small fixes, commit message.]
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - keep the requests push/pop synced

This patch updates the Inside Secure SafeXcel driver to avoid being
out-of-sync between the number of requests sent and the one being
completed.

The number of requests acknowledged by the driver can be different than
the threshold that was configured if new requests were being pushed to
the h/w in the meantime. The driver wasn't taking those into account,
and the number of remaining requests to handled (to reconfigure the
interrupt threshold) could be out-of sync.

This patch fixes it by not taking in account the number of requests
left, but by taking in account the total number of requests being sent
to the hardware, so that new requests are being taken into account.

Fixes: dc7e28a3286e ("crypto: inside-secure - dequeue all requests at once")
Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - do not process request if no command was issued

This patch adds a check in the SafeXcel dequeue function, to avoid
processing request further if no hardware command was issued. This can
happen in certain cases where the ->send() function caches all the data
that would have been send.

Fixes: 809778e02cd4 ("crypto: inside-secure - fix hash when length is a multiple of a block")
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - do not overwrite the threshold value

This patch fixes the Inside Secure SafeXcel driver not to overwrite the
interrupt threshold value. In certain cases the value of this register,
which controls when to fire an interrupt, was overwritten. This lead to
packet not being processed or acked as the driver never was aware of
their completion.

This patch fixes this behaviour by not setting the threshold when
requests are being processed by the engine.

Fixes: dc7e28a3286e ("crypto: inside-secure - dequeue all requests at once")
Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - make function safexcel_try_push_requests static

The function safexcel_try_push_requests is local to the source and does
not need to be in global scope, so make it static.

Cleans up sparse warning:
symbol 'safexcel_try_push_requests' was not declared. Should it be static?

Signed-off-by: Colin Ian King <colin.king@canonical.com>
[Antoine: fixed alignment]
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - EIP97 support

The Inside Secure SafeXcel driver was firstly designed to support the
EIP197 cryptographic engine which is an evolution (with much more
feature, better performances) of the EIP97 cryptographic engine. This
patch convert the Inside Secure SafeXcel driver to support both engines
(EIP97 + EIP197).

The main differences are the register offsets and the context
invalidation process which is EIP197 specific. This patch adds an
indirection on the register offsets and adds checks not to send any
invalidation request when driving the EIP97. A new compatible is added
as well to bind the driver from device trees.

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - retry to proceed the request later on fail

The dequeueing function was putting back a request in the crypto queue
on failure (when not enough resources are available) which is not
perfect as the request will be handled much later. This patch updates
this logic by keeping a reference on the failed request to try
proceeding it later when enough resources are available.

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - handle more result requests when counter is full

This patch modifies the result handling logic to continue handling
results when the completed requests counter is full and not showing the
actual number of requests to handle.

Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - acknowledge the result requests all at once

This patches moves the result request acknowledgment from a per request
process to acknowledging all the result requests handled at once.

Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - dequeue all requests at once

This patch updates the dequeueing logic to dequeue all requests at once.
Since we can have many requests in the queue, the interrupt coalescing
is kept so that the ring interrupt fires every EIP197_MAX_BATCH_SZ at
most.

To allow dequeueing all requests at once while still using reasonable
settings for the interrupt coalescing, the result handling function was
updated to setup the threshold interrupt when needed (i.e. when more
requests than EIP197_MAX_BATCH_SZ are in the queue). When using this
capability the ring is marked as busy so that the dequeue function
enqueue new requests without setting the threshold interrupt.

Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - use threaded IRQs for result handling

This patch moves the result handling from an IRQ handler to a threaded
IRQ handler, to improve the number of complete requests being handled at
once.

Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - move request dequeueing into a workqueue

This patch moves the request dequeueing into a workqueue to improve the
coalescing of interrupts when sending requests to the engine; as the
engine is capable of having one single interrupt for n requests sent.
Using a workqueue allows to send more request at once.

Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - remove unused parameter in invalidate_cache

The SafeXcel context isn't used in the cache invalidation function. This
cosmetic patch removes it (as well as from the function prototype in the
header file and when the function is called).

Signed-off-by: Ofer Heifetz <oferh@marvell.com>
[Antoine: commit message]
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - free requests even if their handling failed

This patch frees the request private data even if its handling failed,
as it would never be freed otherwise.

Fixes: 1b44c5a60c13 ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - fix an error handling path in safexcel_probe()

'ret' is known to be 0 at this point.
If 'safexcel_request_ring_irq()' fails, it returns an error code.
Return this value instead of 0 which means success.

Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - do not parse the dma mask from dt

Remove the dma mask parsing from dt as this should not be encoded into
the engine device tree node. Keep the fallback value for now, which
should work for the boards already supported upstream.

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - only dequeue when needed

This force the need_dequeue flag to be unset whenever the dequeue
function is called, to avoid calling it when it is not necessary.

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - get the backlog before dequeueing the request

Get the backlog before dequeuing the request otherwise we'll miss the
first request in line.

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - stop requeueing failed requests

This update the dequeue function of the inside-secure safexcel driver so
that failed requests aren't requeued when they fail (for whatever
reason, which can be because the hw ring is full).

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - use one queue per hw ring

Update the inside-secure safexcel driver from using one global queue to
one queue per hw ring. This ease the request management and keep the hw
in sync with what's done in sw.

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - optimize DSE bufferability control

Configure the data write bufferability to always buffer packets in the
DSE. This change slightly improves performance.

Signed-off-by: Igal Liberman <igall@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - enable single WR in DSE configuration

When enable_single_wr is not enabled, the DSE will only write those
parts of a result descriptor that need updating, which means a final
result descriptor will be written in 2 or 3 smaller transfers.
When enable_single_wr is enabled the DSE will combine these 2-3
updates into one large write transfer, generally improving performance.

Signed-off-by: Igal Liberman <igall@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - fix incorrect DSE data cache setting

Set the correct value to the DSE data cache, using WR_CACHE_3BITS
instead of RD_CACHE_3BITS. This fixes an incorrect setting and helps
improving performances.

Reported-by: Igal Liberman <igall@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: inside-secure - add SafeXcel EIP197 crypto engine driver

Add support for Inside Secure SafeXcel EIP197 cryptographic engine,
which can be found on Marvell Armada 7k and 8k boards. This driver
currently implements: ecb(aes), cbc(aes), sha1, sha224, sha256 and
hmac(sah1) algorithms.

Two firmwares are needed for this engine to work. Their are mostly used
for more advanced operations than the ones supported (as of now), but we
still need them to pass the data to the internal cryptographic engine.

Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>