crypto: hisilicon - Use request_complete helpers

Use the request_complete helpers instead of calling the completion
function directly.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Kunpeng916 crypto driver don't sleep when in softirq

When kunpeng916 encryption driver is used to deencrypt and decrypt
packets during the softirq, it is not allowed to use mutex lock.

Fixes: 915e4e8413da ("crypto: hisilicon - SEC security accelerator driver")
Signed-off-by: Zhengchao Shao <shaozhengchao@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - use the correct HiSilicon copyright

s/Hisilicon/HiSilicon/g,
according to https://www.hisilicon.com/en/terms-of-use.

Signed-off-by: Hao Fang <fanghao11@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: drivers - set the flag CRYPTO_ALG_ALLOCATES_MEMORY

Set the flag CRYPTO_ALG_ALLOCATES_MEMORY in the crypto drivers that
allocate memory.

drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c: sun8i_ce_cipher
drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c: sun8i_ss_cipher
drivers/crypto/amlogic/amlogic-gxl-core.c: meson_cipher
drivers/crypto/axis/artpec6_crypto.c: artpec6_crypto_common_init
drivers/crypto/bcm/cipher.c: spu_skcipher_rx_sg_create
drivers/crypto/caam/caamalg.c: aead_edesc_alloc
drivers/crypto/caam/caamalg_qi.c: aead_edesc_alloc
drivers/crypto/caam/caamalg_qi2.c: aead_edesc_alloc
drivers/crypto/caam/caamhash.c: hash_digest_key
drivers/crypto/cavium/cpt/cptvf_algs.c: process_request
drivers/crypto/cavium/nitrox/nitrox_aead.c: nitrox_process_se_request
drivers/crypto/cavium/nitrox/nitrox_skcipher.c: nitrox_process_se_request
drivers/crypto/ccp/ccp-crypto-aes-cmac.c: ccp_do_cmac_update
drivers/crypto/ccp/ccp-crypto-aes-galois.c: ccp_crypto_enqueue_request
drivers/crypto/ccp/ccp-crypto-aes-xts.c: ccp_crypto_enqueue_request
drivers/crypto/ccp/ccp-crypto-aes.c: ccp_crypto_enqueue_request
drivers/crypto/ccp/ccp-crypto-des3.c: ccp_crypto_enqueue_request
drivers/crypto/ccp/ccp-crypto-sha.c: ccp_crypto_enqueue_request
drivers/crypto/chelsio/chcr_algo.c: create_cipher_wr
drivers/crypto/hisilicon/sec/sec_algs.c: sec_alloc_and_fill_hw_sgl
drivers/crypto/hisilicon/sec2/sec_crypto.c: sec_alloc_req_id
drivers/crypto/inside-secure/safexcel_cipher.c: safexcel_queue_req
drivers/crypto/inside-secure/safexcel_hash.c: safexcel_ahash_enqueue
drivers/crypto/ixp4xx_crypto.c: ablk_perform
drivers/crypto/marvell/cesa/cipher.c: mv_cesa_skcipher_dma_req_init
drivers/crypto/marvell/cesa/hash.c: mv_cesa_ahash_dma_req_init
drivers/crypto/marvell/octeontx/otx_cptvf_algs.c: create_ctx_hdr
drivers/crypto/n2_core.c: n2_compute_chunks
drivers/crypto/picoxcell_crypto.c: spacc_sg_to_ddt
drivers/crypto/qat/qat_common/qat_algs.c: qat_alg_skcipher_encrypt
drivers/crypto/qce/skcipher.c: qce_skcipher_async_req_handle
drivers/crypto/talitos.c : talitos_edesc_alloc
drivers/crypto/virtio/virtio_crypto_algs.c: __virtio_crypto_skcipher_do_req
drivers/crypto/xilinx/zynqmp-aes-gcm.c: zynqmp_aes_aead_cipher

Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
[EB: avoid overly-long lines]
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - don't sleep of CRYPTO_TFM_REQ_MAY_SLEEP was not specified

There is this call chain:
sec_alg_skcipher_encrypt -> sec_alg_skcipher_crypto ->
sec_alg_alloc_and_calc_split_sizes -> kcalloc
where we call sleeping allocator function even if CRYPTO_TFM_REQ_MAY_SLEEP
was not specified.

Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Cc: stable@vger.kernel.org # v4.19+
Fixes: 915e4e8413da ("crypto: hisilicon - SEC security accelerator driver")
Acked-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Matching the dma address for dma_pool_free()

When dma_pool_zalloc() fail in sec_alloc_and_fill_hw_sgl(),
dma_pool_free() is invoked, but the parameters that sgl_current and
sgl_current->next_sgl is not match.

Using sec_free_hw_sgl() instead of the original free routine.

Fixes: 915e4e8413da ("crypto: hisilicon - SEC security accelerator driver")
Signed-off-by: Yunfeng Ye <yeyunfeng@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Fix double free in sec_free_hw_sgl()

There are two problems in sec_free_hw_sgl():

First, when sgl_current->next is valid, @hw_sgl will be freed in the
first loop, but it free again after the loop.

Second, sgl_current and sgl_current->next_sgl is not match when
dma_pool_free() is invoked, the third parameter should be the dma
address of sgl_current, but sgl_current->next_sgl is the dma address
of next chain, so use sgl_current->next_sgl is wrong.

Fix this by deleting the last dma_pool_free() in sec_free_hw_sgl(),
modifying the condition for while loop, and matching the address for
dma_pool_free().

Fixes: 915e4e8413da ("crypto: hisilicon - SEC security accelerator driver")
Signed-off-by: Yunfeng Ye <yeyunfeng@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/des - switch to new verification routines

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Forbid 2-key 3DES in FIPS mode

This patch forbids the use of 2-key 3DES (K1 == K3) in FIPS mode.

It also removes a couple of unnecessary key length checks that
are already performed by the crypto API.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

cross-tree: phase out dma_zalloc_coherent()

We already need to zero out memory for dma_alloc_coherent(), as such
using dma_zalloc_coherent() is superflous. Phase it out.

This change was generated with the following Coccinelle SmPL patch:

@ replace_dma_zalloc_coherent @
expression dev, size, data, handle, flags;
@@

-dma_zalloc_coherent(dev, size, handle, flags)
+dma_alloc_coherent(dev, size, handle, flags)

Suggested-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
[hch: re-ran the script on the latest tree]
Signed-off-by: Christoph Hellwig <hch@lst.de>

crypto: hisilicon - Fix reference after free of memories on error path

coccicheck currently warns of the following issues in the driver:
drivers/crypto/hisilicon/sec/sec_algs.c:864:51-66: ERROR: reference preceded by free on line 812
drivers/crypto/hisilicon/sec/sec_algs.c:864:40-49: ERROR: reference preceded by free on line 813
drivers/crypto/hisilicon/sec/sec_algs.c:861:8-24: ERROR: reference preceded by free on line 814
drivers/crypto/hisilicon/sec/sec_algs.c:860:41-51: ERROR: reference preceded by free on line 815
drivers/crypto/hisilicon/sec/sec_algs.c:867:7-18: ERROR: reference preceded by free on line 816

It would appear than on certain error paths that we may attempt reference-
after-free some memories.

This patch fixes those issues. The solution doesn't look perfect, but
having same memories free'd possibly from separate functions makes it
tricky.

Fixes: 915e4e8413da ("crypto: hisilicon - SEC security accelerator driver")
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Fix NULL dereference for same dst and src

When the source and destination addresses for the cipher are the same, we
will get a NULL dereference from accessing the split destination
scatterlist memories, as shown:

[ 56.565719] tcrypt:
[ 56.565719] testing speed of async ecb(aes) (hisi_sec_aes_ecb) encryption
[ 56.574683] tcrypt: test 0 (128 bit key, 16 byte blocks):
[ 56.587585] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000
[ 56.596361] Mem abort info:
[ 56.599151] ESR = 0x96000006
[ 56.602196] Exception class = DABT (current EL), IL = 32 bits
[ 56.608105] SET = 0, FnV = 0
[ 56.611149] EA = 0, S1PTW = 0
[ 56.614280] Data abort info:
[ 56.617151] ISV = 0, ISS = 0x00000006
[ 56.620976] CM = 0, WnR = 0
[ 56.623930] user pgtable: 4k pages, 48-bit VAs, pgdp = (____ptrval____)
[ 56.630533] [0000000000000000] pgd=0000041fc7e4d003, pud=0000041fcd9bf003, pmd=0000000000000000
[ 56.639224] Internal error: Oops: 96000006 [#1] PREEMPT SMP
[ 56.644782] Modules linked in: tcrypt(+)
[ 56.648695] CPU: 21 PID: 2326 Comm: insmod Tainted: G W 4.19.0-rc6-00001-g3fabfb8-dirty #716
[ 56.658420] Hardware name: Huawei Taishan 2280 /D05, BIOS Hisilicon D05 IT17 Nemo 2.0 RC0 10/05/2018
[ 56.667537] pstate: 20000005 (nzCv daif -PAN -UAO)
[ 56.672322] pc : sec_alg_skcipher_crypto+0x318/0x748
[ 56.677274] lr : sec_alg_skcipher_crypto+0x178/0x748
[ 56.682224] sp : ffff0000118e3840
[ 56.685525] x29: ffff0000118e3840 x28: ffff841fbb3f8118
[ 56.690825] x27: 0000000000000000 x26: 0000000000000000
[ 56.696125] x25: ffff841fbb3f8080 x24: ffff841fbadc0018
[ 56.701425] x23: ffff000009119000 x22: ffff841fbb24e280
[ 56.706724] x21: ffff841ff212e780 x20: ffff841ff212e700
[ 56.712023] x19: 0000000000000001 x18: ffffffffffffffff
[ 56.717322] x17: 0000000000000000 x16: 0000000000000000
[ 56.722621] x15: ffff0000091196c8 x14: 72635f7265687069
[ 56.727920] x13: 636b735f676c615f x12: ffff000009119940
[ 56.733219] x11: 0000000000000000 x10: 00000000006080c0
[ 56.738519] x9 : 0000000000000000 x8 : ffff841fbb24e480
[ 56.743818] x7 : ffff841fbb24e500 x6 : ffff841ff00cdcc0
[ 56.749117] x5 : 0000000000000010 x4 : 0000000000000000
[ 56.754416] x3 : ffff841fbb24e380 x2 : ffff841fbb24e480
[ 56.759715] x1 : 0000000000000000 x0 : ffff000008f682c8
[ 56.765016] Process insmod (pid: 2326, stack limit = 0x(____ptrval____))
[ 56.771702] Call trace:
[ 56.774136] sec_alg_skcipher_crypto+0x318/0x748
[ 56.778740] sec_alg_skcipher_encrypt+0x10/0x18
[ 56.783259] test_skcipher_speed+0x2a0/0x700 [tcrypt]
[ 56.788298] do_test+0x18f8/0x48c8 [tcrypt]
[ 56.792469] tcrypt_mod_init+0x60/0x1000 [tcrypt]
[ 56.797161] do_one_initcall+0x5c/0x178
[ 56.800985] do_init_module+0x58/0x1b4
[ 56.804721] load_module+0x1da4/0x2150
[ 56.808456] __se_sys_init_module+0x14c/0x1e8
[ 56.812799] __arm64_sys_init_module+0x18/0x20
[ 56.817231] el0_svc_common+0x60/0xe8
[ 56.820880] el0_svc_handler+0x2c/0x80
[ 56.824615] el0_svc+0x8/0xc
[ 56.827483] Code: a94c87a3 910b2000 f87b7842 f9004ba2 (b87b7821)
[ 56.833564] ---[ end trace 0f63290590e93d94 ]---
Segmentation fault

Fix this by only accessing these memories when we have different src and
dst.

Fixes: 915e4e8413da ("crypto: hisilicon - SEC security accelerator driver")
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - sec_send_request() can be static

Fixes: 915e4e8413da ("crypto: hisilicon - SEC security accelerator driver")
Signed-off-by: kbuild test robot <fengguang.wu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - SEC security accelerator driver

This accelerator is found inside hisilicon hip06 and hip07 SoCs.
Each instance provides a number of queues which feed a different number of
backend acceleration units.

The queues are operating in an out of order mode in the interests of
throughput. The silicon does not do tracking of dependencies between
multiple 'messages' or update of the IVs as appropriate for training.
Hence where relevant we need to do this in software.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>