crypto: ccp - Failure on re-initialization due to duplicate sysfs filename

The following warning appears during the CCP module re-initialization:

[ 140.965403] sysfs: cannot create duplicate filename
'/devices/pci0000:00/0000:00:07.1/0000:03:00.2/dma/dma0chan0'
[ 140.975736] CPU: 0 PID: 388 Comm: kworker/0:2 Kdump: loaded Not
tainted 6.2.0-0.rc2.18.eln124.x86_64 #1
[ 140.985185] Hardware name: HPE ProLiant DL325 Gen10/ProLiant DL325
Gen10, BIOS A41 07/17/2020
[ 140.993761] Workqueue: events work_for_cpu_fn
[ 140.998151] Call Trace:
[ 141.000613] <TASK>
[ 141.002726] dump_stack_lvl+0x33/0x46
[ 141.006415] sysfs_warn_dup.cold+0x17/0x23
[ 141.010542] sysfs_create_dir_ns+0xba/0xd0
[ 141.014670] kobject_add_internal+0xba/0x260
[ 141.018970] kobject_add+0x81/0xb0
[ 141.022395] device_add+0xdc/0x7e0
[ 141.025822] ? complete_all+0x20/0x90
[ 141.029510] __dma_async_device_channel_register+0xc9/0x130
[ 141.035119] dma_async_device_register+0x19e/0x3b0
[ 141.039943] ccp_dmaengine_register+0x334/0x3f0 [ccp]
[ 141.045042] ccp5_init+0x662/0x6a0 [ccp]
[ 141.049000] ? devm_kmalloc+0x40/0xd0
[ 141.052688] ccp_dev_init+0xbb/0xf0 [ccp]
[ 141.056732] ? __pci_set_master+0x56/0xd0
[ 141.060768] sp_init+0x70/0x90 [ccp]
[ 141.064377] sp_pci_probe+0x186/0x1b0 [ccp]
[ 141.068596] local_pci_probe+0x41/0x80
[ 141.072374] work_for_cpu_fn+0x16/0x20
[ 141.076145] process_one_work+0x1c8/0x380
[ 141.080181] worker_thread+0x1ab/0x380
[ 141.083953] ? __pfx_worker_thread+0x10/0x10
[ 141.088250] kthread+0xda/0x100
[ 141.091413] ? __pfx_kthread+0x10/0x10
[ 141.095185] ret_from_fork+0x2c/0x50
[ 141.098788] </TASK>
[ 141.100996] kobject_add_internal failed for dma0chan0 with -EEXIST,
don't try to register things with the same name in the same directory.
[ 141.113703] ccp 0000:03:00.2: ccp initialization failed

The /dma/dma0chan0 sysfs file is not removed since dma_chan object
has been released in ccp_dma_release() before releasing dma device.
A correct procedure would be: release dma channels first => unregister
dma device => release ccp dma object.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=216888
Fixes: 68dbe80f5b51 ("crypto: ccp - Release dma channels before dmaengine unrgister")
Tested-by: Vladis Dronov <vdronov@redhat.com>
Signed-off-by: Koba Ko <koba.ko@canonical.com>
Reviewed-by: Vladis Dronov <vdronov@redhat.com>
Acked-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - Release dma channels before dmaengine unrgister

A warning is shown during shutdown,

__dma_async_device_channel_unregister called while 2 clients hold a reference
WARNING: CPU: 15 PID: 1 at drivers/dma/dmaengine.c:1110 __dma_async_device_channel_unregister+0xb7/0xc0

Call dma_release_channel for occupied channles before dma_async_device_unregister.

Fixes: 54cce8ecb925 ("crypto: ccp - ccp_dmaengine_unregister release dma channels")
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Koba Ko <koba.ko@canonical.com>
Acked-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - ccp_dmaengine_unregister release dma channels

ccp_dmaengine_register adds dma_chan->device_node to dma_dev->channels list
but ccp_dmaengine_unregister didn't remove them.
That can cause crashes in various dmaengine methods that tries to use dma_dev->channels

Fixes: 58ea8abf4904 ("crypto: ccp - Register the CCP as a DMA...")
Signed-off-by: Dāvis Mosāns <davispuh@gmail.com>
Acked-by: John Allen <john.allen@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - Use list_move_tail instead of list_del/list_add_tail in ccp-dmaengine.c

Using list_move_tail() instead of list_del() + list_add_tail() in ccp-dmaengine.c.

Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Baokun Li <libaokun1@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - fix uninitialized list head

A NULL-pointer dereference was reported in fedora bz#1762199 while
reshaping a raid6 array after adding a fifth drive to an existing
array.

[ 47.343549] md/raid:md0: raid level 6 active with 3 out of 5 devices, algorithm 2
[ 47.804017] md0: detected capacity change from 0 to 7885289422848
[ 47.822083] Unable to handle kernel read from unreadable memory at virtual address 0000000000000000
...
[ 47.940477] CPU: 1 PID: 14210 Comm: md0_raid6 Tainted: G W 5.2.18-200.fc30.aarch64 #1
[ 47.949594] Hardware name: AMD Overdrive/Supercharger/To be filled by O.E.M., BIOS ROD1002C 04/08/2016
[ 47.958886] pstate: 00400085 (nzcv daIf +PAN -UAO)
[ 47.963668] pc : __list_del_entry_valid+0x2c/0xa8
[ 47.968366] lr : ccp_tx_submit+0x84/0x168 [ccp]
[ 47.972882] sp : ffff00001369b970
[ 47.976184] x29: ffff00001369b970 x28: ffff00001369bdb8
[ 47.981483] x27: 00000000ffffffff x26: ffff8003b758af70
[ 47.986782] x25: ffff8003b758b2d8 x24: ffff8003e6245818
[ 47.992080] x23: 0000000000000000 x22: ffff8003e62450c0
[ 47.997379] x21: ffff8003dfd6add8 x20: 0000000000000003
[ 48.002678] x19: ffff8003e6245100 x18: 0000000000000000
[ 48.007976] x17: 0000000000000000 x16: 0000000000000000
[ 48.013274] x15: 0000000000000000 x14: 0000000000000000
[ 48.018572] x13: ffff7e000ef83a00 x12: 0000000000000001
[ 48.023870] x11: ffff000010eff998 x10: 00000000000019a0
[ 48.029169] x9 : 0000000000000000 x8 : ffff8003e6245180
[ 48.034467] x7 : 0000000000000000 x6 : 000000000000003f
[ 48.039766] x5 : 0000000000000040 x4 : ffff8003e0145080
[ 48.045064] x3 : dead000000000200 x2 : 0000000000000000
[ 48.050362] x1 : 0000000000000000 x0 : ffff8003e62450c0
[ 48.055660] Call trace:
[ 48.058095] __list_del_entry_valid+0x2c/0xa8
[ 48.062442] ccp_tx_submit+0x84/0x168 [ccp]
[ 48.066615] async_tx_submit+0x224/0x368 [async_tx]
[ 48.071480] async_trigger_callback+0x68/0xfc [async_tx]
[ 48.076784] ops_run_biofill+0x178/0x1e8 [raid456]
[ 48.081566] raid_run_ops+0x248/0x818 [raid456]
[ 48.086086] handle_stripe+0x864/0x1208 [raid456]
[ 48.090781] handle_active_stripes.isra.0+0xb0/0x278 [raid456]
[ 48.096604] raid5d+0x378/0x618 [raid456]
[ 48.100602] md_thread+0xa0/0x150
[ 48.103905] kthread+0x104/0x130
[ 48.107122] ret_from_fork+0x10/0x18
[ 48.110686] Code: d2804003 f2fbd5a3 eb03003f 54000320 (f9400021)
[ 48.116766] ---[ end trace 23f390a527f7ad77 ]---

ccp_tx_submit is passed a dma_async_tx_descriptor which is contained in
a ccp_dma_desc and adds it to a ccp channel's pending list:

list_del(&desc->entry);
list_add_tail(&desc->entry, &chan->pending);

The problem is that desc->entry may be uninitialized in the
async_trigger_callback path where the descriptor was gotten
from ccp_prep_dma_interrupt which got it from ccp_alloc_dma_desc
which doesn't initialize the desc->entry list head. So, just
initialize the list head to avoid the problem.

Cc: <stable@vger.kernel.org>
Reported-by: Sahaj Sarup <sahajsarup@gmail.com>
Signed-off-by: Mark Salter <msalter@redhat.com>
Acked-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - Include DMA declarations explicitly

ccp-dev.h uses dma_direction, which is defined in linux/dma-direction.h.
Include that explicitly instead of relying on it being included via
linux/pci.h, since ccp-dev.h requires nothing else from linux/pci.h.

Similarly, ccp-dmaengine.c uses dma_get_mask(), which is defined in
linux/dma-mapping.h, so include that explicitly since it requires nothing
else from linux/pci.h.

A future patch will remove the includes of linux/pci.h where it is not
needed.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - Add a module parameter to control registration for DMA

The CCP driver is able to act as a DMA engine. Add a module parameter that
allows this feature to be enabled/disabled.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - Switch to SPDX license identifiers

Add an SPDX identifier and remove any specific statements.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500

Based on 2 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation #

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-only

has been chosen to replace the boilerplate/reference in 4122 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Enrico Weigelt <info@metux.net>
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Allison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

crypto: ccp - Make function ccp_get_dma_chan_attr static

Function ccp_get_dma_chan_attr is local to the source and does not
need to be in global scope, so make it static.

Cleans up sparse warning:
drivers/crypto/ccp/ccp-dmaengine.c:41:14: warning: symbol
'ccp_get_dma_chan_attr' was not declared. Should it be static?

Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto:ccp - invoke the DMA callback in a standard way

Use the provided mechanism in dmaengine.h to invoke the
completion callback.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - unmap pages and remove unmap objects in callback

Clean up the mapped pages and the unmap object once we are done with
it. This enables the final clean-up of the object once the transfer
is complete.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

dmaengine: remove DMA_SG as it is dead code in kernel

There are no in kernel consumers for DMA_SG op. Removing operation,
dead code, and test code in dmatest.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Cc: Gary Hook <gary.hook@amd.com>
Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
Cc: Kedareswara rao Appana <appana.durga.rao@xilinx.com>
Cc: Li Yang <leoyang.li@nxp.com>
Cc: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

crypto: ccp - Update copyright dates for 2017.

Some updates this year have not had copyright dates changed in modified
files. Correct this for 2017.

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - Make some CCP DMA channels private

The CCP registers its queues as channels capable of handling
general DMA operations. The NTB driver will use DMA if
directed, but as public channels can be reserved for use in
asynchronous operations some channels should be held back
as private. Since the public/private determination is
handled at a device level, reserve the "other" (secondary)
CCP channels as private.

Add a module parameter that allows for override, to be
applied to all channels on all devices.

CC: <stable@vger.kernel.org> # 4.10.x-
Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - Assign DMA commands to the channel's CCP

The CCP driver generally uses a round-robin approach when
assigning operations to available CCPs. For the DMA engine,
however, the DMA mappings of the SGs are associated with a
specific CCP. When an IOMMU is enabled, the IOMMU is
programmed based on this specific device.

If the DMA operations are not performed by that specific
CCP then addressing errors and I/O page faults will occur.

Update the CCP driver to allow a specific CCP device to be
requested for an operation and use this in the DMA engine
support.

Cc: <stable@vger.kernel.org> # 4.9.x-
Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - Fix double add when creating new DMA command

Eliminate a double-add by creating a new list to manage
command descriptors when created; move the descriptor to
the pending list when the command is submitted.

Cc: <stable@vger.kernel.org>
Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - Fix return value check in ccp_dmaengine_register()

Fix the retrn value check which testing the wrong variable
in ccp_dmaengine_register().

Fixes: 58ea8abf4904 ("crypto: ccp - Register the CCP as a DMA resource")
Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com>
Acked-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - use kmem_cache_zalloc instead of kmem_cache_alloc/memset

Using kmem_cache_zalloc() instead of kmem_cache_alloc() and memset().

Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com>
Acked-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - add missing release in ccp_dmaengine_register

ccp_dmaengine_register used to return with an error code before
releasing all resource. This patch adds a jump to the appropriate label
ensuring that the resources are properly released before returning.

This issue was found with Hector.

Signed-off-by: Quentin Lambert <lambert.quentin@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: ccp - Register the CCP as a DMA resource

The CCP has the ability to provide DMA services to the
kernel using pass-through mode of the device. Register
these services as general purpose DMA channels.

Changes since v2:
- Add a Signed-off-by

Changes since v1:
- Allocate memory for a string in ccp_dmaengine_register
- Ensure register/unregister calls are properly ordered
- Verified all changed files are listed in the diffstat
- Undo some superfluous changes
- Added a cc:

Signed-off-by: Gary R Hook <gary.hook@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>