crypto: hisilicon/qm - change function type to void

The function qm_stop_qp_nolock() always return zero, so
function type is changed to void.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - obtain stop queue status

The debugfs files 'dev_state' and 'dev_timeout' are added.
Users can query the current queue stop status through these two
files. And set the waiting timeout when the queue is released.

dev_state: if dev_timeout is set, dev_state indicates the status
of stopping the queue. 0 indicates that the queue is stopped
successfully. Other values indicate that the queue stops fail.
If dev_timeout is not set, the value of dev_state is 0;

dev_timeout: if the queue fails to stop, the queue is released
after waiting dev_timeout * 20ms.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add stop function by hardware

Hardware V3 could be able to drain function by sending mailbox
to hardware which will trigger tasks in device to be flushed out.
When the function is reset, the function can be stopped by this way.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - delete a dbg function

Deleted a dbg function because this function has the risk of
address leakage. In addition, this function is only used for
debugging in the early stage and is not required in the future.

Signed-off-by: Chenghai Huang <huangchenghai2@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - save capability registers in qm init process

In previous capability register implementation, qm irq related values
were read from capability registers dynamically when needed. But in
abnormal scenario, e.g. the core is timeout and the device needs to
soft reset and reset failed after disabling the MSE, the device can
not be removed normally, causing the following call trace:

| Call trace:
| pci_irq_vector+0xfc/0x140
| hisi_qm_uninit+0x278/0x3b0 [hisi_qm]
| hpre_remove+0x16c/0x1c0 [hisi_hpre]
| pci_device_remove+0x6c/0x264
| device_release_driver_internal+0x1ec/0x3e0
| device_release_driver+0x3c/0x60
| pci_stop_bus_device+0xfc/0x22c
| pci_stop_and_remove_bus_device+0x38/0x70
| pci_iov_remove_virtfn+0x108/0x1c0
| sriov_disable+0x7c/0x1e4
| pci_disable_sriov+0x4c/0x6c
| hisi_qm_sriov_disable+0x90/0x160 [hisi_qm]
| hpre_remove+0x1a8/0x1c0 [hisi_hpre]
| pci_device_remove+0x6c/0x264
| device_release_driver_internal+0x1ec/0x3e0
| driver_detach+0x168/0x2d0
| bus_remove_driver+0xc0/0x230
| driver_unregister+0x58/0xdc
| pci_unregister_driver+0x40/0x220
| hpre_exit+0x34/0x64 [hisi_hpre]
| __arm64_sys_delete_module+0x374/0x620
[...]

| Call trace:
| free_msi_irqs+0x25c/0x300
| pci_disable_msi+0x19c/0x264
| pci_free_irq_vectors+0x4c/0x70
| hisi_qm_pci_uninit+0x44/0x90 [hisi_qm]
| hisi_qm_uninit+0x28c/0x3b0 [hisi_qm]
| hpre_remove+0x16c/0x1c0 [hisi_hpre]
| pci_device_remove+0x6c/0x264
[...]

The reason for this call trace is that when the MSE is disabled, the value
of capability registers in the BAR space become invalid. This will make the
subsequent unregister process get the wrong irq vector through capability
registers and get the wrong irq number by pci_irq_vector().

So add a capability table structure to pre-store the valid value of the irq
information capability register in qm init process, avoid obtaining invalid
capability register value after the MSE is disabled.

Fixes: 3536cc55cada ("crypto: hisilicon/qm - support get device irq information from hardware registers")
Signed-off-by: Zhiqi Song <songzhiqi1@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add a function to set qm algs

Extract a public function to set qm algs and remove
the similar code for setting qm algs in each module.

Signed-off-by: Wenkai Lin <linwenkai6@hisilicon.com>
Signed-off-by: Hao Fang <fanghao11@huawei.com>
Signed-off-by: Zhiqi Song <songzhiqi1@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add comments and remove redundant array element

1. Remove redundant array element, prevent the size obtained by
ARRAY_SIZE() from qm_log_hw_error is greater than actual size.
2. Add comments in function qm_set_vf_mse() and qm_cq_ctx_cfg()
to make it easier to understand.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - simplify the status of qm

The 'QM_INIT' and 'QM_CLOSE' status of qm and 'QP_INIT'
and 'QP_CLOSE' status of queue are not actually used. Currently,
driver only needs to switch status when the device or queue
is enabled or stopped, Therefore, remove unneeded status to
simplify driver. In addition, rename'QM_START to'QM_WORK' for
ease to understand.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - print device abnormal information

When device is abnormal and reports abnormal interrupt event to driver,
the driver can print device information for error analysis. This patch
adds some device error-related information output after the device reports
an abnormal interrupt.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - prevent soft lockup in receive loop

In the scenario where the accelerator business is fully loaded.
When the workqueue receiving messages and performing callback
processing, there are a large number of messages that need to be
received, and there are continuously messages that have been
processed and need to be received.
This will cause the receive loop here to be locked for a long time.
This scenario will cause watchdog timeout problems on OS with kernel
preemption turned off.

The error logs:
watchdog: BUG: soft lockup - CPU#23 stuck for 23s! [kworker/u262:1:1407]
[ 1461.978428][ C23] Call trace:
[ 1461.981890][ C23] complete+0x8c/0xf0
[ 1461.986031][ C23] kcryptd_async_done+0x154/0x1f4 [dm_crypt]
[ 1461.992154][ C23] sec_skcipher_callback+0x7c/0xf4 [hisi_sec2]
[ 1461.998446][ C23] sec_req_cb+0x104/0x1f4 [hisi_sec2]
[ 1462.003950][ C23] qm_poll_req_cb+0xcc/0x150 [hisi_qm]
[ 1462.009531][ C23] qm_work_process+0x60/0xc0 [hisi_qm]
[ 1462.015101][ C23] process_one_work+0x1c4/0x470
[ 1462.020052][ C23] worker_thread+0x150/0x3c4
[ 1462.024735][ C23] kthread+0x108/0x13c
[ 1462.028889][ C23] ret_from_fork+0x10/0x18

Therefore, it is necessary to add an actively scheduled operation in the
while loop to prevent this problem.
After adding it, no matter whether the OS turns on or off the kernel
preemption function. Neither will cause watchdog timeout issues.

Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix EQ/AEQ interrupt issue

During hisilicon accelerator live migration operation. In order to
prevent the problem of EQ/AEQ interrupt loss. Migration driver will
trigger an EQ/AEQ doorbell at the end of the migration.

This operation may cause double interruption of EQ/AEQ events.
To ensure that the EQ/AEQ interrupt processing function is normal.
The interrupt handling functionality of EQ/AEQ needs to be updated.
Used to handle repeated interrupts event.

Fixes: b0eed085903e ("hisi_acc_vfio_pci: Add support for VFIO live migration")
Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - alloc buffer to set and get xqc

If the temporarily applied memory is used to set or get the xqc
information, the driver releases the memory immediately after the
hardware mailbox operation time exceeds the driver waiting time.
However, the hardware does not cancel the operation, so the hardware
may write data to released memory.

Therefore, when the driver is bound to a device, the driver reserves
memory for the xqc configuration. The subsequent xqc configuration
uses the reserved memory to prevent hardware from accessing the
released memory.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - check function qp num before alg register

When the Kunpeng accelerator executes tasks such as encryption
and decryption have minimum requirements on the number of device
queues. If the number of queues does not meet the requirement,
the process initialization will fail. Therefore, the driver checks
the number of queues on the device before registering the algorithm.
If the number does not meet the requirements, the driver does not register
the algorithm to crypto subsystem, the device is still added to the
qm_list.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix the type value of aeq

The type of aeq has only 4bits in dw0 17 to 20bits, but 15bits(17 to
31bits) are read in function qm_aeq_thread(). The remaining 11bits(21
to 31bits) are reserved for aeq, but may not be 0. To avoid getting
incorrect value of type, other bits are cleared.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix PF queue parameter issue

If the queue isolation feature is enabled, the number of queues
supported by the device changes. When PF is enabled using the
current default number of queues, the default number of queues may
be greater than the number supported by the device. As a result,
the PF fails to be bound to the driver.

After modification, if queue isolation feature is enabled, when
the default queue parameter is greater than the number supported
by the device, the number of enabled queues will be changed to
the number supported by the device, so that the PF and driver
can be properly bound.

Fixes: 8bbecfb402f7 ("crypto: hisilicon/qm - add queue isolation support for Kunpeng930")
Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - increase device doorbell timeout

When both the accelerator device and SMMU are busy,
the processing time of the doorbell may be prolonged.
As a result, the doorbell may timeout, especially in the sva
scenario. Therefore, the doorbell timeout is increased.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - stop function and write data to memory

When the system is shut down, the process is killed, but the
accelerator device does not stop executing the tasks. If the
accelerator device still accesses the memory and writes back data
to the memory after the memory is reclaimed by the system,
an NFE error may occur. Therefore, before the system is shut
down, the driver needs to stop the device and write data back
to the memory.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - flush all work before driver removed

Before removing the driver, flush inter-function communication
work, and subsequent communication work is not processed.
This prevents communication threads from accessing released memory.

Fixes: ("crypto: hisilicon/qm - enable PF and VFs communication")
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm: Fix to enable build with RISC-V clang

With CONFIG_ACPI enabled for RISC-V, this driver gets enabled in
allmodconfig build. However, build fails with clang and below
error is seen.

drivers/crypto/hisilicon/qm.c:627:10: error: invalid output constraint '+Q' in asm
"+Q" (*((char __iomem *)fun_base))
^
This is expected error with clang due to the way it is designed.

To fix this issue, move arm64 assembly code under #if.

Link: https://github.com/ClangBuiltLinux/linux/issues/999
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
[sunilvl@ventanamicro.com: Moved tmp0 and tmp1 into the #if]
Signed-off-by: Sunil V L <sunilvl@ventanamicro.com>
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
Link: https://lore.kernel.org/r/20230515054928.2079268-4-sunilvl@ventanamicro.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>

crypto: hisilicon/qm - make struct bus_type * const

In the function, qm_get_qos_value(), a struct bus_type * is used, but it
really should be a const pointer as it is not modified anywhere in the
function, and the driver core function it is used in expects a constant
pointer.

Cc: Weili Qian <qianweili@huawei.com>
Cc: Zhou Wang <wangzhou1@hisilicon.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: linux-crypto@vger.kernel.org
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
Link: https://lore.kernel.org/r/20230313182918.1312597-29-gregkh@linuxfoundation.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

crypto: hisilicon/qm - remove unnecessary aer.h include

<linux/aer.h> is unused, so remove it.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Weili Qian <qianweili@huawei.com>
Cc: Zhou Wang <wangzhou1@hisilicon.com>
Acked-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

mm: replace vma->vm_flags direct modifications with modifier calls

Replace direct modifications to vma->vm_flags with calls to modifier
functions to be able to track flag changes and to keep vma locking
correctness.

[akpm@linux-foundation.org: fix drivers/misc/open-dice.c, per Hyeonggon Yoo]
Link: https://lkml.kernel.org/r/20230126193752.297968-5-surenb@google.com
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Mike Rapoport (IBM) <rppt@kernel.org>
Acked-by: Sebastian Reichel <sebastian.reichel@collabora.com>
Reviewed-by: Liam R. Howlett <Liam.Howlett@Oracle.com>
Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arjun Roy <arjunroy@google.com>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: David Rientjes <rientjes@google.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Greg Thelen <gthelen@google.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jann Horn <jannh@google.com>
Cc: Joel Fernandes <joelaf@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Laurent Dufour <ldufour@linux.ibm.com>
Cc: Lorenzo Stoakes <lstoakes@gmail.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Minchan Kim <minchan@google.com>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Peter Oskolkov <posk@google.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Punit Agrawal <punit.agrawal@bytedance.com>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Shakeel Butt <shakeelb@google.com>
Cc: Soheil Hassas Yeganeh <soheil@google.com>
Cc: Song Liu <songliubraving@fb.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

crypto: hisilicon/qm - define the device isolation strategy

Define the device isolation strategy by the device driver. The
user configures a hardware error threshold value by uacce interface.
If the number of hardware errors exceeds the value of setting error
threshold in one hour. The device will not be available in user space.
The VF device use the PF device isolation strategy. All the hardware
errors are processed by PF driver.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
Link: https://lore.kernel.org/r/20221119074817.12063-4-yekai13@huawei.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

crypto: hisilicon/qm - fix coding style issues

1. Remove extra blank lines.
2. Remove extra spaces.
3. Use spaces instead of tabs around '=' and '\',
to ensure consistent coding styles.
4. Macros should be capital letters, change 'QM_SQC_VFT_NUM_MASK_v2'
to 'QM_SQC_VFT_NUM_MASK_V2'.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - update comments to match function

The return values of some functions have been modified,
but the comments have not been modified together. The
comments must be updated to be consistent with the functions.

Also move comments over the codes instead of right place
to ensure consistent coding styles.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - change function names

The accelerator devices support multiple interrupts.
To better reflect purpose of each interrupt function,
change function name 'qm_irq' to 'qm_eq_irq' and 'do_qm_irq'
to 'do_qm_eq_irq'.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - use min() instead of min_t()

'act_q_num = min_t(int, act_q_num, max_qp_num)', the type
of 'act_q_num' and 'max_qp_num' are both 'u32', so
use min() instead of min_t().

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - remove some unused defines

1. Remove some macros define since it is not used.
2. Remove enum QM_HW_UNKNOWN since it is not used.
3. Remove unused member 'is_frozen' in 'hisi_qm' structure.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add device status check when start fails

In function 'hisi_qm_resume', if the device fails to be started,
directly returning error code will cause the device to be unavailable.
However, the failure may be caused by device error, which will be
reported to the driver, and driver can reset and restart device.
Therefore, check device status instead of returning error code
directly. Returns 0 if device error has occurred, otherwise returns
error code.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix 'QM_XEQ_DEPTH_CAP' mask value

'QM_XEQ_DEPTH_CAP' mask value is GENMASK(31, 0) instead of GENMASK(15, 0).
If the mask value is incorrect, will cause abnormal events cannot be
handled. So fix it.

Fixes: 129a9f340172 ("crypto: hisilicon/qm - get qp num and depth from hardware registers")
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - split a debugfs.c from qm

Considering that the qm feature and debugfs feature are independent.
The code related to debugfs is getting larger and larger. It should be
separate as a debugfs file. So move some debugfs code to new file from
qm file. The qm code logic is not modified. And maintainability is
enhanced.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - modify the process of regs dfx

The last register logic and different register logic are combined.
Use "u32" instead of 'int' in the regs function input parameter to
simplify some checks.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - delete redundant null assignment operations

There is no security data in the pointer. It is only a value transferred
as a structure. It makes no sense to zero a variable that is on the stack.
So not need to set the pointer to null.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - delete redundancy check

Because the permission on the VF debugfs file is "0444". So
the VF function checking is redundant in qos writing api.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add pci bdf number check

The pci bdf number check is added for qos written by using the pci api.
Directly get the devfn by pci_dev, so delete some redundant code.
And use the kstrtoul instead of sscanf to simplify code.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - increase the memory of local variables

Increase the buffer to prevent stack overflow by fuzz test. The maximum
length of the qos configuration buffer is 256 bytes. Currently, the value
of the 'val buffer' is only 32 bytes. The sscanf does not check the dest
memory length. So the 'val buffer' may stack overflow.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - re-enable communicate interrupt before notifying PF

After the device is reset, the VF needs to re-enable communication
interrupt before the VF sends restart complete message to the PF.
If the interrupt is re-enabled after the VF notifies the PF, the PF
may fail to send messages to the VF after receiving VF's restart
complete message.

Fixes: 760fe22cf5e9 ("crypto: hisilicon/qm - update reset flow")
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix incorrect parameters usage

In qm_get_xqc_depth(), parameters low_bits and high_bits save
the values of the corresponding bits. However, the values saved by the
two parameters are opposite. As a result, the values returned to the
callers are incorrect.

Fixes: 129a9f340172 ("crypto: hisilicon/qm - get qp num and depth from hardware registers")
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - drop unnecessary IS_ENABLE(CONFIG_NUMA) check

dev_to_node() can handle the case when CONFIG_NUMA is not set, so the
check of CONFIG_NUMA is redundant and can be removed.

Signed-off-by: Yicong Yang <yangyicong@hisilicon.com>
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix the qos value initialization

The default qos value is not initialized when sriov is repeatedly enabled
and disabled. So add the vf qos value initialized in the sriov enable
process.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - support get algs by the capability register

The value of qm algorithm can change dynamically according to the
value of the capability register.

Add xxx_set_qm_algs() function to obtain the algs that the
hardware device supported from the capability register and set
them into usr mode attribute files.

Signed-off-by: Zhiqi Song <songzhiqi1@huawei.com>
Signed-off-by: Wenkai Lin <linwenkai6@hisilicon.com>
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - support get device irq information from hardware registers

Support get device irq information from hardware registers
instead of fixed macros.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - get error type from hardware registers

Hardware V3 and later versions support get error type from
registers. To be compatible with later hardware versions,
get error type from registers instead of fixed marco.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add UACCE_CMD_QM_SET_QP_INFO support

To be compatible with accelerator devices of different
versions, 'UACCE_CMD_QM_SET_QP_INFO' ioctl is added to obtain
queue information in userspace, including queue depth and buffer
description size.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - get qp num and depth from hardware registers

Hardware V3 and later versions can obtain qp num and depth supported
by the hardware from registers. To be compatible with later hardware
versions, get qp num and depth from registers instead of fixed marcos.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - get hardware features from hardware registers

Before hardware V3, hardwares do not provide the feature registers,
driver resolves hardware differences based on the hardware version.
As a result, the driver does not support the new hardware.

Hardware V3 and later versions support to obtain hardware features,
such as power-gating management and doorbell isolation, through
the hardware registers. To be compatible with later hardware versions,
the features of the current device is obtained by reading the
hardware registers instead of the hardware version.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - return failure if vfs_num exceeds total VFs

The accelerator drivers supports users to enable VFs through the
module parameter 'vfs_num'. If the number of VFs to be enabled
exceeds the total VFs, all VFs are enabled. Change it to the same
as enabling VF through the 'sriov_numvfs' file. Returns -ERANGE
if the number of VFs to be enabled exceeds total VFs.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix missing put dfx access

In function qm_cmd_write(), if function returns from
branch 'atomic_read(&qm->status.flags) == QM_STOP',
the got dfx access is forgotten to put.

Fixes: 607c191b371d ("crypto: hisilicon - support runtime PM for accelerator device")
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - check mailbox operation result

After the mailbox operation is complete, the result may be unsuccessful.
It needs to check the status bits of the mailbox register,
if it fails, -EIO is returned.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - no judgment in the back process

Judgment should not be added in the back process. So clean it.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - remove unneeded hardware cache write back

Data in the hardware cache needs to be written back to the memory
before the queue memory is released. Currently, the queue memory is
applied for when the driver is loaded and released when the driver is
removed. Therefore, the hardware cache does not need to be written back
when process puts queue.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Yang Shen <shenyang39@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - remove unneeded data storage

The dump_show() is used to output hardware information for error locating.
It is not need to apply for memory to temporarily store the converted data.
It can directly output the data. Therefore, remove some unnecessary code.

Signed-off-by: Junchong Pan <panjunchong@hisilicon.com>
Signed-off-by: Yang Shen <shenyang39@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix missing destroy qp_idr

In the function hisi_qm_memory_init(), if resource alloc fails after
idr_init, the initialized qp_idr needs to be destroyed.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Yang Shen <shenyang39@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Remove pci_aer_clear_nonfatal_status() call

Calls to pci_cleanup_aer_uncorrect_error_status() have already been
removed after commit 62b36c3ea664 ("PCI/AER: Remove
pci_cleanup_aer_uncorrect_error_status() calls"). But in commit
6c6dd5802c2d ("crypto: hisilicon/qm - add controller reset interface")
pci_aer_clear_nonfatal_status() was used again, so remove it in
this patch.

note: pci_cleanup_aer_uncorrect_error_status() was renamed to
pci_aer_clear_nonfatal_status() in commit 894020fdd88c
("PCI/AER: Rationalize error status register clearing")

Signed-off-by: Zhuo Chen <chenzhuo.1@bytedance.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - modify event irq processing

When the driver receives an event interrupt, the driver will enable
the event interrupt after handling all completed tasks on the function,
tasks on the function are parsed through only one thread. If the task's
user callback takes time, other tasks on the function will be blocked.

Therefore, the event irq processing is modified as follows:
1. Obtain the ID of the queue that completes the task.
2. Enable event interrupt.
3. Parse the completed tasks in the queue and call the user callback.
Enabling event interrupt in advance can quickly report pending event
interrupts and process tasks in multiple threads.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - move alloc qm->wq to qm.c

Before stopping the function, the driver needs to flush all the remaining
work about event irq. Therefore, accelerator drivers use a private
workqueue(qm->wq) to handle event irq instead of the system workqueue.
This patch moves alloc workqueue from sec_main.c and zip_main.c to qm.c.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add functions for releasing resources

The resources allocated by hisi_qm_memory_init() are released by
hisi_qm_uninit(). Add hisi_qm_memory_uninit() to release resources,
no functional change.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - remove hisi_qm_get_free_qp_num()

hisi_qm_get_free_qp_num() is to get the free queue number on the function.
It is a simple function and is only called by
hisi_qm_get_available_instances().

This patch modifies to get the free queue directly in
hisi_qm_get_available_instances(), and remove hisi_qm_get_free_qp_num().

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - replace hisi_qm_release_qp() with hisi_qm_free_qps()

hisi_qm_free_qps() can release multiple queues in one call, and it is
already exported. So, replace hisi_qm_release_qp() with hisi_qm_free_qps()
in zip_crypto.c, and do not export hisi_qm_release_qp() outside qm.c.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - set function with static

These functions 'hisi_qm_create_qp' and 'hisi_qm_set_vft' are not
used outside qm.c, so they are marked as static.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add last word dumping for ACC

Add last word dumping function during acc engines controller reset.
The last words are reported to the printed information during the
reset. The dmesg information included qm debugging registers and
engine debugging registers. It can help to improve debugging
capability.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add register checking for ACC

Add register detection function to accelerator. Provided a tool that
user can checking differential register through Debugfs.
e.g.
cd /sys/kernel/debug/hisi_zip/<bdf>/zip_dfx
cat diff_regs

Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - optimize the barrier operation

A 'dma_wmb' barrier is enough to guarantee previous writes
before accessing by acc device in the outer shareable domain.

A 'smp_wmb' barrier is enough to guarantee previous writes
before accessing by other cpus in the inner shareble domain.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm: Set the VF QM state register

We use VF QM state register to record the status of the QM configuration
state. This will be used in the ACC migration driver to determine whether
we can safely save and restore the QM data.

Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Link: https://lore.kernel.org/r/20220308184902.2242-8-shameerali.kolothum.thodi@huawei.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

crypto: hisilicon/qm: Move few definitions to common header

Move Doorbell and Mailbox definitions to common header file.
Also export QM mailbox functions.

This will be useful when we introduce VFIO PCI HiSilicon ACC live
migration driver.

Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Link: https://lore.kernel.org/r/20220308184902.2242-3-shameerali.kolothum.thodi@huawei.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

crypto: hisilicon/qm: Move the QM header to include/linux

Since we are going to introduce VFIO PCI HiSilicon ACC driver for live
migration in subsequent patches, move the ACC QM header file to a
common include dir.

Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
Acked-by: Longfang Liu <liulongfang@huawei.com>
Acked-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Link: https://lore.kernel.org/r/20220308184902.2242-2-shameerali.kolothum.thodi@huawei.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

crypto: hisilicon/qm - fix memset during queues clearing

Due to that extra page addr is used as a qp error flag when the device
resetting. So it not should to clear this qp flag in userspace.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - cleanup warning in qm_vf_read_qos

The kernel test rebot report this warning: Uninitialized variable: ret.
The code flow may return value of ret directly. This value is an
uninitialized variable, here is fix it.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - cleanup warning in qm_get_qos_value()

Building with clang static analysis returns this warning:

qm.c:4382:11: warning: The left operand of '==' is a garbage value
if (*val == 0 || *val > QM_QOS_MAX_VAL || ret) {
~~~~ ^

The call to qm_qos_value_init() can return an error without setting
*val. So check ret before checking *val.

Fixes: 72b010dc33b9 ("crypto: hisilicon/qm - supports writing QoS int the host")
Signed-off-by: Tom Rix <trix@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - disable qm clock-gating

For Kunpeng930, if qm clock-gating is enabled, rate limiter
will be inaccurate. Therefore, disable clock-gating before doing task.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - disable queue when 'CQ' error

If the hardware reports the 'CQ' overflow or 'CQE' error by the abnormal
interrupt, disable the queue and stop tasks send to hardware.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - reset function if event queue overflows

If the hardware reports the event queue overflow by the abnormal interrupt,
the driver needs to reset the function and re-enable the event queue
interrupt and abnormal interrupt.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - use request_threaded_irq instead

The abnormal interrupt method needs to be changed, and the changed method
needs to be locked in order to maintain atomicity. Therefore,
replace request_irq() with request_threaded_irq().

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - modify the handling method after abnormal interruption

After processing an interrupt event and the interrupt function is
enabled by writing the QM_DOORBELL_CMD_AEQ register, the hardware
may generate new interrupt events due to processing other user's task
when the subsequent interrupt events have not been processed. The new
interrupt event will disrupt the current normal processing flow and
cause other problems.

Therefore, the operation of writing the QM_DOORBELL_CMD_AEQ doorbell
register needs to be placed after all interrupt events processing
are completed.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - code movement

This patch does not change any code, just code movement. Preparing for
next patch.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - remove unnecessary device memory reset

The internal memory of the device needs to be reset only when
the device is globally initialized. Other scenarios, such as
function reset, do not need to perform reset.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix deadlock for remove driver

When remove the driver and executing the task occur at the same time,
the following deadlock will be triggered:

Chain exists of:
sva_lock --> uacce_mutex --> &qm->qps_lock
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&qm->qps_lock);
lock(uacce_mutex);
lock(&qm->qps_lock);
lock(sva_lock);

And the lock 'qps_lock' is used to protect qp. Therefore, it's reasonable
cycle is to continue until the qp memory is released. So move the release
lock infront of 'uacce_remove'.

Signed-off-by: Yang Shen <shenyang39@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix incorrect return value of hisi_qm_resume()

When hisi_qm_resume() returns 0, it indicates that the device has started
successfully. If the device fails to start, hisi_qm_resume() needs to
return the actual error code to the caller instead of 0.

Fixes: d7ea53395b72 ("crypto: hisilicon - add runtime PM ops")
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - simplified the calculation of qos shaper parameters

Some optimize for the calculation of qos shaper parameters.
and modify the comments.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - some optimizations of ths qos write process

1. Optimize overly long functions.
2. Fix the format symbol does not match the actual type.
3. Use the PCI_DEVFN to get the function id.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - modify the value of qos initialization

The value of qos should be reset after flr resetting or device
resetting. So set the max of qos value for every function. Then
update the value of qos when user writing the alg_qos.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - use dev_driver_string() instead of pci_dev->driver->name

Replace dev->driver_name() by dev_driver_string() for the corresponding
struct device. This is a step toward removing pci_dev->driver.

[bhelgaas: split to separate patch]
Link: https://lore.kernel.org/r/20211004125935.2300113-8-u.kleine-koenig@pengutronix.de
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

crypto: hisilicon/qm - modify the uacce mode check

As qm should register to uacce in UACCE_DEV_SVA mode,
this patch modifies to checks uacce mode before doing uacce
registration.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - support the userspace task resetting

Allocate an extra memory page for qp in the qp memory initialization.
Set a qp error flag in the extra page addr when device resetting.
This error flag can be seen in the userspace. This flag can helps
users to stop tasks when device resetting. After resetting, this error
flag will be reset when this qp is created again. So app should release
the old qp and request a new one, and do the task on the new queue
again.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix the uacce mmap failed

Allocate an extra memory page for qp. This extra page is
used to set the device or qp status. But this page not
be used currently. Meanwhile it leads to dus size not equal
to mmap size as using uacce sva mode, and cause the app task
broken.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Fix sscanf format signedness

The function qm_qos_value_init expects an unsigned integer but
is incorrectly supplying a signed format to sscanf. This patch
fixes it.

Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - check _PS0 and _PR0 method

To support runtime PM, use the function 'pci_set_power_state' to change
the power state. Therefore, method _PS0 or _PR0 needs to be filled by
platform. So check whether the method is supported, if not, print a
prompt information.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - change parameter passing of debugfs function

To avoid repeatedly obtaining 'qm' from 'filp', parameter passing of
debugfs function directly use 'qm' instead of 'filp'.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - support runtime PM for accelerator device

Add runtime PM support for Kunpeng930 accelerator device.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - add runtime PM ops

Accelerator devices support runtime PM to reduce power consumption.
This patch adds the runtime PM suspend/resume callbacks to the
accelerator devices.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - using 'debugfs_create_file' instead of 'debugfs_create_regset32'

The accelerator devices support runtime PM, when device is in suspended, an
exception will occur if reading registers. Therefore, this patch uses
'debugfs_create_file' instead of 'debugfs_create_regset32' to create
debugfs file, and then the driver can get the device status before
reading the register.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - implement for querying hardware tasks status.

This patch adds a function hisi_qm_is_q_updated to
check if the task is ready in hardware queue when
user polls an UACCE queue.This prevents users from
repeatedly querying whether the accelerator has
completed tasks, which wastes CPU resources.

Signed-off-by: Wenkai Lin <linwenkai6@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - supports to inquiry each function's QoS

1. The ACC driver supports to inquiry each function's QoS in the Host
and VM. The driver supports reading QoS by the device debug SysFS
attribute file "alg_qos", like "cat alg_qos".
2. Modify the communication process between pf and vf as needed.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add pf ping single vf function

According to the function communication, add pf ping single
vf function to be used in the vf read QoS.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - merges the work initialization process into a single function

Merges the work initialization process into a single function from
qm initialization.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add the "alg_qos" file node

1. Just move the code as needed.
2. Add the "alg_qos" file node in the qm debug sysfs.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - supports writing QoS int the host

Based on the Token bucket algorithm. The HAC driver supports to configure
each function's QoS in the host. The driver supports writing QoS by the
debugfs node that named "alg_qos". The qos value is 1~1000.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - update reset flow

This patch updates the reset flow based on PF/VF communications. VFs
will be stopped after receiving reset message from PF, and wait for
reset finish to restart VFs.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add callback to support communication

This patch adds 'ping_all_vfs' callback that supports pf send message to
all vfs and 'ping_pf' callback that supports vf send message to pf. After
receiving the interrupt, the communication destination gets the message
by sending mailbox.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - enable PF and VFs communication

Kunpeng930 hardware supports the communication between PF and VFs.

This patch enables communication between PF and VFs by writing hardware
registers, and requests an irq for communication.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - adjust reset interface

Kunpeng930 hardware supports PF/VF communications. When the device is
reset, PF can send message to VF to stop function and restart function.

This patch adjusts the reset interface to support sending message through
PF/VF communication.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - switch to memdup_user_nul()

Use memdup_user_nul() helper instead of open-coding to
simplify the code.

v1-->v2:
fixed patch title error
v2-->v3:
return the actual error

Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Zou Wei <zou_wei@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - support address prefetching

Kunpeng930 hardware supports address prefetching to improve performance
before doing tasks in SVA scenario.

This patch enables this function in device initialization by writing
hardware registers. In the process of reset, address prefetching is
disabled to avoid the failure of interaction between accelerator device
and SMMU.

Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add MSI detection steps on Kunpeng930

Compared with Kunpeng920, Kunpeng930 adds MSI configuration steps to wait
for the interrupt to be emptied. In order to be compatible with the
kunpeng920 driver, 'set_msi' callback is added in 'hisi_qm_hw_ops' to
configure hardware register. Call 'set_msi' to disable or enable MSI
during reset.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix the process of VF's list adding

If Kunpeng 920 enabled the sva mode, the "qm alg register" process will
return directly. So the list of VF wasn't added to QM list.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add dfx log if not use hardware crypto algs

Add print information necessary if not use hardware crypto algs.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - enable to close master ooo when NFE occurs

Kunpeng930 could be able to close master ooo when NFE occurs, which will
disable memory accessing from device and execute tasks. This ensures that
errors do not spread.

This patch enables the hardware to close master ooo when an error occurs
by writing hardware registers, and ensures that the driver will not drain
qp because the hardware will empty the tasks automatically.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - adjust order of device error configuration

In order to avoid reporting an exception but the error type is not
configured, the driver needs to configure the error type first, and then
enable the error interrupt. Before executing the task, hardware error
initialization is needed so that the hardware can detect the error in time.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - modify 'QM_RESETTING' clearing error

Before device reset, the driver sets 'QM_RESETTING' flag, but after reset,
the wrong flag is cleared. This patch modifies the problem of inconsistent
flags.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - initialize the device before doing tasks

The device needs to be initialized first, and then restart the queue to
execute tasks after PF reset.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - enable new error types for QM

QM adds 'qm_mailbox_timeout' and 'qm_flr_timeout' hardware error types on
Kunpeng930. This patch enables the new error types and configures the error
types as NFE.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - dynamic configuration 'err_info'

'err_info' does not support dynamic configuration since it is const type.
Therefore, in order to support new error type later, 'err_info' is changed
to dynamic configuration.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add stop queue by hardware

Kunpeng930 could be able to stop queue by writing hardware registers,
which will trigger tasks in device to be flushed out.

In order to be compatible with the kunpeng920 driver, add 'qm_hw_ops_v3' to
adapt Kunpeng930. And 'stop_qp' callback is added in 'qm_hw_ops_v3' to
write hardware registers. Call 'stop_qp' to drain the tasks in device
before releasing queue.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - delete redundant code

The "qdma" is a structure variable instead of actual data. This
structure doesn't need to be zerod, The memset is useless and redundant.
So delete it.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add queue isolation support for Kunpeng930

Kunpeng930 supports doorbell isolation to ensure that each queue
has an independent doorbell address space.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - set the number of queues for function

Kunpeng930 supports queue doorbell isolation.
When doorbell isolation is enabled, it supports to obtain the
maximum number of queues of one function from hardware register.
Otherwise, the 'max_qp_num' is the total number of queues.

When assigning queues to VF, it is necessary to ensure that the number
of VF queues does not exceed 'max_qp_num'.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - move 'CURRENT_QM' code to qm.c

Since the code related to 'CURRENT_QM' debugfs is exactly same in
sec/hpre/zip driver, move 'CURRENT_QM' to qm.c to reduce duplicate code.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - set the total number of queues

Move the configuration of the total number of queues 'ctrl_qp_num'
from sec2/hpre/zip to qm.c. And get the total number of queues
from the hardware register for Kunpeng930.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - fix the check on dma address

System may be able to get physical address of zero if not reserved by
firmware.

The dma address obtained by 'dma_alloc_coherent' is valid, since already
checking cpu va before, so do not check again.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - add version adapt to new algorithms

A new generation of accelerator Kunpeng930 has appeared, and the
corresponding driver needs to be updated to support some new
algorithms of Kunpeng930. To be compatible with Kunpeng920, we
add parameter 'struct hisi_qm *qm' to sec_algs_(un)register to
identify the chip's version.

Signed-off-by: Meng Yu <yumeng18@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Reviewed-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix printing format issue

This patch fixes inconsistent of printing format with argument type.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - do not reset hardware when CE happens

There is no need to reset hardware when Corrected Error(CE) happens.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - update irqflag

There is no need to share IRQ among several devices,
and set 'irqflag' as 0.

Signed-off-by: Sihang Chen <chensihang1@hisilicon.com>
Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix the value of 'QM_SQC_VFT_BASE_MASK_V2'

Since the size of base number is 16 bits, update the value of
'QM_SQC_VFT_BASE_MASK_V2' as 'GENMASK(15, 0)'.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix request missing error

Add 'qp_stop_fail_cb' to ensure it is called as device is resetting.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - removing driver after reset

Add waiting logic for resetting as removing driver,
otherwise call trace will occur due to releasing resource.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - PASID fixed on Kunpeng 930

Enable PASID by setting 'sqc' and 'cqc' pasid bits
per queue in Kunpeng 930.

For Kunpeng 920, PASID is effective for all queues once set
in SVA scenarios.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix use of 'dma_map_single'

Calling 'dma_map_single' after the data is written to
ensure that the cpu cache and dma cache are consistent.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - add ZIP device using mode parameter

Add 'uacce_mode' parameter for ZIP, which can be set as 0(default) or 1.
'0' means ZIP is only registered to kernel crypto, and '1' means it's
registered to both kernel crypto and UACCE.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - SVA bugfixed on Kunpeng920

Kunpeng920 SEC/HPRE/ZIP cannot support running user space SVA and kernel
Crypto at the same time. Therefore, the algorithms should not be registered
to Crypto as user space SVA is enabled.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - split 'hisi_qm_init' into smaller pieces

'hisi_qm_init' initializes configuration of QM.
To improve code readability, split it into two pieces.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - split 'qm_eq_ctx_cfg' into smaller pieces

'qm_eq_ctx_cfg' initializes configuration of EQ and AEQ,
split it into two pieces to improve code readability.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - split 'qm_qp_ctx_cfg' into smaller pieces

'qm_qp_ctx_cfg' initializes configuration of SQ and CQ,
split it into two pieces to improve code readability.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - replace 'sprintf' with 'scnprintf'

Replace 'sprintf' with 'scnprintf' to avoid overrun.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - modify return type of 'qm_set_sqctype'

Since 'qm_set_sqctype' always returns 0, change it as 'void'.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - modify the return type of debugfs interface

Since 'qm_create_debugfs_file' always returns 0, change it as 'void'.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - modify the return type of function

The returns of 'qm_get_hw_error_status' and 'qm_get_dev_err_status'
are values from the hardware registers, which should not be defined
as 'int', so update as 'u32'.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - numbers are replaced by macros

Some numbers are replaced by macros to avoid incomprehension.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Fix doc warnings in sgl.c and qm.c

Fix following warnings caused by mismatch between
function parameters and function comments.

drivers/crypto/hisilicon/sgl.c:256: warning: Excess function parameter 'hw_sgl_dma' description in 'hisi_acc_sg_buf_unmap'
drivers/crypto/hisilicon/sgl.c:256: warning: Excess function parameter 'pool' description in 'hisi_acc_sg_buf_unmap'
drivers/crypto/hisilicon/qm.c:1849: warning: Function parameter or member 'qp' not described in 'qm_drain_qp'
drivers/crypto/hisilicon/qm.c:2420: warning: Function parameter or member 'qm' not described in 'hisi_qm_set_vft'
drivers/crypto/hisilicon/qm.c:2420: warning: Function parameter or member 'fun_num' not described in 'hisi_qm_set_vft'
drivers/crypto/hisilicon/qm.c:2420: warning: Function parameter or member 'base' not described in 'hisi_qm_set_vft'
drivers/crypto/hisilicon/qm.c:2420: warning: Function parameter or member 'number' not described in 'hisi_qm_set_vft'
drivers/crypto/hisilicon/qm.c:2620: warning: Function parameter or member 'qm' not described in 'qm_clear_queues'

Signed-off-by: Shiju Jose <shiju.jose@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - remove the update of flags

'qm_init_qp_status' is just a help function to initiate some 'QP' status.
'QP' status should be updated separately.
This patch removes the updating flags in 'QP' status.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - Convert to DEFINE_SHOW_ATTRIBUTE

Use DEFINE_SHOW_ATTRIBUTE macro to simplify the code.

Signed-off-by: Qinglang Miao <miaoqinglang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix the process of register algorithms to crypto

When the devices are removed or not existing, the corresponding algorithms
which are registered by 'hisi-zip' driver can't be used.

Move 'hisi_zip_register_to_crypto' from 'hisi_zip_init' to
'hisi_zip_probe'. The algorithms will be registered to crypto only when
there is device bind on the driver. And when the devices are removed,
the algorithms will be unregistered.

In the previous process, the function 'xxx_register_to_crypto' need a lock
and a static variable to judge if the registration is the first time.
Move this action into the function 'hisi_qm_alg_register'. Each device
will call 'hisi_qm_alg_register' to add itself to qm list in probe process
and registering algs when the qm list is empty.

Signed-off-by: Yang Shen <shenyang39@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix the call trace when unbind device

Call trace will appear in the Hisilicon crypto driver unbinding or
disabling SRIOV during task running with TFMs on the corresponding
function.
The log looks like this:
[ 293.908078] Call trace:
[ 293.908080] __queue_work+0x494/0x548
[ 293.908081] queue_work_on+0x84/0xd8
[ 293.908092] qm_irq+0x4c/0xd0 [hisi_qm]
[ 293.908096] __handle_irq_event_percpu+0x74/0x2a0
[ 293.908098] handle_irq_event_percpu+0x40/0x98
[ 293.908099] handle_irq_event+0x4c/0x80
[ 293.908101] handle_fasteoi_irq+0xb0/0x170
[ 293.908102] generic_handle_irq+0x3c/0x58
[ 293.908103] __handle_domain_irq+0x68/0xc0
[ 293.908104] gic_handle_irq+0xb4/0x298
[ 293.908105] el1_irq+0xcc/0x180
[ 293.908107] arch_cpu_idle+0x38/0x228
[ 293.908110] default_idle_call+0x20/0x40
[ 293.908113] do_idle+0x1cc/0x2b8
[ 293.908114] cpu_startup_entry+0x2c/0x30
[ 293.908115] rest_init+0xdc/0xe8
[ 293.908117] arch_call_rest_init+0x14/0x1c
[ 293.908117] start_kernel+0x490/0x4c4

This patch adds a waiting logic as user doing the above two operations
to avoid panic. The two operations will hold on in the driver
remove function until the tasks release all their relative TFMs.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Yang Shen <shenyang39@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - register callback function to 'pci_driver.shutdown'

Since the drivers such as HPRE/SEC/ZIP do not implement
'pci_driver.shutdow', a RAS will be triggered at OS rebooting or shutting
down as the hardware device is processing request.
The log looks like this:
NOTICE: [NimbusSecNodeType1]:[2372L]This is sec, Base = 0x141800000
NOTICE: [NimbusSecHandle]:[2319L] SecIntSt = 0x3
NOTICE: [NimbusSecHandle]:[2320L] SecQmIntStatus = 0x2
NOTICE: [PrintSecurityType]:[344L] SecurityType is RECOVERABLE!

This patch offers a new API in qm to shutdown devices, and add shutdown
callbacks in ACC driver based on this new API.

So the running devices will be stopped when the OS reboot or shutdown.

Signed-off-by: Yang Shen <shenyang39@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix VF not available after PF FLR

When PF FLR, the hardware will actively trigger the VF FLR. Configuration
space of VF needs to be saved and restored to ensure that it is available
after the PF FLR.

Fixes: 7ce396fa12a9("crypto: hisilicon - add FLR support")
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Signed-off-by: Yang Shen <shenyang39@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix no stop reason when use 'hisi_qm_stop'

Now, there are three reasons of stopping: 'NORMAL', 'SOFT_RESET' and 'FLR'.
In order to keep this, explicitly pass the stop reason as an input
parameter of 'hisi_qm_stop' function.

Fixes: b67202e8ed30("crypto: hisilicon/qm - add state machine for QM")
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Signed-off-by: Yang Shen <shenyang39@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix event queue depth to 2048

Increasing depth of 'event queue' from 1024 to 2048, which equals to twice
depth of 'completion queue'. It will fix the easily happened 'event queue
overflow' as using 1024 queue depth for 'event queue'.

Fixes: 263c9959c937("crypto: hisilicon - add queue management driver...")
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Signed-off-by: Yang Shen <shenyang39@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix judgement of queue is full

The queue depth is 1024, so the condition for judging the queue full
should be 1023, otherwise the hardware cannot judge whether the queue
is empty or full.

Fixes: 263c9959c937("crypto: hisilicon - add queue management driver...")
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Yang Shen <shenyang39@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix print frequence in hisi_qp_send

Requests will be sent continuously as resetting, which will cause 'printk'
flooding. Using 'dev_info_ratelimited' can solve this problem well.

Fixes: b67202e8ed30("crypto: hisilicon/qm - add state machine for QM")
Signed-off-by: Yang Shen <shenyang39@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - clear used reference count when start qp

The used reference count is used for counting the number of 'sqe' which
is under processing. This reference count should be cleared as starting
'qp', otherwise the 'used' will be messy when allocating this 'qp' again.

Fixes: 5308f6600a39("crypto: hisilicon - QM memory management...")
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Signed-off-by: Yang Shen <shenyang39@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix wrong release after using strsep

Save the string address before pass to strsep, release it at end.
Because strsep will update the string address to point after the
token.

Fixes: c31dc9fe165d("crypto: hisilicon/qm - add DebugFS for xQC and...")
Signed-off-by: Sihang Chen <chensihang1@hisilicon.com>
Signed-off-by: Yang Shen <shenyang39@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - Change type of pasid to u32

PASID is defined as "int" although it's a 20-bit value and shouldn't be
negative int. To be consistent with PASID type in iommu, define PASID
as "u32".

Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - fix strncpy warning with strscpy

Use strscpy to fix the warning
warning: 'strncpy' specified bound 64 equals destination size

Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - allow smaller reads in debugfs

Originally this code rejected any read less than 256 bytes. There
is no need for this artificial limit. We should just use the normal
helper functions to read a string from the kernel.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - fix driver compatibility issue with different versions of devices

In order to be compatible with devices of different versions, V1 in the
accelerator driver is now isolated, and other versions are the previous
V2 processing flow.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - change debugfs file name from qm_regs to regs

The debugfs qm_regs file is already in the qm directory, so no qm_ prefix
is required.

Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add DebugFS for xQC and xQE dump

Add dump information of SQC/CQC/EQC/AEQC/SQE/CQE/EQE/AEQE.

Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add debugfs to the QM state machine

The QM driver uses debugfs to provides the current state of
the QM state machine

Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add debugfs for QM

Add DebugFS method to get the information of IRQ/Requests/QP .etc of QM
for HPRE/ZIP/SEC drivers.

Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - add device error report through abnormal irq

By configuring the device error in firmware to report through abnormal
interruption, process all NFE errors in irq handler.

Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - remove codes of directly report device errors through MSI

The hardware device can be configured to report directly through MSI, but
this method will not go through RAS, configure all hardware errors that
should be processed by driver to NFE.

Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - QM memory management optimization

Put all the code for the memory allocation into the QM initialization
process. Before, The qp memory was allocated when the qp was created,
and released when the qp was released, It is now changed to allocate
all the qp memory once.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - unify initial value assignment into QM

Some initial value assignment of struct hisi_qm could put into QM.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - remove use_dma_api related codes

The codes related use_dma_api is useless which should be removed.

Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - add FLR support

Add callback reset_prepare and reset_done in QM, The callback
reset_prepare will uninit device error configuration and stop
the QM, the callback reset_done will init the device error
configuration and restart the QM.

Uninit the error configuration will disable device block master OOO
when Multi-bit ECC error occurs to avoid the request of FLR will not
return.

Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add state machine for QM

Add specific states for qm and qp, every state change under critical region
to prevent from race condition. Meanwhile, qp state change will also depend
on qm state.

Due to the introduction of these states, it is necessary to pay attention
to the calls of public logic, such as concurrent scenarios resetting and
releasing queue will call hisi_qm_stop, which needs to add additional
status to distinguish and process.

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - Make qm_controller_reset() static

Fix the following sparse warning:

drivers/crypto/hisilicon/qm.c:3079:5: warning: symbol 'qm_controller_reset'
was not declared. Should it be static?

Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Zou Wei <zou_wei@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - stop qp by judging sq and cq tail

It is not working well to determine whether the queue is empty based on
whether the used count is 0. It is more stable to get if the queue is
stopping by checking if the tail pointer of the send queue and the
completion queue are equal.

Signed-off-by: Yang Shen <shenyang39@huawei.com>
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - add controller reset interface

Add the main implementation of the controller reset interface, which is
roughly divided into three parts, stop, reset, and reinitialization.

Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - unify SR-IOV related codes into QM

Clean the duplicate SR-IOV related codes, put all into qm.c.

Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - Put device finding logic into QM

Use struct hisi_qm to maintain device list. Meanwhile, add two external
interface into qm, merge find proper device and create qp into QP alloc
logic, merge release qps into QP free logic.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Use one workqueue per qm instead of per qp

Since SEC need not so many workqueues as our test, we just use
one workqueue created by the device driver of QM if necessary,
which will also reduce CPU waste without any throughput decreasing.

Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Signed-off-by: Zaibo Xu <xuzaibo@huawei.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - register zip engine to uacce

Register qm to uacce framework for user crypto driver

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Fix duplicate print when qm occur multiple errors

If all possible errors occurs at the same time, the error_status will be
all 1s. The doorbell timeout error and FIFO overflow error will be print
in each cycle, which should be print just once.

Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Unify error detect process into qm

In error detect process, a lot of duplicate code can put into qm. We add
two callback(get_dev_hw_err_status and log_dev_hw_err) into struct
hisi_qm_err_ini to handle device error detect, meanwhile the qm error
detect not changed.

Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Signed-off-by: Zaibo Xu <xuzaibo@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Unify hardware error init/uninit into QM

The initialization and uninitialization of zip/hpre/sec/qm hardware error
is processed in respective drivers, which could be unified into qm.c. We
add struct hisi_qm_err_ini into struct hisi_qm, which involve all error
handlers of device and assignment should be done in driver probe.

Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Signed-off-by: Zaibo Xu <xuzaibo@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - no need to check return value of debugfs_create functions

When calling debugfs functions, there is no need to ever check the
return value. The function can work or not, but the code logic should
never do something different based on this.

Cc: Zhou Wang <wangzhou1@hisilicon.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: linux-crypto@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon: no need to check return value of debugfs_create functions

When calling debugfs functions, there is no need to ever check the
return value. The function can work or not, but the code logic should
never do something different based on this.

Cc: Zhou Wang <wangzhou1@hisilicon.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: linux-crypto@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Link: https://lore.kernel.org/r/20191107085200.GB1274176@kroah.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

crypto: hisilicon - fix to return sub-optimal device when best device has no qps

Currently find_zip_device() finds zip device which has the min NUMA
distance with current CPU.

This patch modifies find_zip_device to return sub-optimal device when best
device has no qps. This patch sorts all devices by NUMA distance, then
finds the best zip device which has free qp.

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - fix endianness verification problem of QM

This patch fixes following sparse warning:

qm.c:345:33: warning: cast removes address space '<asn:2>' of expression
qm.c:359:20: warning: incorrect type in assignment (different base types)
qm.c:359:20: expected restricted __le16 [usertype] w0
qm.c:359:20: got int
qm.c:362:27: warning: incorrect type in assignment (different base types)
qm.c:362:27: expected restricted __le16 [usertype] queue_num
qm.c:362:27: got unsigned short [usertype] queue
qm.c:363:24: warning: incorrect type in assignment (different base types)
qm.c:363:24: expected restricted __le32 [usertype] base_l
qm.c:363:24: got unsigned int [usertype]
qm.c:364:24: warning: incorrect type in assignment (different base types)
qm.c:364:24: expected restricted __le32 [usertype] base_h
qm.c:364:24: got unsigned int [usertype]
qm.c:451:22: warning: restricted __le32 degrades to integer
qm.c:471:24: warning: restricted __le16 degrades to integer
......
qm.c:1617:19: warning: incorrect type in assignment (different base types)
qm.c:1617:19: expected restricted __le32 [usertype] dw6
qm.c:1617:19: got int
qm.c:1891:24: warning: incorrect type in return expression (different base types)
qm.c:1891:24: expected int
qm.c:1891:24: got restricted pci_ers_result_t
qm.c:1894:40: warning: incorrect type in return expression (different base types)
qm.c:1894:40: expected int
qm.c:1894:40: got restricted pci_ers_result_t

Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - tiny fix about QM/ZIP error callback print

Tiny fix to make QM/ZIP error callback print clear and right. If one version
hardware does not support error handling, we directly print this.

And QM is embedded in ZIP, we can use ZIP print only, so remove unnecessary
QM print.

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - allow compile-testing on x86

To avoid missing arm64 specific warnings that get introduced
in this driver, allow compile-testing on all 64-bit architectures.

The only actual arm64 specific code in this driver is an open-
coded 128 bit MMIO write. On non-arm64 the same can be done
using memcpy_toio. What I also noticed is that the mmio store
(either one) is not endian-safe, this will only work on little-
endian configurations, so I also add a Kconfig dependency on
that, regardless of the architecture.
Finally, a depenndecy on CONFIG_64BIT is needed because of the
writeq().

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Fix warning on printing %p with dma_addr_t

This patch fixes a printk format warning by replacing %p with %#llx
for dma_addr_t.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - add missing single_release

Fix to add missing single_release in qm_regs_fops.

Fixes: 263c9959c937 ("crypto: hisilicon - add queue management driver for HiSilicon QM module")
Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - fix kbuild warnings

Fix to use proper type of argument for dma_addr_t and size_t.

Fixes: 263c9959c937 ("crypto: hisilicon - add queue management driver for HiSilicon QM module")
Reported-by: kbuild test robot <lkp@intel.com>
Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - add debugfs for ZIP and QM

HiSilicon ZIP engine driver uses debugfs to provide debug information,
the usage can be found in /Documentation/ABI/testing/debugfs-hisi-zip.

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - add SRIOV support for ZIP

HiSilicon ZIP engine supports PCI SRIOV. This patch enable this feature.
User can enable VFs and pass through them to VM, same ZIP driver can work
in VM to provide ZLIB and GZIP algorithm by crypto acomp interface.

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - add queue management driver for HiSilicon QM module

QM is a general IP used by HiSilicon accelerators. It provides a general
PCIe interface for the CPU and the accelerator to share a group of queues.

A QM integrated in an accelerator provides queue management service.
Queues can be assigned to PF and VFs, and queues can be controlled by
unified mailboxes and doorbells. Specific task request are descripted by
specific description buffer, which will be controlled and pass to related
accelerator IP by QM.

This patch adds a QM driver used by the accelerator driver to access
the QM hardware.

Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Kenneth Lee <liguozhu@hisilicon.com>
Signed-off-by: Shiju Jose <shiju.jose@huawei.com>
Signed-off-by: Hao Fang <fanghao11@huawei.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: John Garry <john.garry@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>