#
ed3d95fe |
|
16-Feb-2024 |
Damian Muszynski <damian.muszynski@intel.com> |
crypto: qat - make ring to service map common for QAT GEN4 The function get_ring_to_svc_map() is present in both 420xx and 4xxx drivers. Rework the logic to make it generic to GEN4 devices and move it to qat_common/adf_gen4_hw_data.c. Signed-off-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
df018f82 |
|
16-Feb-2024 |
Damian Muszynski <damian.muszynski@intel.com> |
crypto: qat - fix ring to service map for dcc in 4xxx If a device is configured for data compression chaining (dcc), half of the engines are loaded with the symmetric crypto image and the rest are loaded with the compression image. However, in such configuration all rings can handle compression requests. Fix the ring to service mapping so that when a device is configured for dcc, the ring to service mapping reports that all rings in a bank can be used for compression. Fixes: a238487f7965 ("crypto: qat - fix ring to service map for QAT GEN4") Signed-off-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
e1d54d15 |
|
19-Jan-2024 |
Damian Muszynski <damian.muszynski@intel.com> |
crypto: qat - fix arbiter mapping generation algorithm for QAT 402xx The commit "crypto: qat - generate dynamically arbiter mappings" introduced a regression on qat_402xx devices. This is reported when the driver probes the device, as indicated by the following error messages: 4xxx 0000:0b:00.0: enabling device (0140 -> 0142) 4xxx 0000:0b:00.0: Generate of the thread to arbiter map failed 4xxx 0000:0b:00.0: Direct firmware load for qat_402xx_mmp.bin failed with error -2 The root cause of this issue was the omission of a necessary function pointer required by the mapping algorithm during the implementation. Fix it by adding the missing function pointer. Fixes: 5da6a2d5353e ("crypto: qat - generate dynamically arbiter mappings") Signed-off-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
5da6a2d53 |
|
22-Dec-2023 |
Damian Muszynski <damian.muszynski@intel.com> |
crypto: qat - generate dynamically arbiter mappings The thread-to-arbiter mapping describes which arbiter can assign jobs to an acceleration engine thread. The existing mappings are functionally correct, but hardcoded and not optimized. Replace the static mappings with an algorithm that generates optimal mappings, based on the loaded configuration. The logic has been made common so that it can be shared between all QAT GEN4 devices. Signed-off-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
eb527077 |
|
22-Dec-2023 |
Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com> |
crypto: qat - add support for ring pair level telemetry Expose through debugfs ring pair telemetry data for QAT GEN4 devices. This allows to gather metrics about the PCIe channel and device TLB for a selected ring pair. It is possible to monitor maximum 4 ring pairs at the time per device. For details, refer to debugfs-driver-qat_telemetry in Documentation/ABI. This patch is based on earlier work done by Wojciech Ziemba. Signed-off-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Damian Muszynski <damian.muszynski@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
69e7649f |
|
22-Dec-2023 |
Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com> |
crypto: qat - add support for device telemetry Expose through debugfs device telemetry data for QAT GEN4 devices. This allows to gather metrics about the performance and the utilization of a device. In particular, statistics on (1) the utilization of the PCIe channel, (2) address translation, when SVA is enabled and (3) the internal engines for crypto and data compression. If telemetry is supported by the firmware, the driver allocates a DMA region and a circular buffer. When telemetry is enabled, through the `control` attribute in debugfs, the driver sends to the firmware, via the admin interface, the `TL_START` command. This triggers the device to periodically gather telemetry data from hardware registers and write it into the DMA memory region. The device writes into the shared region every second. The driver, every 500ms, snapshots the DMA shared region into the circular buffer. This is then used to compute basic metric (min/max/average) on each counter, every time the `device_data` attribute is queried. Telemetry counters are exposed through debugfs in the folder /sys/kernel/debug/qat_<device>_<BDF>/telemetry. For details, refer to debugfs-driver-qat_telemetry in Documentation/ABI. This patch is based on earlier work done by Wojciech Ziemba. Signed-off-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Damian Muszynski <damian.muszynski@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
98a4f29f |
|
15-Dec-2023 |
Jie Wang <jie.wang@intel.com> |
crypto: qat - move fw config related structures Relocate the structures adf_fw_objs and adf_fw_config from the file adf_4xxx_hw_data.c to the newly created adf_fw_config.h. These structures will be used by new device drivers. This does not introduce any functional change. Signed-off-by: Jie Wang <jie.wang@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
de51d223 |
|
15-Dec-2023 |
Jie Wang <jie.wang@intel.com> |
crypto: qat - relocate portions of qat_4xxx code Move logic that is common between QAT GEN4 accelerators to the qat_common folder. This includes addresses of CSRs, setters and configuration logic. When moved, functions and defines have been renamed from 4XXX to GEN4. Code specific to the device is moved to the file adf_gen4_hw_data.c. Code related to configuration is moved to the newly created adf_gen4_config.c. This does not introduce any functional change. Signed-off-by: Jie Wang <jie.wang@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
b34bd0fd |
|
15-Dec-2023 |
Jie Wang <jie.wang@intel.com> |
crypto: qat - change signature of uof_get_num_objs() Add accel_dev as parameter of the function uof_get_num_objs(). This is in preparation for the introduction of the QAT 420xx driver as it will allow to reconfigure the ae_mask when a configuration that does not require all AEs is loaded on the device. This does not introduce any functional change. Signed-off-by: Jie Wang <jie.wang@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
4db87a5f |
|
15-Dec-2023 |
Jie Wang <jie.wang@intel.com> |
crypto: qat - relocate and rename get_service_enabled() Move the function get_service_enabled() from adf_4xxx_hw_data.c to adf_cfg_services.c and rename it as adf_get_service_enabled(). This function is not specific to the 4xxx and will be used by other QAT drivers. This does not introduce any functional change. Signed-off-by: Jie Wang <jie.wang@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
03c76e8e |
|
20-Oct-2023 |
Giovanni Cabiddu <giovanni.cabiddu@intel.com> |
crypto: qat - move adf_cfg_services The file adf_cfg_services.h cannot be included in header files since it instantiates the structure adf_cfg_services. Move that structure to its own file and export the symbol. This does not introduce any functional change. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Damian Muszynski <damian.muszynski@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
d9fb8408 |
|
20-Oct-2023 |
Damian Muszynski <damian.muszynski@intel.com> |
crypto: qat - add rate limiting feature to qat_4xxx The Rate Limiting (RL) feature allows to control the rate of requests that can be submitted on a ring pair (RP). This allows sharing a QAT device among multiple users while ensuring a guaranteed throughput. The driver provides a mechanism that allows users to set policies, that are programmed to the device. The device is then enforcing those policies. Configuration of RL is accomplished through entities called SLAs (Service Level Agreement). Each SLA object gets a unique identifier and defines the limitations for a single service across up to four ring pairs (RPs count allocated to a single VF). The rate is determined using two fields: * CIR (Committed Information Rate), i.e., the guaranteed rate. * PIR (Peak Information Rate), i.e., the maximum rate achievable when the device has available resources. The rate values are expressed in permille scale i.e. 0-1000. Ring pair selection is achieved by providing a 64-bit mask, where each bit corresponds to one of the ring pairs. This adds an interface and logic that allow to add, update, retrieve and remove an SLA. Signed-off-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Tero Kristo <tero.kristo@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
8e6857f7 |
|
20-Oct-2023 |
Giovanni Cabiddu <giovanni.cabiddu@intel.com> |
crypto: qat - move admin api The admin API is growing and deserves its own include. Move it from adf_common_drv.h to adf_admin.h. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Tero Kristo <tero.kristo@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
a238487f |
|
20-Oct-2023 |
Giovanni Cabiddu <giovanni.cabiddu@intel.com> |
crypto: qat - fix ring to service map for QAT GEN4 The 4xxx drivers hardcode the ring to service mapping. However, when additional configurations where added to the driver, the mappings were not updated. This implies that an incorrect mapping might be reported through pfvf for certain configurations. Add an algorithm that computes the correct ring to service mapping based on the firmware loaded on the device. Fixes: 0cec19c761e5 ("crypto: qat - add support for compression for 4xxx") Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Tero Kristo <tero.kristo@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
f7df2329 |
|
20-Oct-2023 |
Giovanni Cabiddu <giovanni.cabiddu@intel.com> |
crypto: qat - use masks for AE groups The adf_fw_config structures hardcode a bit mask that represents the acceleration engines (AEs) where a certain firmware image will have to be loaded to. Remove the hardcoded masks and replace them with defines. This does not introduce any functional change. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Tero Kristo <tero.kristo@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
2990d2ed |
|
20-Oct-2023 |
Giovanni Cabiddu <giovanni.cabiddu@intel.com> |
crypto: qat - refactor fw config related functions The logic that selects the correct adf_fw_config structure based on the configured service is replicated twice in the uof_get_name() and uof_get_ae_mask() functions. Refactor the code so that there is no replication. This does not introduce any functional change. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Tero Kristo <tero.kristo@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
895f7d53 |
|
20-Oct-2023 |
Shashank Gupta <shashank.gupta@intel.com> |
crypto: qat - add handling of errors from ERRSOU2 for QAT GEN4 Add logic to detect, report and handle uncorrectable errors reported through the ERRSOU2 register in QAT GEN4 devices. Signed-off-by: Shashank Gupta <shashank.gupta@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Tero Kristo <tero.kristo@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
4926e89d |
|
20-Oct-2023 |
Shashank Gupta <shashank.gupta@intel.com> |
crypto: qat - add reporting of errors from ERRSOU1 for QAT GEN4 Add logic to detect and report uncorrectable errors reported through the ERRSOU1 register in QAT GEN4 devices. This also introduces the adf_dev_err_mask structure as part of adf_hw_device_data which will allow to provide different error masks per device generation. Signed-off-by: Shashank Gupta <shashank.gupta@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Tero Kristo <tero.kristo@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
93b2f7de |
|
20-Oct-2023 |
Shashank Gupta <shashank.gupta@intel.com> |
crypto: qat - add infrastructure for error reporting Add infrastructure for enabling, disabling and reporting errors in the QAT driver. This adds a new structure, adf_ras_ops, to adf_hw_device_data that contains the following methods: - enable_ras_errors(): allows to enable RAS errors at device initialization. - disable_ras_errors(): allows to disable RAS errors at device shutdown. - handle_interrupt(): allows to detect if there is an error and report if a reset is required. This is executed immediately after the error is reported, in the context of an ISR. An initial, empty, implementation of the methods above is provided for QAT GEN4. Signed-off-by: Shashank Gupta <shashank.gupta@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Tero Kristo <tero.kristo@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
37b14f2d |
|
14-Sep-2023 |
Adam Guerin <adam.guerin@intel.com> |
crypto: qat - enable dc chaining service QAT GEN4 devices support chained compression operations. These allow, with a single request to firmware, to hash then compress data. Extend the configuration to enable such mode. The cfg_services operations in sysfs are extended to allow the string "dcc". When selected, the driver downloads to the device both the symmetric crypto and the compression firmware images and sends an admin message to firmware which enables `chained` operations. In addition, it sets the device's capabilities as the combination of compression and symmetric crypto capabilities, while excluding the ICP_ACCEL_CAPABILITIES_CRYPTO_SYMMETRIC bit to indicate that in this mode, symmetric crypto instances are not supported. When "dcc" is enabled, the device will handle compression requests as if the "dc" configuration is loaded ("dcc" is a variation of "dc") and the driver will register the acomp algorithms. As for the other extended configurations, "dcc" is only available for qat_4xxx devices and the chaining service will be only accessible from user space. Signed-off-by: Adam Guerin <adam.guerin@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
71713766 |
|
14-Sep-2023 |
Giovanni Cabiddu <giovanni.cabiddu@intel.com> |
crypto: qat - consolidate services structure The data structure that associates a service id with its name is replicated across the driver. Remove duplication by moving this data structure to a new include file, adf_cfg_services.h in order to have consistency across the drivers. Note that the data structure is re-instantiated every time the new include is added to a compilation unit. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
fab9516f |
|
14-Aug-2023 |
Adam Guerin <adam.guerin@intel.com> |
crypto: qat - fix crypto capability detection for 4xxx When extending the capability detection logic for 4xxx devices the SMx algorithms were accidentally missed. Enable these SMx capabilities by default for QAT GEN4 devices. Check for device variants where the SMx algorithms are explicitly disabled by the GEN4 hardware. This is indicated in fusectl1 register. Mask out SM3 and SM4 based on a bit specific to those algorithms. Mask out SM2 if the PKE slice is not present. Fixes: 4b44d28c715d ("crypto: qat - extend crypto capability detection for 4xxx") Signed-off-by: Adam Guerin <adam.guerin@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Fiona Trahe <fiona.trahe@intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
359b84f8 |
|
30-Jun-2023 |
Damian Muszynski <damian.muszynski@intel.com> |
crypto: qat - add heartbeat feature Under some circumstances, firmware in the QAT devices could become unresponsive. The Heartbeat feature provides a mechanism to detect unresponsive devices. The QAT FW periodically writes to memory a set of counters that allow to detect the liveness of a device. This patch adds logic to enable the reporting of those counters, analyze them and report if a device is alive or not. In particular this adds (1) heartbeat enabling, reading and detection logic (2) reporting of heartbeat status and configuration via debugfs (3) documentation for the newly created sysfs entries (4) configuration of FW settings related to heartbeat, e.g. tick period (5) logic to convert time in ms (provided by the user) to clock ticks This patch introduces a new folder in debugfs called heartbeat with the following attributes: - status - queries_sent - queries_failed - config All attributes except config are reading only. In particular: - `status` file returns 0 when device is operational and -1 otherwise. - `queries_sent` returns the total number of heartbeat queries sent. - `queries_failed` returns the total number of heartbeat queries failed. - `config` allows to adjust the frequency at which the firmware writes counters to memory. This period is given in milliseconds and it is fixed for GEN4 devices. Signed-off-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
e2980ba5 |
|
30-Jun-2023 |
Damian Muszynski <damian.muszynski@intel.com> |
crypto: qat - add measure clock frequency The QAT hardware does not expose a mechanism to report its clock frequency. This is required to implement the Heartbeat feature. Add a clock measuring algorithm that estimates the frequency by comparing the internal timestamp counter incremented by the firmware with the time measured by the kernel. The frequency value is only used internally and not exposed to the user. Signed-off-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
fd77d8da |
|
30-Jun-2023 |
Damian Muszynski <damian.muszynski@intel.com> |
crypto: qat - add internal timer for qat 4xxx The power management feature in QAT 4xxx devices can disable clock sources used to implement timers. Because of that, the firmware needs to get an external reliable source of time. Add a kernel delayed work that periodically sends an event to the firmware. This is triggered every 200ms. At each execution, the driver sends a sync request to the firmware reporting the current timestamp counter value. This is a pre-requisite for enabling the heartbeat, telemetry and rate limiting features. Signed-off-by: Damian Muszynski <damian.muszynski@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
50053275 |
|
09-Jun-2023 |
Adam Guerin <adam.guerin@intel.com> |
crypto: qat - extend configuration for 4xxx A QAT GEN4 device can be currently configured for crypto (sym;asym) or compression (dc). This patch extends the configuration to support more variations of these services, download the correct FW images on the device and report the correct capabilities on the device based on the configured service. The device can now be configured with the following services: "sym", "asym", "dc", "sym;asym", "asym;sym", "sym;dc", "dc;sym", "asym;dc", "dc;asym". With this change, the configuration "sym", "asym", "sym;dc", "dc;sym", "asym;dc", "dc;asym" will be accessible only via userspace, i.e. the driver for those configurations will not register into the crypto framework. Support for such configurations in kernel will be enabled in a later patch. The pairs "sym;asym" and "asym;sym" result in identical device config. As do "sym;dc", "dc;sym", and "asym;dc", "dc;asym". Signed-off-by: Adam Guerin <adam.guerin@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
10484c64 |
|
09-Jun-2023 |
Giovanni Cabiddu <giovanni.cabiddu@intel.com> |
crypto: qat - refactor fw config logic for 4xxx The data structure adf_fw_config is used to select which firmware image is loaded on a certain set of accelerator engines. When support for 402xx was added, the adf_fw_config arrays were duplicated in order to select different firmware images. Since the configurations are the same regardless of the QAT GEN4 flavour, in preparation for adding support for multiple configurations, refactor the logic that retrieves the firmware names in the 4xxx driver. The structure adf_fw_config has been changed to contain a firmware object id that is then mapped to a firmware name depending of the device type. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Adam Guerin <adam.guerin@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
52f9a284 |
|
09-Jun-2023 |
Giovanni Cabiddu <giovanni.cabiddu@intel.com> |
crypto: qat - make fw images name constant Update fw image names to be constant throughout the driver. Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Adam Guerin <adam.guerin@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
038ccc28 |
|
09-Jun-2023 |
Adam Guerin <adam.guerin@intel.com> |
crypto: qat - move returns to default case Make use of the default statements by changing the pattern: switch(condition) { case COND_A: ... break; case COND_b: ... break; } return ret; in switch(condition) { case COND_A: ... break; case COND_b: ... break; default: return ret; } Signed-off-by: Adam Guerin <adam.guerin@intel.com> Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
#
a4b16dad |
|
28-Mar-2023 |
Tom Zanussi <tom.zanussi@linux.intel.com> |
crypto: qat - Move driver to drivers/crypto/intel/qat With the growing number of Intel crypto drivers, it makes sense to group them all into a single drivers/crypto/intel/ directory. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|