dmaengine: axi-dmac: Improve cyclic DMA transfers in SG mode

For cyclic transfers, chain the last descriptor to the first one, and
disable IRQ generation if there is no callback registered with the
cyclic transfer.

Signed-off-by: Paul Cercueil <paul@crapouillou.net>
Link: https://lore.kernel.org/r/20231215131313.23840-6-paul@crapouillou.net
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Use only EOT interrupts when doing scatter-gather

Instead of notifying userspace in the end-of-transfer (EOT) interrupt
and program the hardware in the start-of-transfer (SOT) interrupt, we
can do both things in the EOT, allowing us to mask the SOT, and halve
the number of interrupts sent by the HDL core.

Signed-off-by: Paul Cercueil <paul@crapouillou.net>
Link: https://lore.kernel.org/r/20231215131313.23840-5-paul@crapouillou.net
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Add support for scatter-gather transfers

Implement support for scatter-gather transfers. Build a chain of
hardware descriptors, each one corresponding to a segment of the
transfer, and linked to the next one. The hardware will transfer the
chain and only fire interrupts when the whole chain has been
transferred.

Support for scatter-gather is automatically enabled when the driver
detects that the hardware supports it, by writing then reading the
AXI_DMAC_REG_SG_ADDRESS register. If not available, the driver will fall
back to standard DMA transfers.

Signed-off-by: Paul Cercueil <paul@crapouillou.net>
Link: https://lore.kernel.org/r/20231215131313.23840-4-paul@crapouillou.net
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Allocate hardware descriptors

Change where and how the DMA transfers meta-data is stored, to prepare
for the upcoming introduction of scatter-gather support.

Allocate hardware descriptors in the format that the HDL core will be
expecting them when the scatter-gather feature is enabled, and use these
fields to store the data that was previously stored in the axi_dmac_sg
structure.

Note that the 'x_len' and 'y_len' fields now contain the transfer length
minus one, since that's what the hardware will expect in these fields.

Signed-off-by: Paul Cercueil <paul@crapouillou.net>
Link: https://lore.kernel.org/r/20231215131313.23840-3-paul@crapouillou.net
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Small code cleanup

Use a for() loop instead of a while() loop in axi_dmac_fill_linear_sg().
This makes the code leaner and cleaner overall, and does not introduce
any functional change.

Signed-off-by: Paul Cercueil <paul@crapouillou.net>
Link: https://lore.kernel.org/r/20231215131313.23840-2-paul@crapouillou.net
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Annotate struct axi_dmac_desc with __counted_by

Prepare for the coming implementation by GCC and Clang of the __counted_by
attribute. Flexible array members annotated with __counted_by can have
their accesses bounds-checked at run-time checking via CONFIG_UBSAN_BOUNDS
(for array indexing) and CONFIG_FORTIFY_SOURCE (for strcpy/memcpy-family
functions).

As found with Coccinelle[1], add __counted_by for struct axi_dmac_desc.
Additionally, since the element count member must be set before accessing
the annotated flexible array member, move its initialization earlier.

[1] https://github.com/kees/kernel-tools/blob/trunk/coccinelle/examples/counted_by.cocci

Cc: Vinod Koul <vkoul@kernel.org>
Cc: Lars-Peter Clausen <lars@metafoo.de>
Cc: dmaengine@vger.kernel.org
Signed-off-by: Kees Cook <keescook@chromium.org>
Reviewed-by: "Gustavo A. R. Silva" <gustavoars@kernel.org>
Link: https://lore.kernel.org/r/20230817235859.49846-3-keescook@chromium.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: dma-axi-dmac: Convert to platform remove callback returning void

The .remove() callback for a platform driver returns an int which makes
many driver authors wrongly assume it's possible to do error handling by
returning an error code. However the value returned is ignored (apart
from emitting a warning) and this typically results in resource leaks.
To improve here there is a quest to make the remove callback return
void. In the first step of this quest all drivers are converted to
.remove_new() which already returns void. Eventually after all drivers
are converted, .remove_new() is renamed to .remove().

Trivially convert this driver from always returning zero in the remove
callback to the void returning variant.

Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Link: https://lore.kernel.org/r/20230919133207.1400430-9-u.kleine-koenig@pengutronix.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Don't set chancnt

The dma framework will calculate the dma channels chancnt, setting it
ourself is wrong.

Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
Acked-by: Lars-Peter Clausen <lars@metafoo.de>
Link: https://lore.kernel.org/r/20230521100252.3197-3-jszhang@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: drivers: Use devm_platform_ioremap_resource()

platform_get_resource() and devm_ioremap_resource() are wrapped up in the
devm_platform_ioremap_resource() helper. Use the helper and get rid of the
local variable for struct resource *. We now have a function call less.

Signed-off-by: Tudor Ambarus <tudor.ambarus@microchip.com>
Acked-by: Jernej Skrabec <jernej.skrabec@gmail.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20221110152528.7821-1-tudor.ambarus@microchip.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: check cache coherency register

Marking the DMA as cache coherent (dma-coherent in devicetree) is only
safe with versions of axi_dmac that have this feature enabled.

Cc: Lars-Peter Clausen <lars@metafoo.de>
Cc: Vinod Koul <vkoul@kernel.org>
Acked-by: Nuno Sá <nuno.sa@analog.com>
Signed-off-by: Mathias Tausen <mta@satlab.com>
Link: https://lore.kernel.org/r/20220726140213.786939-1-mta@satlab.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Drop local dma_parms

Since commit 9495b7e92f71 ("driver core: platform: Initialize dma_parms
for platform devices"), struct platform_device already provides a
dma_parms structure, so we can save allocating another one.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Link: https://lore.kernel.org/r/9b759e4c9eb37c90a3616d31abe13af6a6dafcd2.1599164692.git.robin.murphy@arm.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: add support for reading bus attributes from registers

Starting with core version 4.3.a the DMA bus attributes can (and should) be
read from the INTERFACE_DESCRIPTION (0x10) register.

For older core versions, this will still need to be provided from the
device-tree.

The bus-type values are identical to the ones stored in the device-trees,
so we just need to read them. Bus-width values are stored in log2 values,
so we just need to use them as shift values to make them equivalent to the
current format.

Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Link: https://lore.kernel.org/r/20200825151950.57605-7-alexandru.ardelean@analog.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: wrap channel parameter adjust into function

The channel parameters (which are read from the device-tree) are adjusted
for the DMAEngine framework in the axi_dmac_parse_chan_dt() function, after
they are read from the device-tree.

When we want to read these from registers, we will need to use the same
logic, so this change splits the logic into a separate function.

Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Link: https://lore.kernel.org/r/20200825151950.57605-6-alexandru.ardelean@analog.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: wrap entire dt parse in a function

All these attributes will be read from registers in newer core versions, so
just wrap the logic into a function.

Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Link: https://lore.kernel.org/r/20200825151950.57605-5-alexandru.ardelean@analog.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: move clock enable earlier

The clock may also be required to read registers from the IP core (if it is
provided and the driver needs to control it).
So, move it earlier in the probe.

Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Link: https://lore.kernel.org/r/20200825151950.57605-4-alexandru.ardelean@analog.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: move active_descs list init after device-tree init

We want to enable the clock right after it is obtained. Then later we'll
want to read the core version via register-access (which requires the clock
to be enabled).

The initialization of the active_descs list can be postponed after reading
from registers (or reading the device-tree).

Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Link: https://lore.kernel.org/r/20200825151950.57605-3-alexandru.ardelean@analog.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: move version read in probe

The 'version' of the IP core will be needed to adapt the driver to a new
feature (i.e. reading some DMA parameters from registers).
To do that, the version will be checked, so this is being moved out of the
axi_dmac_detect_caps() function.

Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Link: https://lore.kernel.org/r/20200825151950.57605-2-alexandru.ardelean@analog.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: add a check for devm_regmap_init_mmio

The driver misses checking the result of devm_regmap_init_mmio().
Add a check to fix it.

Fixes: fc15be39a827 ("dmaengine: axi-dmac: add regmap support")
Signed-off-by: Chuhong Yuan <hslester96@gmail.com>
Reviewed-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Link: https://lore.kernel.org/r/20191209085711.16001-1-hslester96@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: add regmap support

The registers for AXI DMAC are detailed at:
https://wiki.analog.com/resources/fpga/docs/axi_dmac#register_map

This change adds regmap support for these registers, in case some wants to
have a more direct access to them via this interface.

Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
[vkoul: fixed code style issue]
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: terminate early DMA transfers after a partial one

When a partial transfer is received, the driver should not submit any more
segments to the hardware, as they will be ignored/unused until a new
transfer start operation is done.

This change implements this by adding a new flag on the AXI DMAC
descriptor. This flags is set to true, if there was a partial transfer in
a previously completed segment. When that flag is true, the TLAST flag is
added to the to the submitted segment, signaling the controller to stop
receiving more segments.

Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: populate residue info for completed xfers

Starting with version 4.2.a, the AXI DMAC controller can report partial
transfers that have been issued.

This change implements computing DMA residue information for transfers,
based on that reported information.

The way this is done, is to dequeue the partial transfers from the FIFO of
partial transfers, store the partial length to the correct segment &
descriptor, and compute the residue before submitting the DMA cookie to the
DMA framework.

Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: update license header

The change replaces the old license information in the comment header with
the new SPDX license specifier.
As well as bumping the year range from 2013-2015 to 2013-2019.

The latter also reflects recent changes that were added to the driver.

Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 177

Based on 1 normalized pattern(s):

licensed under the gpl 2

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-only

has been chosen to replace the boilerplate/reference in 135 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Allison Randal <allison@lohutok.net>
Reviewed-by: Richard Fontana <rfontana@redhat.com>
Reviewed-by: Alexios Zavras <alexios.zavras@intel.com>
Reviewed-by: Steve Winslow <swinslow@gmail.com>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190528170026.071193225@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

dmaengine: axi-dmac: assign `copy_align` property

The `copy_align` property is a generic property that describes alignment
for DMA memcpy & sg ops.
It serves mostly an informational purpose, and can be used in DMA tests, to
pass the info to know what alignment to expect.

Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Discover length alignment requirement

Starting with version 4.1.a the AXI-DMAC is capable of reporting the
required length alignment.

The LSBs that are required to be set for alignment will always read back as
set from the transfer length register. It is not possible to clear them by
writing a 0. This means the driver can discover the length alignment
requirement by writing 0 to that register and reading back the value.

Since the DMA will support length alignment requirements that are different
from the address alignment requirement track both of them independently.

For older versions of the peripheral assume that the length alignment
requirement is equal to the address alignment requirement.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Sanity check memory mapped interface support

The AXI-DMAC supports different types of interface for the data source and
destination ports. Typically one of those ports is a memory-mapped
interface while the other is some kind of streaming interface.

The information about which kind of interface is used for each port is
encoded in the devicetree.

It is also possible in the driver to detect whether a port supports
memory-mapped transfers or not. For streaming interfaces the address
register is read-only and will always return 0. So in order to check if a
port supports memory-mapped transfers write a non-zero value to the
corresponding address register and check that the value read-back is still
non zero.

This allows to detect mismatches between the devicetree description and the
actual hardware configuration.

Unfortunately it is not possible to autodetect the interface types since
there is no method to distinguish between the different streaming ports. So
the best thing that can be done is to error out when a memory mapped port
is described in the devicetree but none is detected in the hardware.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Enable TLAST handling

The TLAST flag is used by the DMAC HDL controller to signal to the
controller that the following segment (to be submitted) is the last one (in
a series of segments).

A receiver DMA (typically another DMAC) can read this parameter (from the
transfer), and terminate the transfer earlier. A typical use-case for this,
is when the receiver expects a certain amount of segments, but for some
reason (e.g. an ADC capture which can have an unknown number of digital
samples) the number of actual segments is smaller. The receiver would read
this flag, and then the DMAC would finish.

Signed-off-by: Michael Hennerich <michael.hennerich@analog.com>
Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Add support for interleaved cyclic transfers

The DMAC HDL core supports interleaved & cyclic transfers.
An example use-case for this mode is when the controller is used as a
video DMA.

This change sets the `cyclic` field to true, so that when the IRQ comes and
the `axi_dmac_transfer_done()` callback is called (from the interrupt
handler) the proper `vchan_cyclic_callback()` is called. This way the
DMAEngine framework will process data correctly for interleaved + cyclic
transfers.

This doesn't fix anything. It's an enhancement to the driver.

Signed-off-by: Dragos Bogdan <dragos.bogdan@analog.com>
Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Enable DMA_INTERLEAVE capability

Since device_prep_interleaved_dma() is already implemented, the
DMA_INTERLEAVE capability should be set.

Signed-off-by: Dragos Bogdan <dragos.bogdan@analog.com>
Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Don't check the number of frames for alignment

In 2D transfers (for the AXI DMAC), the number of frames (numf) represents
Y_LENGTH, and the length of a frame is X_LENGTH. 2D transfers are useful
for video transfers where screen resolutions ( X * Y ) are typically
aligned for X, but not for Y.

There is no requirement for Y_LENGTH to be aligned to the bus-width (or
anything), and this is also true for AXI DMAC.

Checking the Y_LENGTH for alignment causes false errors when initiating DMA
transfers. This change fixes this by checking only that the Y_LENGTH is
non-zero.

Fixes: 0e3b67b348b8 ("dmaengine: Add support for the Analog Devices AXI-DMAC DMA controller")
Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Infer synthesis configuration parameters hardware

Some synthesis time configuration parameters of the DMA controller can be
inferred from the hardware itself.

Use this information as it is more reliably than the information specified
in the devicetree which might be outdated if the HDL project got changed.

Deprecate the devicetree properties that can be inferred from the hardware
itself.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Alexandru Ardelean <alexandru.ardelean@analog.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Split too large segments

The axi-dmac driver currently rejects transfers with segments that are
larger than what the hardware can handle.

Re-work the driver so that these large segments are split into multiple
segments instead where each segment is smaller or equal to the maximum
segment size.

This allows the driver to handle transfers with segments of arbitrary size.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Bogdan Togorean <bogdan.togorean@analog.com>
Signed-off-by: Alexandru Ardelean <alex.ardelean@analog.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Use struct_size() in kzalloc()

One of the more common cases of allocation size calculations is finding the
size of a structure that has a zero-sized array at the end, along with memory
for some number of elements for that array. For example:

struct foo {
int stuff;
void *entry[];
};

instance = kzalloc(sizeof(struct foo) + sizeof(void *) * count, GFP_KERNEL);

Instead of leaving these open-coded and prone to type mistakes, we can now
use the new struct_size() helper:

instance = kzalloc(struct_size(instance, entry, count), GFP_KERNEL);

This code was detected with the help of Coccinelle.

Signed-off-by: Gustavo A. R. Silva <gustavo@embeddedor.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Request IRQ with IRQF_SHARED

Request IRQ with IRQF_SHARED flag to enable setups with multiple
instances of the core sharing a single IRQ line.
This works out since the IRQ handler already checks if there is
an actual IRQ pending and returns IRQ_NONE otherwise.

Acked-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Moritz Fischer <mdf@kernel.org>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: axi-dmac: Fix software cyclic mode

When running in software cyclic mode the driver currently does not go back
to the first segment once the last segment has been reached. Effectively
making the transfer non-cyclic.

Fix this by going back to the first segment once the last segment has been
reached for cyclic transfers.

Special care need to be taken to avoid a segment from being submitted
multiple times concurrently, which could happen for transfers with a number
of segments that is smaller than the DMA controller's internal queue.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: axi-dmac: Only use hardware cyclic mode for single segment transfers

In hardware cyclic mode the submitted segment is repeated. This means
hardware cyclic mode can only be used if the transfer has a single segment.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: axi-dmac: Return IRQ_NONE if no IRQs are pending

Return IRQ_NONE in the interrupt handler when it is called but no IRQs are
pending. This allows the system to recover in case of an interrupt storm
e.g. due to a wrong interrupt configuration setup.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: axi-dmac: Propagate errors from platform_get_irq()

Propagate errors returned by platform_get_irq() to the driver core. This
will enable proper probe deferring for the driver in case the IRQ provider
has not been registered yet.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: axi-dmac: Add MODULE_DEVICE_TABLE()

Add MODULE_DEVICE_TABLE() for the axi-dmac driver. This allows the driver
to be loaded on demand when built as a module.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: axi_dmac: Add synchronization support

Implement the new device_synchronize() callback to allow proper
synchronization when stopping a channel. Since the driver already makes
sure that no new complete callbacks are scheduled after the
device_terminate_all() callback has been called, all left to do in the
device_synchronize() callback is to wait for all currently running complete
callbacks to finish.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: Add support for the Analog Devices AXI-DMAC DMA controller

Add support for the Analog Devices AXI-DMAC DMA controller. This controller
is a soft peripheral that can be instantiated in a FPGA and is often used
in Analog Devices' reference designs for FPGA platforms.

The peripheral has various configuration options that can be selected at
synthesis time and influence the supported features of the instantiated
peripheral, those options are represented as device-tree properties to
allow the driver to behave accordingly.

The peripheral has a zero latency architecture, which means it is possible
to switch from one to the next descriptor without any delay. This is
archived by having a internal queue which can hold multiple descriptors.
The driver supports this, which means it will submit new descriptors
directly to the hardware until the queue is full and not wait for a
descriptor to complete before the next one is submitted. Interrupts are
used for the descriptor queue flow control.

Currently the driver supports SG, cyclic and interleaved slave DMA.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>