dmaengine: fsl-edma: integrate TCD64 support for i.MX95

In i.MX95's edma version 5, the TCD structure is extended to support 64-bit
addresses for fields like saddr and daddr. To prevent code duplication,
employ help macros to handle the fields, as the field names remain the same
between TCD and TCD64.

Change local variables related to TCD addresses from 'u32' to 'dma_addr_t'
to accept 64-bit DMA addresses.

Change 'vtcd' type to 'void *' to avoid direct use. Use helper macros to
access the TCD fields correctly.

Call 'dma_set_mask_and_coherent(64)' when TCD64 is supported.

Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://lore.kernel.org/r/20231221153528.1588049-7-Frank.Li@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: add address for channel mux register in fsl_edma_chan

iMX95 move channel mux register to management page address space. This
prepare to support iMX95.

Add mux_addr in struct fsl_edma_chan. No function change.

Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://lore.kernel.org/r/20231221153528.1588049-4-Frank.Li@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: involve help macro fsl_edma_set(get)_tcd()

Using help macro fsl_edma_set(get)_tcd() and edma_cp_tcd_to_reg() to handle
difference field size. This is not function change and prepare for 64bit
tcd in imx95.

Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://lore.kernel.org/r/20231221153528.1588049-2-Frank.Li@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: correct calculation of 'nbytes' in multi-fifo scenario

The 'nbytes' should be equivalent to burst * width in audio multi-fifo
setups. Given that the FIFO width is fixed at 32 bits, adjusts the burst
size for multi-fifo configurations to match the slave maxburst in the
configuration.

Cc: stable@vger.kernel.org
Fixes: 72f5801a4e2b ("dmaengine: fsl-edma: integrate v3 support")
Signed-off-by: Joy Zou <joy.zou@nxp.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://lore.kernel.org/r/20240131163318.360315-1-Frank.Li@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: fix DMA channel leak in eDMAv4

Allocate channel count consistently increases due to a missing source ID
(srcid) cleanup in the fsl_edma_free_chan_resources() function at imx93
eDMAv4.

Reset 'srcid' at fsl_edma_free_chan_resources().

Cc: stable@vger.kernel.org
Fixes: 72f5801a4e2b ("dmaengine: fsl-edma: integrate v3 support")
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://lore.kernel.org/r/20231127214325.2477247-1-Frank.Li@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: fix edma4 channel enable failure on second attempt

When attempting to start DMA for the second time using
fsl_edma3_enable_request(), channel never start.

CHn_MUX must have a unique value when selecting a peripheral slot in the
channel mux configuration. The only value that may overlap is source 0.
If there is an attempt to write a mux configuration value that is already
consumed by another channel, a mux configuration of 0 (SRC = 0) will be
written.

Check CHn_MUX before writing in fsl_edma3_enable_request().

Fixes: 72f5801a4e2b ("dmaengine: fsl-edma: integrate v3 support")
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://lore.kernel.org/r/20230823182635.2618118-1-Frank.Li@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-dma: fix DMA error when enabling sg if 'DONE' bit is set

In eDMAv3, clearing 'DONE' bit (bit 30) of CHn_CSR is required when
enabling scatter-gather (SG). eDMAv4 does not require this change.

Cc: stable@vger.kernel.org
Fixes: 72f5801a4e2b ("dmaengine: fsl-edma: integrate v3 support")
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://lore.kernel.org/r/20230921144652.3259813-1-Frank.Li@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: integrate v3 support

Significant alterations have been made to the EDMA v3's register layout.
Now, each channel possesses a separate address space, encapsulating all
channel-related controls and statuses, including IRQs. There are changes
in bit position definitions as well. However, the fundamental control flow
remains analogous to the previous versions.

EDMA v3 was utilized in imx8qm, imx93, and will be in forthcoming chips.

Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://lore.kernel.org/r/20230821161617.2142561-13-Frank.Li@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: move tcd into struct fsl_dma_chan

Relocates the tcd into the fsl_dma_chan structure. This adjustment reduces
the need to reference back to fsl_edma_engine, paving the way for EDMA V3
support.

Unified the edma_writel and edma_writew functions for accessing TCD
(Transfer Control Descriptor) registers. A new macro is added that can
automatically detect whether a 32-bit or 16-bit access should be used
based on the structure field definition. This provide better support
64-bit TCD with future v5 version.

Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202305271951.gmRobs3a-lkp@intel.com/
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://lore.kernel.org/r/20230821161617.2142561-11-Frank.Li@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: simply ATTR_DSIZE and ATTR_SSIZE by using ffs()

Removes all ATTR_DSIZE_*BIT(BYTE) and ATTR_SSIZE_*BIT(BYTE) definitions
in edma. Uses ffs() instead, as it gives identical results. This simplifies
the code and avoids adding more similar definitions in future V3 version.

Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://lore.kernel.org/r/20230821161617.2142561-7-Frank.Li@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: move common IRQ handler to common.c

Move the common part of IRQ handler from fsl-edma-main.c and
mcf-edma-main.c to fsl-edma-common.c. This eliminates redundant code, as
the both files contains mostly identical code.

Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://lore.kernel.org/r/20230821161617.2142561-6-Frank.Li@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: Remove enum edma_version

The enum edma_version, which defines v1, v2, and v3, is a software concept
used to distinguish IP differences. However, it is not aligned with the
chip reference manual. According to the 7ulp reference manual, it should
be edma2. In the future, there will be edma3, edma4, and edma5, which
could cause confusion. To avoid this confusion, remove the edma_version
and instead use drvdata->flags to distinguish the IP difference.

Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://lore.kernel.org/r/20230821161617.2142561-5-Frank.Li@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: transition from bool fields to bitmask flags in drvdata

Replace individual bool fields with bitmask flags within drvdata. This
will facilitate future extensions, making it easier to add more flags to
accommodate new versions of the edma IP.

Signed-off-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Peng Fan <peng.fan@nxp.com>
Link: https://lore.kernel.org/r/20230821161617.2142561-4-Frank.Li@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: clean up EXPORT_SYMBOL_GPL in fsl-edma-common.c

Exported functions in fsl-edma-common.c are only used within
fsl-edma.c and mcf-edma.c. Global export is unnecessary.

This commit removes all EXPORT_SYMBOL_GPL in fsl-edma-common.c,
and renames fsl-edma.c and mcf-edma.c to maintain the same
final module names as before, thereby simplifying the codebase.

Signed-off-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Peng Fan <peng.fan@nxp.com>
Link: https://lore.kernel.org/r/20230821161617.2142561-3-Frank.Li@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: remove redundant assignment to pointer last_sg

The pointer last_sg is being assigned a value at the start of a loop
however it is never read and is being re-assigned later on in both
brances of an if-statement. The assignment is redundant and can be
removed.

Cleans up clang scan-build warning:
drivers/dma/fsl-edma-common.c:563:3: warning: Value stored to 'last_sg'
is never read [deadcode.DeadStores]

Signed-off-by: Colin Ian King <colin.i.king@gmail.com>
Link: https://lore.kernel.org/r/20220614184759.164379-1-colin.i.king@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: support edma memcpy

Add memcpy in edma. The edma has the capability to transfer data by
software trigger so that it could be used for memory copy. Enable
MEMCPY for edma driver and it could be test directly by dmatest.

Signed-off-by: Joy Zou <joy.zou@nxp.com>
Link: https://lore.kernel.org/r/20211026090025.2777292-1-joy.zou@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: fix for missing dmamux module

Fix following panic on system halt:

Requesting system halt
[ 10.600000] spi spi0.1: spi_device 0.1 cleanup
[ 10.630000] fsl_edma_chan_mux() fsl_chan->edma->n_chans 64 dmamux_nr 0
[ 10.630000] *** ZERO DIVIDE *** FORMAT=4
[ 10.630000] Current process id is 38
[ 10.630000] BAD KERNEL TRAP: 00000000
[ 10.630000] PC: [<402f09ba>] fsl_edma_chan_mux+0x7c/0x12e
...

Some architecture as mcf5441x (ColdFire) may not have
a dmamux, so dmamux_nr is set to 0. This patch considers this case.

Signed-off-by: Angelo Dureghello <angelo.dureghello@timesys.com>
Link: https://lore.kernel.org/r/20210901211610.662077-1-angelo.dureghello@timesys.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: fix wrong tcd endianness for big-endian cpu

Due to recent fixes in m68k arch-specific I/O accessor macros, this
driver is not working anymore for ColdFire. Fix wrong tcd endianness
removing additional swaps, since edma_writex() functions should already
take care of any eventual swap if needed.

Note, i could only test the change in ColdFire mcf54415 and Vybrid
vf50 / Colibri where i don't see any issue. So, every feedback and
test for all other SoCs involved is really appreciated.

Signed-off-by: Angelo Dureghello <angelo.dureghello@timesys.com>
Reported-by: kbuild test robot <lkp@intel.com>
Link: https://lore.kernel.org/r/20200701225205.1674463-1-angelo.dureghello@timesys.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: Add lockdep assert for exported function

Add lockdep assert for an exported function expected to be called under
spin lock. Since this function is called in different modules, the
lockdep assert will be self-documenting note about need for locking.

Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
Reviewed-by: Robin Gong <yibin.gong@nxp.com>
Link: https://lore.kernel.org/r/1591877861-28156-1-git-send-email-krzk@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: Add eDMA support for QorIQ LS1028A platform

Our platforms(such as LS1021A, LS1012A, LS1043A, LS1046A, LS1028A) with
below registers(CHCFG0 - CHCFG15) of eDMA as follows:
*-----------------------------------------------------------*
| Offset | OTHERS | LS1028A |
|--------------|--------------------|-----------------------|
| 0x0 | CHCFG0 | CHCFG3 |
|--------------|--------------------|-----------------------|
| 0x1 | CHCFG1 | CHCFG2 |
|--------------|--------------------|-----------------------|
| 0x2 | CHCFG2 | CHCFG1 |
|--------------|--------------------|-----------------------|
| 0x3 | CHCFG3 | CHCFG0 |
|--------------|--------------------|-----------------------|
| ... | ...... | ...... |
|--------------|--------------------|-----------------------|
| 0xC | CHCFG12 | CHCFG15 |
|--------------|--------------------|-----------------------|
| 0xD | CHCFG13 | CHCFG14 |
|--------------|--------------------|-----------------------|
| 0xE | CHCFG14 | CHCFG13 |
|--------------|--------------------|-----------------------|
| 0xF | CHCFG15 | CHCFG12 |
*-----------------------------------------------------------*

This patch is to improve edma driver to fit LS1028A platform.

Signed-off-by: Peng Ma <peng.ma@nxp.com>
Reviewed-by: Robin Gong <yibin.gong@nxp.com>
Link: https://lore.kernel.org/r/20191212033714.4090-1-peng.ma@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: change alignment of mux_configure32 and fsl_edma_chan_mux

The alignment of mux_configure32() and fsl_edma_chan_mux() need
to be adjusted, it must start precisely at the first column after
the openning parenthesis of the first line.

Fixes: 9d831528a656 ("dmaengine: fsl-edma: extract common fsl-edma code (no changes in behavior intended)")
Signed-off-by: Mao Wenan <maowenan@huawei.com>
Link: https://lore.kernel.org/r/20190814072105.144107-3-maowenan@huawei.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: make mux_configure32 static

There is one sparse warning in drivers/dma/fsl-edma-common.c:
drivers/dma/fsl-edma-common.c:93:6: warning: symbol 'mux_configure32'
was not declared. Should it be static?

Fix it by setting mux_configure32() as static.

Fixes: 232a7f18cf8ec ("dmaengine: fsl-edma: add i.mx7ulp edma2 version support")
Signed-off-by: Mao Wenan <maowenan@huawei.com>
Link: https://lore.kernel.org/r/20190814072105.144107-2-maowenan@huawei.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: add i.mx7ulp edma2 version support

Add edma2 for i.mx7ulp by version v3, since v2 has already
been used by mcf-edma.
The big changes based on v1 are belows:
1. only one dmamux.
2. another clock dma_clk except dmamux clk.
3. 16 independent interrupts instead of only one interrupt for
all channels.

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Link: https://lore.kernel.org/r/1563952834-7731-1-git-send-email-yibin.gong@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: Revert "dmaengine: fsl-edma: add i.mx7ulp edma2 version support"

This reverts commit 7144afd025b2 ("dmaengine: fsl-edma: add i.mx7ulp
edma2 version support") as this fails to build with module option due to
usage of of_irq_count() which is not an exported symbol as kernel
drivers are *not* expected to use it (rightly so).

Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: add i.mx7ulp edma2 version support

Add edma2 for i.mx7ulp by version v3, since v2 has already
been used by mcf-edma.
The big changes based on v1 are belows:
1. only one dmamux.
2. another clock dma_clk except dmamux clk.
3. 16 independent interrupts instead of only one interrupt for
all channels.

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Tested-by: Angelo Dureghello <angelo@sysam.it>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma-common: version check for v2 instead

The next v3 i.mx7ulp edma is based on v1, so change version
check logic for v2 instead.

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Tested-by: Angelo Dureghello <angelo@sysam.it>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma-common: move dmamux register to another single function

Prepare for edmav2 on i.mx7ulp whose dmamux register is 32bit. No function
impacted.

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Tested-by: Angelo Dureghello <angelo@sysam.it>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: add drvdata for fsl-edma

There are some differences between vf610 and next i.mx7ulp. Put such
differences into static driver data for distinguishing easily at
driver level. Change mcf-edma accordingly.

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Tested-by: Angelo Dureghello <angelo@sysam.it>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: Revert "dmaengine: fsl-edma: support little endian for edma driver"

This reverts commit 002905eca5bedab08bafd9e325bbbb41670c7712.

Commit 002905eca5be ("dmaengine: fsl-edma: support little endian for edma
driver") incorrectly assumed that there was not little endian support
in the driver.

This causes hangs on Vybrid, so revert it so that Vybrid systems
could boot again.

Reported-by: Krzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: Fabio Estevam <festevam@gmail.com>
Tested-by: Krzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: support little endian for edma driver

Our platforms with below registers(CHCFG0 - CHCFG15) of eDMA
*-----------------------------------------------------------*
| Offset | Big endian Register| Little endian Register|
|--------------|--------------------|-----------------------|
| 0x0 | CHCFG0 | CHCFG3 |
|--------------|--------------------|-----------------------|
| 0x1 | CHCFG1 | CHCFG2 |
|--------------|--------------------|-----------------------|
| 0x2 | CHCFG2 | CHCFG1 |
|--------------|--------------------|-----------------------|
| 0x3 | CHCFG3 | CHCFG0 |
|--------------|--------------------|-----------------------|
| ... | ...... | ...... |
|--------------|--------------------|-----------------------|
| 0xC | CHCFG12 | CHCFG15 |
|--------------|--------------------|-----------------------|
| 0xD | CHCFG13 | CHCFG14 |
|--------------|--------------------|-----------------------|
| 0xE | CHCFG14 | CHCFG13 |
|--------------|--------------------|-----------------------|
| 0xF | CHCFG15 | CHCFG12 |
*-----------------------------------------------------------*

Current eDMA driver does not support Little endian, so this
patch is to improve edma driver to support little endian.

Signed-off-by: Peng Ma <peng.ma@nxp.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: dma map slave device address

This mapping needs to be created in order for slave dma transfers
to work on systems with SMMU. The implementation mostly mimics the
one in pl330 dma driver, authored by Robin Murphy.

Signed-off-by: Laurentiu Tudor <laurentiu.tudor@nxp.com>
Suggested-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Angelo Dureghello <angelo@sysam.it>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: use struct_size() in kzalloc()

One of the more common cases of allocation size calculations is finding the
size of a structure that has a zero-sized array at the end, along with memory
for some number of elements for that array. For example:

struct foo {
int stuff;
void *entry[];
};

instance = kzalloc(sizeof(struct foo) + sizeof(void *) * count, GFP_KERNEL);

Instead of leaving these open-coded and prone to type mistakes, we can now
use the new struct_size() helper:

instance = kzalloc(struct_size(instance, entry, count), GFP_KERNEL);

This code was detected with the help of Coccinelle.

Signed-off-by: Gustavo A. R. Silva <gustavo@embeddedor.com>
Tested-by: Angelo Dureghello <angelo@sysam.it>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: remove dma_slave_config direction usage

dma_slave_config direction was marked as deprecated quite some
time back, remove the usage from this driver so that the field
can be removed

Tested-by: Krzysztof Kozlowski <krzk@kernel.org>
Tested-by: Angelo Dureghello <angelo@sysam.it>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: add ColdFire mcf5441x edma support

This patch adds support for ColdFire mcf5441x-family edma
module.

The ColdFire edma module is slightly different from fsl-edma,
so a new driver is added. But most of the code is common
between fsl-edma and mcf-edma so it has been collected into a
separate common module fsl-edma-common (patch 1/3).

Signed-off-by: Angelo Dureghello <angelo@sysam.it>
Tested-by: Krzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: add edma version and configurable registers

This patch adds configurable registers (using __iomem addresses)
to allow the use of fsl-edma-common code with slightly different
edma module versions, as Vybrid (v1) and ColdFire (v2) are.

Signed-off-by: Angelo Dureghello <angelo@sysam.it>
Tested-by: Krzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: fsl-edma: extract common fsl-edma code (no changes in behavior intended)

This patch adds a new fsl-edma-common module to allow new
mcf-edma module code to use most of the fsl-edma code.

Signed-off-by: Angelo Dureghello <angelo@sysam.it>
Tested-by: Krzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: Vinod Koul <vkoul@kernel.org>