dmaengine: imx-sdma: fix Excess kernel-doc warnings

Fix warnings of "Excess struct member" by removing those lines.
They are extraneous.

imx-sdma.c:467: warning: Excess struct member 'context_loaded' description in 'sdma_channel'
imx-sdma.c:467: warning: Excess struct member 'bd_pool' description in 'sdma_channel'
imx-sdma.c:500: warning: Excess struct member 'script_addrs' description in 'sdma_firmware_header'

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Shawn Guo <shawnguo@kernel.org>
Cc: Vinod Koul <vkoul@kernel.org>
Cc: dmaengine@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Link: https://lore.kernel.org/r/20240119032832.4051-1-rdunlap@infradead.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Convert to platform remove callback returning void

The .remove() callback for a platform driver returns an int which makes
many driver authors wrongly assume it's possible to do error handling by
returning an error code. However the value returned is ignored (apart
from emitting a warning) and this typically results in resource leaks.
To improve here there is a quest to make the remove callback return
void. In the first step of this quest all drivers are converted to
.remove_new() which already returns void. Eventually after all drivers
are converted, .remove_new() is renamed to .remove().

Trivially convert this driver from always returning zero in the remove
callback to the void returning variant.

Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Link: https://lore.kernel.org/r/20230919133207.1400430-20-u.kleine-koenig@pengutronix.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: Explicitly include correct DT includes

The DT of_device.h and of_platform.h date back to the separate
of_platform_bus_type before it as merged into the regular platform bus.
As part of that merge prepping Arm DT support 13 years ago, they
"temporarily" include each other. They also include platform_device.h
and of.h. As a result, there's a pretty much random mix of those include
files used throughout the tree. In order to detangle these headers and
replace the implicit includes with struct declarations, users need to
explicitly include the correct includes.

Signed-off-by: Rob Herring <robh@kernel.org>
Link: https://lore.kernel.org/r/20230718143138.1066177-1-robh@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Set DMA channel to be private

If async-tx is loaded before device drivers that requires imx-sdma, the
dmaengine_get() routine from async-tx grabs all non-private channels,
so devices that require DMA fail to work.

So mark imx-sdma with DMA_PRIVATE to avoid such situation.

Signed-off-by: Kai-Heng Feng <kai.heng.feng@canonical.com>
Link: https://lore.kernel.org/r/20230207045745.1029959-1-kai.heng.feng@canonical.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: drivers: Use devm_platform_ioremap_resource()

platform_get_resource() and devm_ioremap_resource() are wrapped up in the
devm_platform_ioremap_resource() helper. Use the helper and get rid of the
local variable for struct resource *. We now have a function call less.

Signed-off-by: Tudor Ambarus <tudor.ambarus@microchip.com>
Acked-by: Jernej Skrabec <jernej.skrabec@gmail.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20221110152528.7821-1-tudor.ambarus@microchip.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: support hdmi in sdma

The hdmi script already supported in sdma firmware. So add support hdmi
in sdma driver.

The design of hdmi script is different from common script such as sai.
There is no need to config buffer descriptor for HDMI. The cyclic
capability is achieved by the hdmi script. The slave config is so simple,
only config src_addr, dts_addr and direction DMA_TRANS_NONE.

Signed-off-by: Joy Zou <joy.zou@nxp.com>
Reviewed-by: Sascha Hauer <s.hauer@pengutronix.de>
Link: https://lore.kernel.org/r/20221115093823.2879128-3-joy.zou@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Fix a possible memory leak in sdma_transfer_init

If the function sdma_load_context() fails, the sdma_desc will be
freed, but the allocated desc->bd is forgot to be freed.

We already met the sdma_load_context() failure case and the log as
below:
[ 450.699064] imx-sdma 30bd0000.dma-controller: Timeout waiting for CH0 ready
...

In this case, the desc->bd will not be freed without this change.

Signed-off-by: Hui Wang <hui.wang@canonical.com>
Reviewed-by: Sascha Hauer <s.hauer@pengutronix.de>
Link: https://lore.kernel.org/r/20221130090800.102035-1-hui.wang@canonical.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Add FIFO stride support for multi FIFO script

The peripheral may have several FIFOs, but some case just select
some FIFOs from them for data transfer, which means FIFO0 and FIFO2
may be selected. So add FIFO address stride support, 0 means all FIFOs
are continuous, 1 means 1 word stride between FIFOs. All stride between
FIFOs should be same.

Another option words_per_fifo means how many audio channel data copied
to one FIFO one time, 1 means one channel per FIFO, 2 means 2 channels
per FIFO.

If 'n_fifos_src = 4' and 'words_per_fifo = 2', it means the first two
words(channels) fetch from FIFO0 and then jump to FIFO1 for next two words,
and so on after the last FIFO3 fetched, roll back to FIFO0.

Signed-off-by: Joy Zou <joy.zou@nxp.com>
Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
Link: https://lore.kernel.org/r/1657162829-9273-1-git-send-email-shengjiu.wang@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Add missing struct documentation

Fix compile warning that 'Function parameter or member not described'
with 'W=1' option:

Add missing description for struct sdma_desc

There is not any description for struct sdma_script_start_addrs,
so use /* instead of /**

Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
Link: https://lore.kernel.org/r/1657086309-7964-1-git-send-email-shengjiu.wang@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Improve the SDMA irq name

On SoCs with several SDMA instances, such as i.MX8M for example,
all the SDMA related interrupts appear with the same "sdma" name.

Improve the SDMA irq name by associating it with the SDMA instance
via dev_name(), so that the SDMA irq names can be unique.

Signed-off-by: Fabio Estevam <festevam@denx.de>
Link: https://lore.kernel.org/r/20220623123353.2570410-1-festevam@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: only restart cyclic channel when enabled

An interrupt for a channel might be pending even after struct
dma_device::device_terminate_all has been called. In that case the
recently introduced warning message "restart cyclic channel..." triggers
and the channel will be restarted. This is not desired as the channel
has just been stopped. Only restart the channel when we still have a
descriptor set for it (which will be set to NULL in
sdma_terminate_all()).

Fixes: 5b215c28b9235 ("dmaengine: imx-sdma: restart cyclic channel if needed")
Cc: stable@vger.kernel.org
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Link: https://lore.kernel.org/r/20220617115042.4004062-1-s.hauer@pengutronix.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Allow imx8m for imx7 FW revs

The revision of the imx-sdma IP that is in the i.MX8M series is the
same is that as that in the i.MX7 series but the imx7d MODULE_FIRMWARE
directive is wrapped in a condiditional which means it's not defined
when built for aarch64 SOC_IMX8M platforms and hence you get the
following errors when the driver loads on imx8m devices:

imx-sdma 302c0000.dma-controller: Direct firmware load for imx/sdma/sdma-imx7d.bin failed with error -2
imx-sdma 302c0000.dma-controller: external firmware not found, using ROM firmware

Add the SOC_IMX8M into the check so the firmware can load on i.MX8.

Fixes: 1474d48bd639 ("arm64: dts: imx8mq: Add SDMA nodes")
Fixes: 941acd566b18 ("dmaengine: imx-sdma: Only check ratio on parts that support 1:1")
Signed-off-by: Peter Robinson <pbrobinson@gmail.com>
Cc: stable@vger.kernel.org # v5.2+
Reviewed-by: Fabio Estevam <festevam@gmail.com>
Link: https://lore.kernel.org/r/20220606161034.3544803-1-pbrobinson@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Add multi fifo support

The i.MX SDMA engine can read from / write to multiple successive
hardware FIFO registers, referred to as "Multi FIFO support". This is
needed for the micfil driver and certain configurations of the SAI
driver. This patch adds support for this feature.

The number of FIFOs to read from / write to must be communicated from
the client driver to the SDMA engine. For this the struct
dma_slave_config::peripheral_config field is used.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Acked-By: Vinod Koul <vkoul@kernel.org>
Link: https://lore.kernel.org/r/20220414162249.3934543-12-s.hauer@pengutronix.de
Signed-off-by: Mark Brown <broonie@kernel.org>

dmaengine: imx-sdma: error out on unsupported transfer types

The i.MX SDMA driver currently silently ignores unsupported transfer
types. These transfer types are specified in the dma channel description
in the device tree, so they should really be checked.
Issue a message and error out when we hit unsupported transfer types.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Acked-By: Vinod Koul <vkoul@kernel.org>
Link: https://lore.kernel.org/r/20220414162249.3934543-11-s.hauer@pengutronix.de
Signed-off-by: Mark Brown <broonie@kernel.org>

dmaengine: imx: Move header to include/dma/

The i.MX DMA drivers are device tree only, nothing in
include/linux/platform_data/dma-imx.h has platform_data in it, so move
the file to include/linux/dma/imx-dma.h.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Acked-By: Vinod Koul <vkoul@kernel.org>
Link: https://lore.kernel.org/r/20220414162249.3934543-10-s.hauer@pengutronix.de
Signed-off-by: Mark Brown <broonie@kernel.org>

dmaengine: imx-sdma: Fix error checking in sdma_event_remap

of_parse_phandle() returns NULL on errors, rather than error
pointers. Using NULL check on grp_np to fix this.

Fixes: d078cd1b4185 ("dmaengine: imx-sdma: Add imx6sx platform support")
Signed-off-by: Miaoqian Lin <linmq006@gmail.com>
Link: https://lore.kernel.org/r/20220308064952.15743-1-linmq006@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: fix init of uart scripts

Commit b98ce2f4e32b ("dmaengine: imx-sdma: add uart rom script") broke
uart rx on imx5 when using sdma firmware from older Freescale 2.6.35
kernel. In this case reading addr->uartXX_2_mcu_addr was going out of
bounds of the firmware memory and corrupting the uart script addresses.

Simply adding a bounds check before accessing addr->uartXX_2_mcu_addr
does not work as the uartXX_2_mcu_addr members are now beyond the size
of the older firmware and the uart addresses would never be populated
in that case. There are other ways to fix this but overall the logic
seems clearer to me to revert the uartXX_2_mcu_ram_addr structure
entries back to uartXX_2_mcu_addr, change the newer entries to
uartXX_2_mcu_rom_addr and update the logic accordingly.

I have tested this patch on:
1. An i.MX53 system with sdma firmware from Freescale 2.6.35 kernel.
Without this patch uart rx is broken in this scenario, with the
patch uart rx is restored.
2. An i.MX6D system with no external sdma firmware. uart is okay with
or without this patch.
3. An i.MX8MM system using current sdma-imx7d.bin firmware from
linux-firmware. uart is okay with or without this patch and I
confirmed the rom version of the uart script is being used which was
the intention and reason for commit b98ce2f4e32b ("dmaengine:
imx-sdma: add uart rom script") in the first place.

Fixes: b98ce2f4e32b ("dmaengine: imx-sdma: add uart rom script")
Cc: stable@vger.kernel.org
Signed-off-by: Kevin Groeneveld <kgroeneveld@lenbrook.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Reviewed-by: Fabio Estevam <festevam@gmail.com>
Acked-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
Link: https://lore.kernel.org/r/20220410223118.15086-1-kgroeneveld@lenbrook.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: clean up some inconsistent indenting

Eliminate the following coccicheck warning:
./drivers/dma/imx-sdma.c:896:3-16: code aligned with following code on
line 897

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Signed-off-by: Yang Li <yang.lee@linux.alibaba.com>
Link: https://lore.kernel.org/r/20220217011604.123106-1-yang.lee@linux.alibaba.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: fix cyclic buffer race condition

Assign buffer ownership to SDMA after invoking descriptor callback to
make sure that SDMA does not write to the buffer before it is read by
the CPU.

Signed-off-by: Tomasz Moń <tomasz.mon@camlingroup.com>
Link: https://lore.kernel.org/r/20220117091955.1038937-2-tomasz.mon@camlingroup.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: restart cyclic channel if needed

Under heavy load resulting in high interrupt latencies, it is possible
for imx UART requests to completely fill DMA buffer. When DMA channel
is triggered and no SDMA owned buffer is available, SDMA stops. Thanks
to the autoRTS feature, there is no data loss due to the SDMA stop if
the UART is using hardware flow control.

According to DMA Engine API Guide, DMA cyclic operation is performed
until explicitly stopped. Restart the buffer after handling channel loop
if the channel was stopped by SDMA.

Signed-off-by: Tomasz Moń <tomasz.mon@camlingroup.com>
Link: https://lore.kernel.org/r/20220117091955.1038937-1-tomasz.mon@camlingroup.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: remove space after sizeof

Space prohibited between function name and
open parenthesis '('

Signed-off-by: Flavio Suligoi <f.suligoi@asem.it>
Link: https://lore.kernel.org/r/20210928151833.589843-4-f.suligoi@asem.it
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: align statement to open parenthesis

Alignment should match open parenthesis.

Signed-off-by: Flavio Suligoi <f.suligoi@asem.it>
Link: https://lore.kernel.org/r/20210928151833.589843-3-f.suligoi@asem.it
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: add missed braces

The "if" conditional statement is not a single statement,
so both branches require braces.

Signed-off-by: Flavio Suligoi <f.suligoi@asem.it>
Link: https://lore.kernel.org/r/20210928151833.589843-2-f.suligoi@asem.it
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: remove useless braces

Braces {} are not necessary for single statement blocks.

Signed-off-by: Flavio Suligoi <f.suligoi@asem.it>
Link: https://lore.kernel.org/r/20210928151833.589843-1-f.suligoi@asem.it
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: add terminated list for freed descriptor in worker

Add terminated list for keeping descriptor so that it could be freed in
worker without any potential involving next descriptor raised up before
this descriptor freed, because vchan_get_all_descriptors get all
descriptors including the last terminated descriptor and the next
descriptor, hence, the next descriptor maybe freed unexpectly when it's
done in worker without this patch.
https://www.spinics.net/lists/dmaengine/msg23367.html

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Reported-by: Richard Leitner <richard.leitner@skidata.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Shawn Guo <shawnguo@kernel.org>

dmaengine: imx-sdma: add uart rom script

For the compatibility of NXP internal legacy kernel before 4.19 which
is based on uart ram script and upstreaming kernel based on uart rom
script, add both uart ram/rom script in latest sdma firmware. By default
uart rom script used.
Besides, add two multi-fifo scripts for SAI/PDM on i.mx8m/8mm and add
back qspi script miss for v4(i.mx7d/8m/8mm family, but v3 is for i.mx6).

rom script:
uart_2_mcu_addr
uartsh_2_mcu_addr /* through spba bus */
am script:
uart_2_mcu_ram_addr
uartsh_2_mcu_ram_addr /* through spba bus */

Please get latest sdma firmware from the below and put them into the path
(/lib/firmware/imx/sdma/):
https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
/tree/imx/sdma

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Acked-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Shawn Guo <shawnguo@kernel.org>

dmaengine: imx-sdma: remove ERR009165 on i.mx6ul

ECSPI issue fixed from i.mx6ul at hardware level, no need
ERR009165 anymore on those chips such as i.mx8mq.

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Acked-by: Vinod Koul <vkoul@kernel.org>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Shawn Guo <shawnguo@kernel.org>

dmaengine: imx-sdma: add mcu_2_ecspi script

Add mcu_2_ecspi script to fix ecspi errata ERR009165.

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Acked-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Shawn Guo <shawnguo@kernel.org>

dmaengine: dma: imx-sdma: add fw_loaded and is_ram_script

Add 'fw_loaded' and 'is_ram_script' to check if the script used by channel
is ram script and it's loaded or not, so that could prevent meaningless
following malloc dma descriptor and bd allocate in sdma_transfer_init(),
otherwise memory may be consumed out potentially without free in case
that spi fallback into pio while dma transfer failed by sdma firmware not
ready(next ERR009165 patch depends on sdma RAM scripts/firmware).

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Acked-by: Vinod Koul <vkoul@kernel.org>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Shawn Guo <shawnguo@kernel.org>

dmaengine: imx-sdma: remove duplicated sdma_load_context

Since sdma_transfer_init() will do sdma_load_context before any
sdma transfer, no need once more in sdma_config_channel().

Fixes: ad0d92d7ba6a ("dmaengine: imx-sdma: refine to load context only once")
Cc: <stable@vger.kernel.org>
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Acked-by: Vinod Koul <vkoul@kernel.org>
Tested-by: Richard Leitner <richard.leitner@skidata.com>
Signed-off-by: Shawn Guo <shawnguo@kernel.org>

Revert "dmaengine: imx-sdma: refine to load context only once"

This reverts commit ad0d92d7ba6aecbe2705907c38ff8d8be4da1e9c, because
in spi-imx case, burst length may be changed dynamically.

Fixes: ad0d92d7ba6a ("dmaengine: imx-sdma: refine to load context only once")
Cc: <stable@vger.kernel.org>
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Tested-by: Richard Leitner <richard.leitner@skidata.com>
Signed-off-by: Shawn Guo <shawnguo@kernel.org>

dmaengine: imx-sdma: Remove platform data header

Since commit 6c5f05a6cd88 ("ARM: imx3: Remove imx3 soc_init()")
there are no more users of struct sdma_script_start_addrs outside
of the driver itself, thus let's move the struct declaration just
to the driver source code and remove the header file as unused one.

Signed-off-by: Vladimir Zapolskiy <vz@mleia.com>
Cc: Fabio Estevam <festevam@gmail.com>
Cc: Shawn Guo <shawnguo@kernel.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Fabio Estevam <festevam@gmail.com>
Link: https://lore.kernel.org/r/20210620191103.156626-1-vz@mleia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

firmware: replace HOTPLUG with UEVENT in FW_ACTION defines

With commit 312c004d36ce ("[PATCH] driver core: replace "hotplug" by
"uevent"") already in the tree over a decade, update the name of
FW_ACTION defines to follow semantics, and reflect what the defines are
really meant for, i.e. whether or not generate user space event.

Acked-by: Lee Jones <lee.jones@linaro.org>
Signed-off-by: Shawn Guo <shawn.guo@linaro.org>
Link: https://lore.kernel.org/r/20210425020024.28057-1-shawn.guo@linaro.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

dmaengine: imx-sdma: Use of_device_get_match_data()

Use of_device_get_match_data() to make the code simpler.

Signed-off-by: Fabio Estevam <festevam@gmail.com>
Link: https://lore.kernel.org/r/20210118121549.1625217-2-festevam@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Remove platform data support

Since 5.10-rc1, i.MX has been converted to a devicetree-only platform.

The platform data support in this driver was only used for non-DT
platforms.

Remove the platform data support as it has no more users.

Signed-off-by: Fabio Estevam <festevam@gmail.com>
Link: https://lore.kernel.org/r/20210118121549.1625217-1-festevam@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Remove unused .id_table support

Since 5.10-rc1 i.MX is a devicetree-only platform and the existing
.id_table support in this driver was only useful for old non-devicetree
platforms.

Signed-off-by: Fabio Estevam <festevam@gmail.com>
Link: https://lore.kernel.org/r/20201116202403.29749-1-festevam@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Drop local dma_parms

Since commit 9495b7e92f71 ("driver core: platform: Initialize dma_parms
for platform devices"), struct platform_device already provides a
dma_parms structure, so we can save allocating another one.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Link: https://lore.kernel.org/r/d9b551dcf712a91860af3c5dd01a31b9b97ac1c5.1599164692.git.robin.murphy@arm.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Correct formatting issue and provide 2 new descriptions

Fixes the following W=1 kernel build warning(s):

drivers/dma/imx-sdma.c:383: warning: Function parameter or member 'slave_config' not described in 'sdma_channel'
drivers/dma/imx-sdma.c:383: warning: Function parameter or member 'context_loaded' not described in 'sdma_channel'
drivers/dma/imx-sdma.c:383: warning: Function parameter or member 'terminate_worker' not described in 'sdma_channel'

Signed-off-by: Lee Jones <lee.jones@linaro.org>
Cc: Shawn Guo <shawnguo@kernel.org>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
Cc: Fabio Estevam <festevam@gmail.com>
Cc: NXP Linux Team <linux-imx@nxp.com>
Link: https://lore.kernel.org/r/20200714111546.1755231-12-lee.jones@linaro.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Fix: Remove 'always true' comparison

event_id0 is defined as 'unsigned int', so it is always greater or
equal to zero.

Remove the unneeded comparisons to fix the following W=1 build
warning:

drivers/dma/imx-sdma.c: In function 'sdma_free_chan_resources':
drivers/dma/imx-sdma.c:1334:23: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
1334 | if (sdmac->event_id0 >= 0)
| ^~
drivers/dma/imx-sdma.c: In function 'sdma_config':
drivers/dma/imx-sdma.c:1635:23: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits]
1635 | if (sdmac->event_id0 >= 0) {
|

Fixes: 25962e1a7f1d ("dmaengine: imx-sdma: Fix the event id check to include RX event for UART6")
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Fabio Estevam <festevam@gmail.com>
Reviewed-by: Frieder Schrempf <frieder.schrempf@kontron.de>
Link: https://lore.kernel.org/r/20200621155730.28766-1-festevam@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: initialize all script addresses

The script addresses array increases with each new version. The driver
initializes the array to -EINVAL initially, but only up to the size
of the v1 array. Initialize the additional addresses for the newer
versions as well. Without this uninitialized values of the newer arrays
are treated as valid.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Reviewed-by: Robin Gong <yibin.gong@nxp.com>
Link: https://lore.kernel.org/r/20200513060405.18685-1-s.hauer@pengutronix.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Fix the event id check to include RX event for UART6

On i.MX6UL/ULL and i.MX6SX the DMA event id for the RX channel of
UART6 is '0'. To fix the broken DMA support for UART6, we change
the check for event_id0 to include '0' as a valid id.

Fixes: 1ec1e82f2510 ("dmaengine: Add Freescale i.MX SDMA support")
Signed-off-by: Frieder Schrempf <frieder.schrempf@kontron.de>
Reviewed-by: Fabio Estevam <festevam@gmail.com>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/20200225082139.7646-1-frieder.schrempf@kontron.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: fix context cache

There is a DMA problem with the serial ports on i.MX6.

When the following sequence is performed:

1) Open a port
2) Write some data
3) Close the port
4) Open a *different* port
5) Write some data
6) Close the port

The second write sends nothing and the second close hangs.
If the first close() is omitted it works.

Adding logs to the the UART driver shows that the DMA is being setup but
the callback is never invoked for the second write.

This used to work in 4.19.

Git bisect leads to:
ad0d92d: "dmaengine: imx-sdma: refine to load context only once"

This commit adds a "context_loaded" flag used to avoid unnecessary context
setups.
However the flag is only reset in sdma_channel_terminate_work(),
which is only invoked in a worker triggered by sdma_terminate_all() IF
there is an active descriptor.

So, if no active descriptor remains when the channel is terminated, the
flag is not reset and, when the channel is later reused the old context
is used.

Fix the problem by always resetting the flag in sdma_free_chan_resources().

Cc: stable@vger.kernel.org
Signed-off-by: Martin Fuzzey <martin.fuzzey@flowbird.group>
Fixes: ad0d92d7ba6a ("dmaengine: imx-sdma: refine to load context only once")
Reviewed-by: Fabio Estevam <festevam@gmail.com>
Link: https://lore.kernel.org/r/1580305274-27274-1-git-send-email-martin.fuzzey@flowbird.group
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Fix memory leak

The current descriptor is not on any list of the virtual DMA channel.
Once sdma_terminate_all() is called when a descriptor is currently
in flight then this one is forgotten to be freed. We have to call
vchan_terminate_vdesc() on this descriptor to re-add it to the lists.
Now that we also free the currently running descriptor we can (and
actually have to) remove the current descriptor from its list also
for the cyclic case.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Reviewed-by: Robin Gong <yibin.gong@nxp.com>
Tested-by: Robin Gong <yibin.gong@nxp.com>
Link: https://lore.kernel.org/r/20191216105328.15198-10-s.hauer@pengutronix.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: find desc first in sdma_tx_status

In sdma_tx_status() we must first find the current sdma_desc. In cyclic
mode we assume that this can always be found with vchan_find_desc().
This is true because do not remove the current descriptor from the
desc_issued list:

/*
* Do not delete the node in desc_issued list in cyclic mode, otherwise
* the desc allocated will never be freed in vchan_dma_desc_free_list
*/
if (!(sdmac->flags & IMX_DMA_SG_LOOP))
list_del(&vd->node);

We will change this in the next step, so check if the current descriptor is
the desired one also for the cyclic case.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Link: https://lore.kernel.org/r/20191216105328.15198-9-s.hauer@pengutronix.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: rename function

Rename sdma_disable_channel_async() after the hook it implements, like
done for all other functions in the SDMA driver.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Link: https://lore.kernel.org/r/20191216105328.15198-8-s.hauer@pengutronix.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: fix size check for sdma script_number

Illegal memory will be touch if SDMA_SCRIPT_ADDRS_ARRAY_SIZE_V3
(41) exceed the size of structure sdma_script_start_addrs(40),
thus cause memory corrupt such as slob block header so that kernel
trap into while() loop forever in slob_free(). Please refer to below
code piece in imx-sdma.c:
for (i = 0; i < sdma->script_number; i++)
if (addr_arr[i] > 0)
saddr_arr[i] = addr_arr[i]; /* memory corrupt here */
That issue was brought by commit a572460be9cf ("dmaengine: imx-sdma: Add
support for version 3 firmware") because SDMA_SCRIPT_ADDRS_ARRAY_SIZE_V3
(38->41 3 scripts added) not align with script number added in
sdma_script_start_addrs(2 scripts).

Fixes: a572460be9cf ("dmaengine: imx-sdma: Add support for version 3 firmware")
Cc: stable@vger.kernel
Link: https://www.spinics.net/lists/arm-kernel/msg754895.html
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Reported-by: Jurgen Lambrecht <J.Lambrecht@TELEVIC.com>
Link: https://lore.kernel.org/r/1569347584-3478-1-git-send-email-yibin.gong@nxp.com
[vkoul: update the patch title]
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Remove call to memset after dma_alloc_coherent

In commit 518a2f1925c3
("dma-mapping: zero memory returned from dma_alloc_*"),
dma_alloc_coherent has already zeroed the memory.
So memset is not needed.

Signed-off-by: Fuqian Huang <huangfq.daxian@gmail.com>
Link: https://lore.kernel.org/r/20190715031716.6328-1-huangfq.daxian@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: remove BD_INTR for channel0

It is possible for an irq triggered by channel0 to be received later
after clks are disabled once firmware loaded during sdma probe. If
that happens then clearing them by writing to SDMA_H_INTR won't work
and the kernel will hang processing infinite interrupts. Actually,
don't need interrupt triggered on channel0 since it's pollling
SDMA_H_STATSTOP to know channel0 done rather than interrupt in
current code, just clear BD_INTR to disable channel0 interrupt to
avoid the above case.
This issue was brought by commit 1d069bfa3c78 ("dmaengine: imx-sdma:
ack channel 0 IRQ in the interrupt handler") which didn't take care
the above case.

Fixes: 1d069bfa3c78 ("dmaengine: imx-sdma: ack channel 0 IRQ in the interrupt handler")
Cc: stable@vger.kernel.org #5.0+
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Reported-by: Sven Van Asbroeck <thesven73@gmail.com>
Tested-by: Sven Van Asbroeck <thesven73@gmail.com>
Reviewed-by: Michael Olbrich <m.olbrich@pengutronix.de>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: fix use-after-free on probe error path

If probe() fails anywhere beyond the point where
sdma_get_firmware() is called, then a kernel oops may occur.

Problematic sequence of events:
1. probe() calls sdma_get_firmware(), which schedules the
firmware callback to run when firmware becomes available,
using the sdma instance structure as the context
2. probe() encounters an error, which deallocates the
sdma instance structure
3. firmware becomes available, firmware callback is
called with deallocated sdma instance structure
4. use after free - kernel oops !

Solution: only attempt to load firmware when we're certain
that probe() will succeed. This guarantees that the firmware
callback's context will remain valid.

Note that the remove() path is unaffected by this issue: the
firmware loader will increment the driver module's use count,
ensuring that the module cannot be unloaded while the
firmware callback is pending or running.

Signed-off-by: Sven Van Asbroeck <TheSven73@gmail.com>
Reviewed-by: Robin Gong <yibin.gong@nxp.com>
[vkoul: fixed braces for if condition]
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Let the core do the device node validation

Let the DMA engine core do the device node validation instead of drivers.

Signed-off-by: Baolin Wang <baolin.wang@linaro.org>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Only check ratio on parts that support 1:1

On imx8mq B0 chip, AHB/SDMA clock ratio 2:1 can't be supported,
since SDMA clock ratio has to be increased to 250Mhz, AHB can't reach
to 500Mhz, so use 1:1 instead.

To limit this change to the imx8mq for now this patch also adds an
im8mq-sdma compatible string.

Signed-off-by: Angus Ainslie (Purism) <angus@akkea.ca>
Acked-by: Robin Gong <yibin.gong@nxp.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: fix consistent dma test failures

Without the copy being aligned sdma1 fails ~10% of the time

Signed-off-by: Angus Ainslie (Purism) <angus@akkea.ca>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: add a test for imx8mq multi sdma devices

On i.mx8mq, there are two sdma instances, and the common dma framework
will get a channel dynamically from any available sdma instance whether
it's the first sdma device or the second sdma device. Some IPs like
SAI only work with sdma2 not sdma1. To make sure the sdma channel is from
the correct sdma device, use the node pointer to match.

Signed-off-by: Angus Ainslie (Purism) <angus@akkea.ca>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Tested-by: Daniel Baluta <daniel.baluta@nxp.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: add clock ratio 1:1 check

On i.mx8 mscale B0 chip, AHB/SDMA clock ratio 2:1 can't be supportted,
since SDMA clock ratio has to be increased to 250Mhz, AHB can't reach
to 500Mhz, so use 1:1 instead.

Based on NXP commit MLK-16841-1 by Robin Gong <yibin.gong@nxp.com>

Signed-off-by: Angus Ainslie (Purism) <angus@akkea.ca>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: pass ->dev to dma_alloc_coherent() API

Pass ->dev to dma_alloc_coherent() API. We need this
because dma_alloc_coherent() makes use of dev parameter
and receiving NULL will result in a crash.

Signed-off-by: Andy Duan <fugang.duan@nxp.com>
Signed-off-by: Daniel Baluta <daniel.baluta@nxp.com>
Reviewed-by: Robin Gong <yibin.gong@nxp.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: refine to load context only once

The context loaded only one time before channel running,but
currently sdma_config_channel() and dma_prep_* duplicated with
sdma_load_context(), so refine it to load context only one time
before channel running and reload after the channel terminated.

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

cross-tree: phase out dma_zalloc_coherent()

We already need to zero out memory for dma_alloc_coherent(), as such
using dma_zalloc_coherent() is superflous. Phase it out.

This change was generated with the following Coccinelle SmPL patch:

@ replace_dma_zalloc_coherent @
expression dev, size, data, handle, flags;
@@

-dma_zalloc_coherent(dev, size, handle, flags)
+dma_alloc_coherent(dev, size, handle, flags)

Suggested-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
[hch: re-ran the script on the latest tree]
Signed-off-by: Christoph Hellwig <hch@lst.de>

dmaengine: imx-sdma: use GFP_NOWAIT for dma descriptor allocations

DMA buffer descriptors aren't allocated from atomic context, so they
can use the less heavyweigth GFP_NOWAIT.

Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: implement channel termination via worker

The dmaengine documentation states that device_terminate_all may be
asynchronous and need not wait for the active transfers to stop.

This allows us to move most of the functionality currently implemented
in the sdma channel termination function to run in a worker, outside
of any atomic context. Moving this out of atomic context has two
benefits: we can now sleep while waiting for the channel to terminate,
instead of busy waiting and the freeing of the dma descriptors happens
with IRQs enabled, getting rid of a warning in the dma mapping code.

As the termination is now async, we need to implement the
device_synchronize dma engine function which simply waits for the
worker to finish its execution.

Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

Revert "dmaengine: imx-sdma: alloclate bd memory from dma pool"

This reverts commit fe5b85c656bc. The SDMA engine needs the descriptors to
be contiguous in memory. As the dma pool API is only able to provide a
single descriptor per alloc invocation there is no guarantee that multiple
descriptors satisfy this requirement. Also the code in question is broken
as it only allocates memory for a single descriptor, without looking at the
number of descriptors required for the transfer, leading to out-of-bounds
accesses when the descriptors are written.

Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

Revert "dmaengine: imx-sdma: Use GFP_NOWAIT for dma allocations"

This reverts commit c1199875d327, as this depends on another commit
that is going to be reverted.

Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: remove dma_slave_config direction usage and leave sdma_event_enable()

dma_slave_config direction was marked as deprecated quite some
time back, remove the usage from this driver so that the field
can be removed

ENBLn bit should be set before any dma request triggered, please
refer to the below information from i.mx6sololite RM. Otherwise,
spi/uart test will be fail because there is dma request from tx
fifo always before dmaengine_prep_slave_sg() in where ENBLn set
and violate the below rule.

https://www.nxp.com/docs/en/reference-manual/IMX6SLRM.pdf:

40.8.28 Channel Enable RAM (SDMAARM_CHNENBLn)
"It is thus essential for the Arm platform to program them before
any DMA request is triggered to the SDMA, otherwise an unpredictable
combination of channels may be started".

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
[vkoul: sqashed patch from Robin into direction change]
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Use a single line for dma_alloc_coherent()

Make the call to dma_alloc_coherent() to fit into a single line, which
helps readability.

Signed-off-by: Fabio Estevam <festevam@gmail.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: add memcpy interface

Add MEMCPY capability for imx-sdma driver.

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: add SDMA_BD_MAX_CNT to replace '0xffff'

Add macro SDMA_BD_MAX_CNT to replace '0xffff'.

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: add missing structure description

Some member description or colons missing cause build warning with
'W=1' as below:

drivers/dma/imx-sdma.c:326: warning: Function parameter or member 'vd' not described in 'sdma_desc'
drivers/dma/imx-sdma.c:326: warning: Function parameter or member 'num_bd' not described in 'sdma_desc'
drivers/dma/imx-sdma.c:326: warning: Function parameter or member 'bd_phys' not described in 'sdma_desc'
drivers/dma/imx-sdma.c:326: warning: Function parameter or member 'buf_tail' not described in 'sdma_desc'
drivers/dma/imx-sdma.c:326: warning: Function parameter or member 'buf_ptail' not described in 'sdma_desc'
drivers/dma/imx-sdma.c:326: warning: Function parameter or member 'period_len' not described in 'sdma_desc'
drivers/dma/imx-sdma.c:326: warning: Function parameter or member 'chn_real_count' not described in 'sdma_desc'
drivers/dma/imx-sdma.c:326: warning: Function parameter or member 'chn_count' not described in 'sdma_desc

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Reported-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: remove unused variable

The variable is no longer used, though it is set, so remove it

drivers/dma/imx-sdma.c:1298:6:
warning: variable ‘ret’ set but not used [-Wunused-but-set-variable]
int ret, i, count;
^~~

Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Use GFP_NOWAIT for dma allocations

The memory allocation in DMA callbacks should use GFP_NOWAIT, so
update this one and fix code alignment for this call while at it.

Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Fix some typos

Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: alloclate bd memory from dma pool

dma_terminate_all maybe called in interrupt context which means
WARN_ON() will be triggered as below when bd memory freed. Allocat
bd memory from dma pool instead.

[ 29.161079] WARNING: CPU: 1 PID: 533 at ./include/linux/dma-mapping.h:541 sdma_free_bd+0xa4/0xb4
[ 29.169883] Modules linked in:
[ 29.172990] CPU: 1 PID: 533 Comm: mpegaudioparse0 Not tainted 4.18.0-rc1-next-20180618-00009-gf79f22c #20
[ 29.182597] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
[ 29.189163] Backtrace:
[ 29.191685] [<c010d1e0>] (dump_backtrace) from [<c010d4a0>] (show_stack+0x18/0x1c)
[ 29.199306] r7:00000000 r6:600f0093 r5:00000000 r4:c107db7c
[ 29.205029] [<c010d488>] (show_stack) from [<c0a5bba0>] (dump_stack+0xb4/0xe8)
[ 29.212312] [<c0a5baec>] (dump_stack) from [<c012703c>] (__warn+0x104/0x130)
[ 29.219411] r9:ec3e817c r8:0000021d r7:00000009 r6:c0d1d440 r5:00000000 r4:00000000
[ 29.227204] [<c0126f38>] (__warn) from [<c0127180>] (warn_slowpath_null+0x44/0x50)
[ 29.234821] r8:ed129dc4 r7:c0b01978 r6:c04d4e90 r5:0000021d r4:c0d1d440
[ 29.241574] [<c012713c>] (warn_slowpath_null) from [<c04d4e90>] (sdma_free_bd+0xa4/0xb4)
[ 29.249706] r6:4c001000 r5:f082e000 r4:00000024
[ 29.254376] [<c04d4dec>] (sdma_free_bd) from [<c04d4eb4>] (sdma_desc_free+0x14/0x20)
[ 29.262163] r7:ec3e8110 r6:00000100 r5:00000200 r4:ecf89a00
[ 29.267873] [<c04d4ea0>] (sdma_desc_free) from [<c04d229c>] (vchan_dma_desc_free_list+0xa4/0xac)
[ 29.276697] r5:00000200 r4:ed129d9c
[ 29.280326] [<c04d21f8>] (vchan_dma_desc_free_list) from [<c04d482c>] (sdma_disable_channel_with_delay+0x14c/0x188)
[ 29.290808] r9:ecae560c r8:ec3e815c r7:00000000 r6:c1008908 r5:ed129dc4 r4:ec3e8110
[ 29.298605] [<c04d46e0>] (sdma_disable_channel_with_delay) from [<c07c5c84>] (snd_dmaengine_pcm_trigger+0x90/0x1b0)
[ 29.309087] r8:ecae5000 r7:ec940800 r6:ed31bd80 r5:ecadb200 r4:ec26a700
[ 29.315855] [<c07c5bf4>] (snd_dmaengine_pcm_trigger) from [<c07dd800>] (soc_pcm_trigger+0xb4/0x130)
[ 29.324953] r8:ecae5000 r7:ec940800 r6:00000000 r5:ecadb200 r4:ec26a700
[ 29.331716] [<c07dd74c>] (soc_pcm_trigger) from [<c07bc008>] (snd_pcm_do_stop+0x58/0x5c)
[ 29.339859] r9:ecaed5a8 r8:ed31bdc0 r7:00000000 r6:00000001 r5:ecadb200 r4:c0b9c4d0
[ 29.347652] [<c07bbfb0>] (snd_pcm_do_stop) from [<c07bbde8>] (snd_pcm_action_single+0x40/0x80)
[ 29.356315] [<c07bbda8>] (snd_pcm_action_single) from [<c07bbf1c>] (snd_pcm_action+0xf4/0xfc)
[ 29.364883] r7:00000001 r6:c0b9c4d0 r5:ecadb2d4 r4:ecadb200
[ 29.370593] [<c07bbe28>] (snd_pcm_action) from [<c07bc8dc>] (snd_pcm_drop+0x58/0x9c)

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Reviewed-by: Sascha Hauer <s.hauer@pengutronix.de>
Tested-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: add sdma_transfer_init to decrease code overlap

There are lot of codes overlap between prep_sg and prep_cyclic function.
Add sdma_transfer_init() function to elimated the code overlap.

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Reviewed-by: Sascha Hauer <s.hauer@pengutronix.de>
Tested-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: remove the maximum limitation for bd numbers

No this limitation now after virtual dma used since bd is allocated
dynamically instead of static.

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Reviewed-by: Sascha Hauer <s.hauer@pengutronix.de>
Tested-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: remove useless 'lock' and 'enabled' in 'struct sdma_channel'

Since 'sdmac->vc.lock' and 'sdmac->desc' can be used as 'lock' and
'enabled' in 'struct sdma_channel sdmac', remove them.

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Reviewed-by: Sascha Hauer <s.hauer@pengutronix.de>
Tested-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: add virt-dma support

The legacy sdma driver has below limitations or drawbacks:
1. Hardcode the max BDs number as "PAGE_SIZE / sizeof(*)", and alloc
one page size for one channel regardless of only few BDs needed
most time. But in few cases, the max PAGE_SIZE maybe not enough.
2. One SDMA channel can't stop immediatley once channel disabled which
means SDMA interrupt may come in after this channel terminated.There
are some patches for this corner case such as commit "2746e2c389f9",
but not cover non-cyclic.

The common virt-dma overcomes the above limitations. It can alloc bd
dynamically and free bd once this tx transfer done. No memory wasted or
maximum limititation here, only depends on how many memory can be requested
from kernel. For No.2, such issue can be workaround by checking if there
is available descript("sdmac->desc") now once the unwanted interrupt
coming. At last the common virt-dma is easier for sdma driver maintain.

Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Reviewed-by: Sascha Hauer <s.hauer@pengutronix.de>
Tested-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: factor out a struct sdma_desc from struct sdma_channel

This is a preparation step to make the adding of virt-dma easier.
We create a struct sdma_desc, move some fields from struct sdma_channel
there and add a pointer from the former to the latter. For now we
allocate the data statically in struct sdma_channel, but with
virt-dma support it will be dynamically allocated.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
Reviewed-by: Sascha Hauer <s.hauer@pengutronix.de>
Tested-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: Switch to SPDX identifier

Adopt the SPDX license identifier headers to ease license compliance
management.

Signed-off-by: Fabio Estevam <fabio.estevam@nxp.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: imx-sdma: fix pagefault when channel is disabled during interrupt

Add a spinlock and a 'enabled' boolean on channel descriptor, to avoid
using buffer descriptors in the interrupt context,
when sdma_disable_channel is called meanwhile.

Signed-off-by: Thierry Bultel <tbultel@pixelsurmer.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Add MODULE_FIRMWARE

This avoid the following error when using an initramfs on wandboard quad
Direct firmware load for imx/sdma/sdma-imx6q.bin failed with error -2

Signed-off-by: Nicolas Chauvet <kwizart@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Correct src_addr_widths and directions

The driver already supports DMA_DEV_TO_DEV in sdma_config(),
DMA_SLAVE_BUSWIDTH_2_BYTES and DMA_SLAVE_BUSWIDTH_1_BYTE in
sdma_prep_slave_sg(). So this patch adds them to the lists.

Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Fix compilation warning.

Replace '%d' by '%zu' to fix the following compilation warning:-

drivers/dma/imx-sdma.c: In function ‘sdma_prep_dma_cyclic’:
drivers/dma/imx-sdma.c:1327:5: warning: format ‘%d’ expects argument of type ‘int’, but argument 4 has type ‘size_t’ [-Wformat=]
channel, period_len, 0xffff);
^
drivers/dma/imx-sdma.c:1350:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 5 has type ‘size_t’ [-Wformat=]
dev_dbg(sdma->dev, "entry %d: count: %d dma: %#llx %s%s\n",

Signed-off-by: Arvind Yadav <arvind.yadav.cs@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Handle return value of clk_prepare_enable

clk_prepare_enable() can fail here and we must check its return value.

Signed-off-by: Arvind Yadav <arvind.yadav.cs@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: add 1ms delay to ensure SDMA channel is stopped

sdma_disable_channel() cannot ensure dma is stopped to access
module's FIFOs. There is chance SDMA core is running and accessing
BD when disable of corresponding channel, this may cause sometimes
even after call of .sdma_disable_channel(), SDMA core still be
running and accessing module's FIFOs.

According to NXP R&D team a delay of one BD SDMA cost time (maximum
is 1ms) should be added after disable of the channel bit, to ensure
SDMA core has really been stopped after SDMA clients call
.device_terminate_all.

This patch introduces adds a new function sdma_disable_channel_with_delay()
which simply adds 1ms delay after call sdma_disable_channel(),
and set it as .device_terminate_all.

Signed-off-by: Jiada Wang <jiada_wang@mentor.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: advertise correct residue granularity

The SDMA hardware/driver does not actually report the transfer residue at
burst size granularity, but in fact is only able to report residue after
each finished segment.

Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Tested-by: Fabio Estevam <fabio.estevam@nxp.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma - correct the dma transfer residue calculation

The residue calculation was taking in consideration that dma
transaction status will be always retrieved in the dma callback
used to inform that dma transfer is complete. However this is not
the case for all subsystems that use dma. Some subsystems use a
timer to check the dma status periodically.

Therefore the calculation was updated and residue is calculated
accordingly by a) update the residue calculation taking in
consideration the last used buffer index by using *buf_ptail* variable
and b) chn_real_count (number of bytes transferred) is initialized to
zero, when dma channel is created, to avoid using an uninitialized
value in residue calculation when dma status is checked without
waiting dma complete event.

Signed-off-by: Nandor Han <nandor.han@ge.com>
Acked-by: Peter Senna Tschudin <peter.senna@collabora.com>
Tested-by: Peter Senna Tschudin <peter.senna@collabora.com>
Tested-by: Marek Vasut <marex@denx.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma - update the residue calculation for cyclic channels

The calculation of the DMA transaction residue supports only fixed
size data transfers. This implementation is not covering all
operations (e.g. data receiving) when we need to know the exact amount
of bytes transferred.

The loop channels handling was changed to clear the buffer
descriptor errors and use the bd->mode.count to calculate the
residue.

Tested-by: Peter Senna Tschudin <peter.senna@collabora.com>
Acked-by: Peter Senna Tschudin <peter.senna@collabora.com>
Reviewed-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Nandor Han <nandor.han@ge.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

dmaengine: imx-sdma - reduce transfer latency for DMA cyclic clients

Having the SDMA driver use a tasklet for running the clients
callback introduce some issues:
- probability to have desynchronized data because of the
race condition created since the DMA transaction status
is retrieved only when the callback is executed, leaving
plenty of time for transaction status to get altered.
- inter-transfer latency which can leave channels idle.

Move the callback execution, for cyclic channels, to SDMA
interrupt (as advised in `Documentation/dmaengine/provider.txt`)
to (a)reduce the inter-transfer latency and (b) eliminate the
race condition possibility where DMA transaction status might
be changed by the time is read.

The responsibility of the SDMA interrupt latency
is moved to the SDMA clients which case by case should defer
the work to bottom-halves when needed.

Tested-by: Peter Senna Tschudin <peter.senna@collabora.com>
Acked-by: Peter Senna Tschudin <peter.senna@collabora.com>
Reviewed-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Nandor Han <nandor.han@ge.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

dmaengine: imx-sdma: Add i.MX7 support

Allow i.MX7 to work with the imx-sdma driver.

Signed-off-by: Fabio Estevam <fabio.estevam@nxp.com>
Tested-by: Stefan Agner <stefan@agner.ch>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: (trivial) fix a typo

Signed-off-by: Martin Kaiser <martin@kaiser.cx>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: convert callback to helper function

This is in preperation of moving to a callback that provides results to the
callback for the transaction. The conversion will maintain current behavior
and the driver must convert to new callback mechanism at a later time in
order to receive results.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: remove assignment never used

David reported:
[drivers/dma/imx-sdma.c:769]: (style) Variable 'emi_2_emi' is assigned a value that is never used

Since emi_2_emi is never used afterwards, remove thsi as well

Reported-by: David Binderman <dcb314@hotmail.com>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Fabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: remove dummy assignment

David reported:
drivers/dma/imx-sdma.c:1003]: (style) Same expression on both sides of '|='

ORing with itself yields same result, So remove this

Reported-by: David Binderman <dcb314@hotmail.com>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Fabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: explicitly freeup irq

dmaengine device should explicitly call devm_free_irq() when using
devm_request_irq().

The irq is still ON when devices remove is executed and irq should be
quiesced before remove is completed.

Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Fabio Estevam <fabio.estevam@freescale.com>

dmaengine: imx-sdma: ack channel 0 IRQ in the interrupt handler

Currently the handler ignores the channel 0 interrupt and thus doesn't ack
it properly. This is done in order to allow sdma_run_channel0() to poll
on the irq status bit, as this function may be called in atomic context,
but needs to know when the channel has finished.

This works mostly, as the polling happens under a spinlock, disabling IRQs
on the local CPU, leaving only a very slight race window for a spurious
IRQ to happen if the handler is executed on another CPU in an SMP system.
Still this is clearly suboptimal.
This behavior turns into a real problem on an RT system, where the spinlock
doesn't disable IRQs on the local CPU. Not acking the IRQ in the handler
in such a setup is very likely to drown the CPU in an IRQ storm, leaving
it unable to make any progress in the polling loop, leading to the IRQ
never being acked.

Fix this by properly acknowledging the channel 0 IRQ in the handler.
As the IRQ status bit can no longer be used to poll for the channel
completion, switch over to using the SDMA_H_STATSTOP register for this
purpose, where bit 0 is cleared by the hardware when the channel is done.

Signed-off-by: Michael Olbrich <m.olbrich@pengutronix.de>
Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: remove __init annotation on sdma_event_remap

The sdma_probe function will call sdma_event_remap, but sdma_event_remap
marked with the __init annotation which make the kbuild complains as the
following log:

WARNING: drivers/dma/built-in.o(.text+0x56fc): Section mismatch in reference
from the function sdma_probe() to the function .init.text:sdma_event_remap()
The function sdma_probe() references
the function __init sdma_event_remap().
This is often because sdma_probe lacks a __init
annotation or the annotation of sdma_event_remap is wrong.

Remove the __init annotation on sdma_event_remap to kill this build warning

Signed-off-by: Jason Liu <r64343@freescale.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Remove unneeded dev_info()

There is no need to print that the driver has been initialized
or removed, so remove such messages.

Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Move message level to debug

Since commit d078cd1b4185 ("dmaengine: imx-sdma: Add imx6sx platform
support") we get this message on every boot on mx6q:

imx-sdma 20ec000.sdma: no event needs to be remapped

, which is not very helpful.

Move the message to debug level instead.

Cc: Zidan Wang <zidan.wang@freescale.com>
Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Check for clk_enable() errors

clk_enable() may fail, so we should better check the return value and
propagate it in the case of error.

Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Add imx6sx platform support

The new Solo X has more requirements for SDMA events. So it creates
a event mux to remap most of event numbers in GPR (General Purpose
Register). If we want to use SDMA support for those module who do
not get the even number as default, we need to configure GPR first.

Thus this patch adds this support of GPR event remapping configuration
to the SDMA driver.

Signed-off-by: Zidan Wang <zidan.wang@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Add device to device support

This patch adds DEV_TO_DEV support for i.MX SDMA driver to support data
transfer between two peripheral FIFOs.
The per_2_per script requires two peripheral addresses and two DMA
requests, and it need to check the src addr and dst addr is in the SPBA
bus space or in the AIPS bus space.

Signed-off-by: Shengjiu Wang <shengjiu.wang@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx: Constify platform_device_id

The platform_device_id is not modified by the driver and core uses it as
const.

Signed-off-by: Krzysztof Kozlowski <k.kozlowski.k@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Add support for version 3 firmware

Currently when version 3.1 of the mx6q SDMA firmware is used we get:

[ 0.392169] imx-sdma 20ec000.sdma: unknown firmware version
[ 0.399281] imx-sdma 20ec000.sdma: initialized

Add support for it.

Based on a patch from Shengjiu Wang from the internal FSL kernel.

Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: switch to dynamic context mode after script loaded

Below comments got from Page4724 of Reference Manual of i.mx6q:
http://cache.freescale.com/files/32bit/doc/ref_manual/IMX6DQRM.pdf

--"Static context mode should be used for the first channel called
after reset to ensure that the all context RAM for that channel is
initialized during the context SAVE phase when the channel is
done or yields. Subsequent calls to the same channel or
different channels may use any of the dynamic context modes.
This will ensure that all context locations for the bootload
channel are initialized, and prevent undefined values in context
RAM from being loaded during the context restore if the
channel is re-started later"

Unfortunately, the rule was broken by commit(5b28aa319bba96987316425a1131813d87cbab35)
.This patch just take them back.

Signed-off-by: Robin Gong <b38343@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Return a proper error code in platform_get_irq()

There is no need to return a 'fake' value upon platform_get_irq() failure.

Just propagate the real error instead.

Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Use devm functions

By using devm functions we can make the code shorter and cleaner.

Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Declare slave capabilities for the generic code

Since ecc19d17868be9c ("dmaengine: Add a warning for drivers not using the
generic slave caps retrieval") the following warning is observed:

[ 0.224981] ------------[ cut here ]------------
[ 0.225013] WARNING: CPU: 0 PID: 1 at drivers/dma/dmaengine.c:830 dma_async_device_register+0x2a0/0x4c8()
[ 0.225023] this driver doesn't support generic slave capabilities reporting

Declare the slave capabilities to avoid such warning.

Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Acked-by: Maxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Split device_control

Split the device_control callback of the Freescale IMX SDMA driver to make use
of the newly introduced callbacks, that will eventually be used to retrieve
slave capabilities.

Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: fix indentation

Fixed a coding style error, switch and case should be at the same indent

Signed-off-by: Asaf Vertz <asaf.vertz@tandemg.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: clarify about firmware not found error

When a firmware cannot be found for the SDMA engine then we can
continue with the internal ROM firmware.

The meaning of this message is frequently asked for, so make clear
that the driver still works with the internal ROM firmware and reduce
the loglevel from err to info.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Add a new DMATYPE for SAI

This patch simply adds a new DMATYPE for SAI which's included
in i.MX6 Solo X.

Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: remove incorrect __init annotation from sdma_init()

When platform_driver_probe() is not used, sdma_probe() can be called
by bind/unbind via sysfs. In addition, sdma_init() can be called by
sdma_probe(). Thus, __init annotation should be removed from sdma_init(),
Also, this patch fixes section mismatch warning.

WARNING: drivers/dma/built-in.o(.text+0xd6e4): Section mismatch in reference from the function sdma_probe() to the function
.init.text:sdma_init()
The function sdma_probe() references
the function __init sdma_init().
This is often because sdma_probe lacks a __init
annotation or the annotation of sdma_init is wrong.

Signed-off-by: Jingoo Han <jg1.han@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: fix another incorrect __init annotation

In e34b731faa7d1 ("dma: imx-sdma: Remove spurious __init annotation on
sdma_probe()"), Mark found an extraneous __init label and fixed it.

However, he missed another one, because now we get this other warning:

WARNING: drivers/dma/imx-sdma.o(.text+0x3bb4): Section mismatch in reference from the function sdma_probe() to the function .init.text:sdma_get_firmware()
The function sdma_probe() references
the function __init sdma_get_firmware().

Same reasoning as the last time, the function may get called at
runtime, so it can't be __init.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: Remove spurious __init annotation on sdma_probe()

We can't annotate probe functions as __init since binding can occur at
any time, not just during kernel init.

Signed-off-by: Mark Brown <broonie@linaro.org>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: Adding tasklet_kill() in sdma_remove function.

Several dma drivers calls tasklet_kill() in remove function. This is done
because all running tasklets should be killed on remove. This is missing
in imx sdma driver, so adding tasklet_kill() in sdma_remove function.

Signed-off-by: Vignesh Raman <Vignesh_Raman@mentor.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: use module_platform_driver for SDMA driver

Currently there is no module_exit declared in SDMA driver, so that once
sdma module is inserted, it's shown with permanent attribute by lsmod,
and it can't be removed.
Use module_platform_driver to register/unregister SDMA driver and modify
SDMA's remove operation, to make SDMA driver possible to be removed.

Signed-off-by: Jiada Wang <jiada_wang@mentor.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: Remove the context argument to the prep_dma_cyclic operation

The argument is always set to NULL and never used. Remove it.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Add a new DMATYPE for Shared Peripheral ASRC

Shared Peripheral ASRC, running on SPBA, needs to use shp sciprts for
DMA transfer. So this patch just adds a new DMATYPE for it.

Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: Save imx_dma_data into sdmac

The filter() function is currently called by xlate() while it transfers
imx_dma_data as a local variable to the filter() but releases the data
right after returning a DMA channel pointer, which results chan->private
pointing an invalid memory space.

So this patch just stores the imx_dma_data into sdmac to make usre the
private pointer valid as long as the channel exists.

Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: Use dma_zalloc_coherent

Use the zeroing function instead of dma_alloc_coherent & memset(,0,)

Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Update imx-sdma cyclic handling to report residue

I received a report this morning from one of the Novena developers that
the behaviour of the iMX6 ASoC codec driver (using imx-pcm-dma.c) was
sub-optimal under high system load.

While there are issues relating to system load remaining, upon reviewing
the ASoC imx-pcm-dma.c driver, it was noticed that it not using the
residue support, because SDMA doesn't support it. This has the effect
that SDMA has to make multiple calls into the ASoC and ALSA code, one
for each period.

Since ALSA's snd_pcm_elapsed() does not need to be called multiple times
and it is entirely sufficient to call it once to update ALSA with the
current buffer position via the pointer method, we can do better here.
We can also avoid stopping the DMA entirely, just like real cyclic DMA
implementations behave. While this means that we replay some old samples,
this is a nicer behaviour than having audio stop and restart.

The changes to achieve this are relatively minor - imx-sdma.c can track
where the DMA is to the nearest descriptor boundary - it does this
already when deciding how many callbacks to issue. In doing this,
buf_tail always points at the descriptor which will complete next.

The residue is defined by the bytes remaining to the end of the buffer,
when the buffer is viewed as a single block of memory [start...end].
So, when we start out, there's a full buffer worth of residue, and this
counts down as we approach the end of the buffer, eventually becoming
zero at the end, before returning to the full buffer worth when we
wrap back to the start.

Moving the walking of the descriptors into the interrupt handler means
that we can update the BD_DONE flag at interrupt time, thus avoiding
a delayed tasklet stopping the cyclic DMA.

This means that the residue can be calculated from (total descriptors -
buf_tail) * descriptor size. This is what the change below does. We
update imx-pcm-dma.c to remove the NO_RESIDUE flag since we now provide
the residue.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx: correct sdmac->status for cyclic dma tx

In cyclic dma tx's handler sdma_handle_channel_loop(),
SDMA channel statue is set to either DMA_ERROR or DMA_IN_PROGRESS
based on each period's status. This has the following issues:

1) If one period's status is BD_RROR, then channel status
will be set to DMA_ERROR, but it will be overwritten to DMA_IN_PROGRESS
if the following periods are OK.
2) DMA client may call sdma_control(DMA_TERMINATE_ALL) to stop the cyclic dma
operation, sdma channel status will be set to DMA_ERROR,
but if after this handler is called, then again the channel status will be overwritten
to DMA_IN_PROGRESS. Then the following dmaengine_prep_dma_cyclic() will always fail,
as channel status is DMA_IN_PROGRESS.

As in cyclic dma tx, channel status will be initially set to DMA_IN_PROGRESS,
driver only needs to change it to DMA_ERROR, when something wrong happens
(one period status is wrong, or stoped by client explicitly).

Signed-off-by: Jiada Wang <jiada_wang@mentor.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: sdma: Add imx25 compatible

imx25 did not work without a firmware previously.

This patch adds a DT compatible to pass the correct data with the
default script addresses for imx25.

Add imx25 compatible to the list of compatibles in the binding
documentation.

Signed-off-by: Markus Pargmann <mpa@pengutronix.de>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: Assign a default script number for ROM firmware cases

i.MX series have inner firmware in its ROM code: when SDMA isn't provided
any firmware from Kernel or rootfs, the default inner ROM firmware will be
activated. However the current driver doesn't assign any script number to
this situation, and those platform running in this case would be broken.

Thus this patch adds a default script number when no external firmware being
loaded so that people would continue to be able to use basic scripts to run
their platform without any firmware.

Reported-by: Fabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: Nicolin Chen <Guangyu.Chen@freescale.com>
Tested-by: Fabio Estevam <fabio.estevam@freescale.com>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: Add new dma type for ssi dual fifo script

This patch adds a new DMA_TYPE for SSI dual FIFO script, included
in SDMA firmware version 2. This script would allow SSI use dual
fifo mode to transimit/receive data without occasional hardware
underrun/overrun.

Signed-off-by: Nicolin Chen <b42378@freescale.com>
Acked-by: Kumar Gala <galak@codeaurora.org>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: Add sdma firmware version 2 support

On i.MX5/6 series, SDMA is using new version firmware to support SSI
dual FIFO feature and HDMI Audio (i.MX6Q/DL only). Thus add it.

Signed-off-by: Nicolin Chen <b42378@freescale.com>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: Fix warnings for LPAE builds

This resolves a number of warnings such as the below when building with
64-bit dma_addr_t on arm:

drivers/dma/imx-sdma.c:1092:3: warning: format '%x' expects argument of
type 'unsigned int', but argument 6 has type 'dma_addr_t' [-Wformat=]

..by upcasting to u64 and using %llx.

Signed-off-by: Olof Johansson <olof@lixom.net>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

ARM: 7857/1: dma: imx-sdma: setup dma mask

The dma mask is not configured in the current code.
This was triggered by soc-dmaengine-pcm which allocate the dma
buffers with the imx-sdma as device.
This commit fix audio on imx31.

Signed-off-by: Philippe Rétornaz <philippe.retornaz@epfl.ch>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

dmaengine: imx-sdma: use DMA_COMPLETE for dma completion status

Acked-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: Staticize sdma_driver_data structures

Sparse report the following warnings:

drivers/dma/imx-sdma.c:330:25: warning: symbol 'sdma_imx31' was not declared. Should it be static?
drivers/dma/imx-sdma.c:351:25: warning: symbol 'sdma_imx25' was not declared. Should it be static?
drivers/dma/imx-sdma.c:357:25: warning: symbol 'sdma_imx35' was not declared. Should it be static?
drivers/dma/imx-sdma.c:375:25: warning: symbol 'sdma_imx51' was not declared. Should it be static?
drivers/dma/imx-sdma.c:395:25: warning: symbol 'sdma_imx53' was not declared. Should it be static?
drivers/dma/imx-sdma.c:414:25: warning: symbol 'sdma_imx6q' was not declared. Should it be static?

Make the sdma_driver_data structures static.

Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: Add ROM script addresses to driver

This adds the ROM script addresses for i.MX25, i.MX5x and i.MX6 to the
SDMA driver needed for the driver to work without additional firmware.

The ROM script addresses are SoC specific and in some cases even tapeout
specific. This patch adds the ROM script addresses only for SoCs which
do not have a tapeout specific SDMA ROM, because currently it's unclear
how this case should be handled.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: Use struct for driver data

Use a struct type instead of an enum type for distinguishing between
different versions. This makes it simpler to handle multiple differences
without cluttering the code with comparisons for certain devtypes.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: remove the unused completion

After the patch: "2ccaef0 dma: imx-sdma: make channel0 operations atomic",
the "done" completion is not used any more.

Just remove it.

Signed-off-by: Huang Shijie <b32955@freescale.com>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: use dev_get_platdata()

Use the wrapper function for retrieving the platform data instead of
accessing dev->platform_data directly.

Signed-off-by: Jingoo Han <jg1.han@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

imx-sdma: remove useless variable

last_used variable is applied only once, so, let's substitute it by its value.

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: move to generic device tree bindings

Update imx-sdma driver to adopt generic DMA device tree bindings. It
calls of_dma_controller_register() with imx-sdma specific of_dma_xlate
to get the generic DMA device tree helper support. The #dma-cells for
imx-sdma must be 3, which includes request ID, peripheral type and
priority.

The existing way of requesting channel, clients directly call
dma_request_channel(), still work there, and will be removed after
all imx-sdma clients get converted to generic DMA device tree helper.

Signed-off-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>

dma: Remove erroneous __exit and __exit_p() references

Removing the annotation with __exit and referencing with __exit_p()
present in dma driver module remove hooks.

Part of the __devexit and __devexit_p() purge.

Signed-off-by: Maxin B. John <maxin.john@enea.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: imx-sdma: remove unneeded mach/hardware.h inclusion

The inclusion of mach/hardware.h is not used in imx-sdma driver at all.
Remove it.

Signed-off-by: Shawn Guo <shawn.guo@linaro.org>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Cc: Vinod Koul <vinod.koul@intel.com>

dmaengine: Pass flags via device_prep_dma_cyclic() callback

Change the parameter list of device_prep_dma_cyclic() so the DMA drivers
can receive the flags coming from clients.
This feature can be used during audio operation to disable all audio
related interrupts when the DMA_PREP_INTERRUPT is cleared from the flags.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: Nicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Acked-by: Vinod Koul <vinod.koul@linux.intel.com>
Signed-off-by: Mark Brown <broonie@opensource.wolfsonmicro.com>

ARM: imx: move platform_data definitions

Platform data for device drivers should be defined in
include/linux/platform_data/*.h, not in the architecture
and platform specific directories.

This moves such data out of the imx include directories

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Mark Brown <broonie@opensource.wolfsonmicro.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
Acked-by: Mauro Carvalho Chehab <mchehab@redhat.com>
Acked-by: Sascha Hauer <kernel@pengutronix.de>
Acked-by: Vinod Koul <vinod.koul@linux.intel.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Dan Williams <djbw@fb.com>
Cc: "Ben Dooks (embedded platforms)" <ben-linux@fluff.org>
Cc: "Wolfram Sang (embedded platforms)" <w.sang@pengutronix.de>
Cc: Chris Ball <cjb@laptop.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Grant Likely <grant.likely@secretlab.ca>
Cc: Alan Cox <alan@linux.intel.com>
Cc: Felipe Balbi <balbi@ti.com>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Florian Tobias Schandinat <FlorianSchandinat@gmx.de>
Cc: Liam Girdwood <lrg@ti.com>
Cc: Javier Martin <javier.martin@vista-silicon.com>
Cc: Shawn Guo <shawn.guo@linaro.org>
Cc: Guennadi Liakhovetski <g.liakhovetski@gmx.de>

dma: imx-sdma: buf_tail should be initialize in prepare function

This fix audio underrun issue. When SNDRV_PCM_TRIGGER_STOP
and SNDRV_PCM_TRIGGER_START, it calls prepare again. buf_tail
should be reset to zero.
So move buf_tail initialization into prepare function.

Signed-off-by: Richard Zhao <richard.zhao@freescale.com>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dma: imx-sdma: make channel0 operations atomic

device_prep_dma_cyclic may be call in audio trigger function which is
atomic context, so we make it atomic too.

- change channel0 lock to spinlock.
- Use polling to wait for channel0 finish running.

Signed-off-by: Richard Zhao <richard.zhao@freescale.com>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: Use dma_sg_len(sg) instead of sg->length

sg->length may or may not contain the length of the dma region to transfer,
depending on the architecture - dma_sg_len(sg) always will though. For the
architectures which use the drivers modified by this patch it probably is the
case that sg->length contains the dma transfer length. But to be consistent and
future proof change them to use dma_sg_len.

To quote Russel King:
sg->length is meaningless to something performing DMA.

In cases where sg_dma_len(sg) and sg->length are the same storage, then
there's no problem. But scatterlists _can_ (and one some architectures) do
split them - especially when you have an IOMMU which can allow you to
combine a scatterlist into fewer entries.

So, anything using sg->length for the size of a scatterlist's DMA transfer
_after_ a call to dma_map_sg() is almost certainly buggy.

The patch has been generated using the following coccinelle patch:
<smpl>
@@
struct scatterlist *sg;
expression X;
@@
-sg[X].length
+sg_dma_len(&sg[X])
@@
struct scatterlist *sg;
@@
-sg->length
+sg_dma_len(sg)
</smpl>

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dma: imx-sdma: keep the callbacks invoked in the tasklet

The current code keeps the callbacks invoked from interrupt context, this
does not conform to the Documentation/dmaengine.txt.

So add tasklet support to fix this issue.

Signed-off-by: Huang Shijie <b32955@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine i.MX SDMA: do not depend on grouped clocks

the current i.MX clock support groups together unrelated clocks
to a single clock which is then used by the driver. This can't
be accomplished with the generic clock framework so we instead
request the individual clocks in the driver. For i.MX there are
generally three different clocks:

ipg: bus clock (needed to access registers)
ahb: dma relevant clock, sometimes referred to as hclk in the datasheet
per: bit clock, pixel clock

This patch changes the driver to request the individual clocks.
Currently all clk_get will get the same clock until the SoCs
are converted to the generic clock framework

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

dmaengine: add context parameter to prep_slave_sg and prep_dma_cyclic

Add context parameter to device_prep_slave_sg() and device_prep_dma_cyclic()
interfaces to allow passing client/target specific information associated
with the data transfer.
Modify all affected DMA engine drivers.

Signed-off-by: Alexandre Bounine <alexandre.bounine@idt.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Felipe Balbi <balbi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dma: imx-sdma: Treat firmware messages as warnings instead of erros

As the SDMA controller can operate without an external firmware being loaded,
treat the firmware related messages as warnings rather than errors.

Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: fix for cookie changes and merge

Fixed trivial issues in drivers:
drivers/dma/imx-sdma.c
drivers/dma/intel_mid_dma.c
drivers/dma/ioat/dma_v3.c
drivers/dma/iop-adma.c
drivers/dma/sirf-dma.c
drivers/dma/timb_dma.c

Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: ensure all DMA engine drivers initialize their cookies

Ensure all DMA engine drivers initialize their cookies in the same way,
so that they all behave in a similar fashion. This means their first
issued cookie will be 2 rather than 1, and will increment to INT_MAX
before returning 1 and starting over.

In connection with this, Dan Williams said:
> Russell King wrote:
> > Secondly, some DMA engine drivers initialize the dma_chan cookie to 0,
> > others to 1.  Is there a reason for this, or are these all buggy?
>
> I know that ioat and iop-adma expect 0 to mean "I have cleaned up this
> descriptor and it is idle", and would break if zero was an in-flight
> cookie value. The reserved usage of zero is an driver internal
> concern, but I have no problem formalizing it as a reserved value.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tested-by: Linus Walleij <linus.walleij@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: provide a common function for completing a dma descriptor

Provide a common function to do the cookie mechanics for completing
a DMA descriptor.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tested-by: Linus Walleij <linus.walleij@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: consolidate assignment of DMA cookies

Everyone deals with assigning DMA cookies in the same way (it's part of
the API so they should be), so lets consolidate the common code into a
helper function to avoid this duplication.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tested-by: Linus Walleij <linus.walleij@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: add private header file

Add a local private header file to contain definitions and declarations
which should only be used by DMA engine drivers.

We also fix linux/dmaengine.h to use LINUX_DMAENGINE_H to guard against
multiple inclusion.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tested-by: Linus Walleij <linus.walleij@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: move last completed cookie into generic dma_chan structure

Every DMA engine implementation declares a last completed dma cookie
in their private dma channel structures. This is pointless, and
forces driver specific code. Move this out into the common dma_chan
structure.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tested-by: Linus Walleij <linus.walleij@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

Revert "drivers/dma: linux/module.h included twice"

This reverts commit 865d9438eb1f7670d2e88849f059db551b320887.

The module.h incsuion twice has been updated tree wide hence this is not
required to be merged.

Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dma: imx-sdma: Print a message when firmare fails to be requested

Print a message when firmare fails to be requested in the case of platform data being used.

While at it, distinguish between the error messages of the device tree and platform data cases.

Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dma: imx-sdma: Let the driver be probed even if no firware is found

On the device tree case the code bails out when the firmware name cannot be found
or if the firmware fails to be requested.

Fix this behaviour as the SDMA engine can still operate with the built-in
ROM scripts.

Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

drivers/dma: linux/module.h included twice

drivers/dma/imx-dma.c and drivers/dma/imx-sdma.c included
'linux/module.h' twice, remove the duplicates.

Signed-off-by: Danny Kukawka <danny.kukawka@bisect.de>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

includecheck: delete any duplicate instances of module.h

Different tree maintainers picked up independently generated
trivial compile fixes based on linux-next testing, resulting
in some cases where a file would have got more than one addition
of module.h once everything was all merged together.

Delete any duplicates so includecheck isn't complaining about
anything related to module.h/export.h changes.

Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>

i.MX SDMA: Fix burstsize settings

Commit 6584cb88 (ARM i.MX dma: Fix burstsize settings) fixed
the mxcmmc driver but forgot to fix the SDMA driver to handle the
correct burtsize.
This make the SD card access works again with DMA on i.MX31 boards.

Signed-off-by: Philippe Rétornaz <philippe.retornaz@epfl.ch>
Tested-by: Sascha Hauer <s.hauer@pengutronix.de>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

i.MX SDMA: Fix burstsize settings

Commit 6584cb88 (ARM i.MX dma: Fix burstsize settings) fixed
the mxcmmc driver but forgot to fix the SDMA driver to handle the
correct burtsize.
This make the SD card access works again with DMA on i.MX31 boards.

Signed-off-by: Philippe Rétornaz <philippe.retornaz@epfl.ch>
Tested-by: Sascha Hauer <s.hauer@pengutronix.de>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dma/imx-sdma: convernt to use bit ops

We don't need extra lock, so we use non-atomic bit ops to set/clear bits,

merge event_mask0 and event_mask1 into an array, it helps use bit ops.

It also fixs the issue:

sdmac->event_mask0 = 1 << sdmac->event_id0;
sdmac->event_mask1 = 1 << (sdmac->event_id0 - 32);

It event_id0 < 32, it shifts negative number. If event_id0 >= 32, it
shifts number >= sizeof(int). Both the cases behavior is undefined.

Signed-off-by: Richard Zhao <richard.zhao@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dma/imx-sdma: use num_events to validate event_id0

event number is not always 32. use num_events for checking instead.

Signed-off-by: Richard Zhao <richard.zhao@linaro.org>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dma/imx-sdma: move clk_enable out of sdma_request_channel

It makes clk_enable/disable pair more readable, and fix one bug:
sdma_init calls sdma_request_channel, but seems don't know
sdma_request_channel enabled the clock.

Signed-off-by: Richard Zhao <richard.zhao@linaro.org>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
[fixed typo in commit log]
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dma/imx-sdma: call sdma_set_channel_priority after sdma_request_channel

sdma_request_channel sets the default priority. sdma_alloc_chan_resources
should call sdma_set_channel_priority thereafter to over write it.

Signed-off-by: Richard Zhao <richard.zhao@linaro.org>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dma/imx-sdma: use readl_relaxed/writel_relaxed and use writel when necessary

use readl_relaxed/writel_relaxed in most places, and use writel when
enable channel, because it needs memory barrier.

Signed-off-by: Richard Zhao <richard.zhao@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dma/imx-sdma: let sdma_run_channel call sdma_enable_channel

Let all enable channel code call sdma_enable_channel.

Signed-off-by: Richard Zhao <richard.zhao@linaro.org>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dma: imx-sdma: start transfer in issue_pending

The DMA engine API requires that transfers are started in issue_pending
instead of tx_submit. Fix this.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
[corrected change log to DMA engine API insteadof DMA API]
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dma/imx-sdma: save irq flags when use spin_lock in sdma_tx_submit

mx53_loco: fix deadlock report from sdma_tx_submit() during boot

BugLink: http://bugs.launchpad.net/bugs/878701

Adjust to use spin_lock_irqsave()/spin_unlock_irqresotre(), so to
make it safe when called from interrupt context.

Signed-off-by: Haitao Zhang <haitao.zhang@linaro.org>
Signed-off-by: Eric Miao <eric.miao@linaro.org>
Signed-off-by: Richard Zhao <richard.zhao@linaro.org>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

IMX/SDMA : save the real count for one DMA transaction.

When we use the SDMA in the UART driver(such as imx6q), we will
meet one situation:
Assume we set 64 bytes for the RX DMA buffer.
The RX DMA buffer has received some data, but not full.
An Aging DMA request will be received by the SDMA controller if we enable the
IDDMAEN(UCR4[6]) in this case.

So the UART driver needs to know the count of the real received bytes,
and push them to upper layer.

Add two new fields to sdmac, and update the `residue` in sdma_tx_status().

Signed-off-by: Huang Shijie <b32955@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

IMX/DMA : set the DMA direction in the sdma_control()

Set the right DMA direction in the sdma_control(), else
we will get the wrong log when enable the DYNAMIC_DEBUG.

Signed-off-by: Huang Shijie <b32955@freescale.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

drivers/dma: Add module.h to files implicitly using it.

Fix files that were implicitly using module.h but not
calling it out for inclusion directly. We'll break those
once we remove the implicit presence otherwise

[With input from Uwe Kleine-König <u.kleine-koenig@pengutronix.de>]

Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>

dmaengine: move drivers to dma_transfer_direction

fixup usage of dma direction by introducing dma_transfer_direction,
this patch moves dma/drivers/* to use new enum

Cc: Jassi Brar <jaswinder.singh@linaro.org>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Viresh Kumar <viresh.kumar@st.com>
Cc: Linus Walleij <linus.walleij@linaro.org>
Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Cc: Mika Westerberg <mika.westerberg@iki.fi>
Cc: H Hartley Sweeten <hartleys@visionengravers.com>
Cc: Li Yang <leoli@freescale.com>
Cc: Zhang Wei <zw@zh-kernel.org>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Cc: Shawn Guo <shawn.guo@freescale.com>
Cc: Yong Wang <yong.y.wang@intel.com>
Cc: Tomoya MORINAGA <tomoya-linux@dsn.lapis-semi.com>
Cc: Boojin Kim <boojin.kim@samsung.com>
Cc: Barry Song <Baohua.Song@csr.com>
Acked-by: Mika Westerberg <mika.westerberg@iki.fi>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Viresh Kumar <viresh.kumar@st.com>
Acked-by: Nicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine i.MX DMA/SDMA: add missing include of linux/module.h

Add missing include of linux/module.h to fix build error.

Signed-off-by: Axel Lin <axel.lin@gmail.com>
Acked-by: Wolfram Sang <w.sang@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine i.MX SDMA: use request_firmware_nowait

The firmware blob may not be available when the driver
probes. Instead of blocking the whole kernel use
request_firmware_nowait() and continue without firmware.
The ROM scripts can already be used then if available.
For the devicetree case the ROM scripts are not available,
still the probe function should not block. The driver
will be unusable in this case, but we have no way of
detecting this properly. The configuration of the dma
channels will fail, so nothing bad should happen.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine i.MX SDMA: set firmware scripts addresses to negative value initially

If we do not have a firmare script for a given transfer,
the setup of this channel must fail. For this the script
addresses have to be < 0 initially, not 0.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine i.MX SDMA: lock channel 0

channel0 of the sdma engine is the configuration channel. It
is a shared resource and thus must be protected by a mutex.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: add device tree probe support

It adds device tree probe support for imx-sdma driver.

Signed-off-by: Shawn Guo <shawn.guo@linaro.org>
Cc: Grant Likely <grant.likely@secretlab.ca>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Acked-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: sdma_get_firmware does not need to copy fw_name

It does not need to allocate space and copy fw_name in function
sdma_get_firmware().

Signed-off-by: Shawn Guo <shawn.guo@linaro.org>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Acked-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: use platform_device_id to identify sdma version

It might be not good to use software defined version to identify sdma
device type, when hardware does not define such version. Instead,
soc name is stable enough to define the device type.

The patch uses platform_device_id rather than version number passed
by platform data to identify sdma device type/version.

Signed-off-by: Shawn Guo <shawn.guo@linaro.org>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Acked-by: Grant Likely <grant.likely@secretlab.ca>
Acked-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: return proper error if kzalloc fails

Signed-off-by: Axel Lin <axel.lin@gmail.com>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: imx-sdma: pass sdma firmware name via platform data

It is not good to have cpu_name and to_version encoded into sdma
firmware name as variables. For example, there are three TOs of
imx51 soc, the sdma script never changes since TO1, which means
all three TOs of imx51 uses TO1 version of sdma script. But we
have to prepare three identical firmwares, sdma-imx51-to1.bin
sdma-imx51-to2.bin and sdma-imx51-to3.bin, to have the kernel
capable of running on all three TOs.

The patch removes cpu_name and to_version from sdma platform data,
and instead uses fw_name to pass the firmware name, so that we can
pass the TO version where it's relevant and skip it where only one
firmware exists.

Signed-off-by: Shawn Guo <shawn.guo@linaro.org>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

dmaengine: imx-sdma: fix up param for the last BD in sdma_prep_slave_sg()

As per the reference manual, bit "L" should be set while bit "C"
should be cleared for the last buffer descriptor in the non-cyclic
chain, so that sdma can stop trying to find the next BD and end
the transfer.

In case of sdma_prep_slave_sg(), BD_LAST needs to be set and BD_CONT
be cleared for the last BD.

Signed-off-by: Shawn Guo <shawn.guo@freescale.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

dmaengine: imx-sdma: correct sdmac->status in sdma_handle_channel_loop()

sdma_handle_channel_loop() is the handler of cyclic tx. One period
success does not really mean the success of the tx. Instead of
DMA_SUCCESS, DMA_IN_PROGRESS should be the one to tell.

Signed-off-by: Shawn Guo <shawn.guo@freescale.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

dmaengine: imx-sdma: return sdmac->status in sdma_tx_status()

The sdmac->status was designed to reflect the status of the tx,
so simply return it in sdma_tx_status(). Then dma client can call
dma_async_is_tx_complete() to know the status of the tx.

Signed-off-by: Shawn Guo <shawn.guo@freescale.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

dmaengine: imx-sdma: set sdmac->status to DMA_ERROR in err_out of sdma_prep_slave_sg()

sdma_prep_dma_cyclic() sets sdmac->status to DMA_ERROR in err_out,
and sdma_prep_slave_sg() needs to do the same. Otherwise,
sdmac->status stays at DMA_IN_PROGRESS, which will make the function
return immediately next time it gets called.

Signed-off-by: Shawn Guo <shawn.guo@freescale.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

dmaengine: imx-sdma: remove IMX_DMA_SG_LOOP handling in sdma_prep_slave_sg()

This is a leftover from the time that the driver did not have
sdma_prep_dma_cyclic callback and implemented sound dma as a looped
sg chain. And it can be removed now.

Signed-off-by: Shawn Guo <shawn.guo@freescale.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

dmaengine i.MX SDMA: reserve channel 0 by not registering it

We need channel 0 of the sdma engine for internal purposes. We
accomplished this by calling dma_request_channel() in the probe
function. This does not work when multiple dma engines are
present which is the case when IPU support for i.MX31/35 is
compiled in. So instead of registering channel 0 and reserving
it afterwards simply do not register it in the first place.
With this the dmaengine channel counting does not match sdma
channel counting anymore, so we have to use sdma channel counting
in the driver.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

dmaengine i.MX SDMA: initialize dma capabilities outside channel loop

The capabilities are device specific fields, not channel specific fields.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

dmaengine i.MX SDMA: do not initialize chan_id field

This is bogus as the dmaengine core will overwrite this field.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

dmaengine i.MX sdma: check sg entries for valid addresses and lengths

This patch lets sdma_prep_slave_sg fail if the entries of an
sg list do not start on multiples of the word size or if the
lengths are not multiple of the word size.
Also, catch the previously unhandled DMA_SLAVE_BUSWIDTH_8_BYTES
and DMA_SLAVE_BUSWIDTH_UNDEFINED cases.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

dmaengine i.MX sdma: set maximum segment size for our device

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>

dmaengine: imx-sdma: fix inconsistent naming in sdma_assign_cookie()

Variable name sdma and sdmac are consistently used as the pointer to
sdma_engine and sdma_channel respectively throughout the file. The
patch fixes the inconsistency seen in function sdma_assign_cookie().

Signed-off-by: Shawn Guo <shawn.guo@freescale.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

dmaengine: imx-sdma: propagate error in sdma_probe() instead of returning 0

Signed-off-by: Shawn Guo <shawn.guo@freescale.com>
Acked-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

dmaengine i.MX SDMA: Fix firmware loading

When loading the microcode to the SDMA engine we have to use
the ram_code_start_addr found in the firmware image. The copy
in the sdma engine is not initialized correctly. This is broken
since:
5b28aa3 dmaengine i.MX SDMA: Allow to run without firmware

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

dmaengine i.MX SDMA: initialize on module_init

The firmware framework gets initialized during fs_initcall time, so
we are not allowed to call request_firmware earlier.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

dmaengine: imx-sdma: fix bug in buffer descriptor initialization

Currently while submitting scatterlists with more than one SG
entry the DMA buffer address from the first SG entry is inserted
into all initialized DMA buffer descriptors. This is due to the
typo in the for_each_sg() loop where the scatterlist pointer is
used for obtaining the DMA buffer address and _not_ the SG list
iterator.

As a result all received data will be written only into the first
DMA buffer while reading. While writing the data from the first
DMA buffer is send to the device multiple times. This caused
the filesystem destruction on the MMC card when using DMA in
mxcmmc driver.

Signed-off-by: Anatolij Gustschin <agust@denx.de>
Acked-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

dmaengine i.MX SDMA: Allow to run without firmware

The SDMA firmware consists of a ROM part and a RAM part.
The ROM part is always present in the SDMA engine and
is sufficient for many cases.
This patch allows to pass in platform data containing
the script addresses in ROM, so loading a firmware is
optional now.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Acked-by: Dan Williams <dan.j.williams@intel.com>

dmaengine: Add Freescale i.MX SDMA support

This patch adds support for the Freescale i.MX SDMA engine.

The SDMA engine is a scatter/gather DMA engine which is implemented
as a seperate coprocessor. SDMA needs its own firmware which is
requested using the standard request_firmware mechanism. The firmware
has different entry points for each peripheral type, so drivers
have to pass the peripheral type to the DMA engine which in turn
picks the correct firmware entry point from a table contained in
the firmware image itself.
The original Freescale code also supports support for transfering
data to the internal SRAM which needs different entry points to
the firmware. Support for this is currently not implemented. Also,
support for the ASRC (asymmetric sample rate converter) is skipped.

I took a very simple approach to implement dmaengine support. Only
a single descriptor is statically assigned to a each channel. This
means that transfers can't be queued up but only a single transfer
is in progress. This simplifies implementation a lot and is sufficient
for the usual device/memory transfers.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Reviewed-by: Linus Walleij <linus.ml.walleij@gmail.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>