dmaengine: pl330: Clear callback_result for re-used descs

The pl330 driver re-uses DMA descriptors rather than reallocating
them each time. At present, upon re-use the .callback member is
cleared, but .callback result is not. This causes problems where a
consuming driver sets the .callback_result for some submissions but
not for others, as eventually the function is invoked erronously.

Clear .callback_result along with .callback

Signed-off-by: Daniel Scally <dan.scally@ideasonboard.com>
Link: https://lore.kernel.org/r/20240118112959.1027471-1-dan.scally@ideasonboard.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: issue_pending waits until WFP state

According to DMA-330 errata notice[1] 71930, DMAKILL
cannot clear internal signal, named pipeline_req_active.
it makes that pl330 would wait forever in WFP state
although dma already send dma request if pl330 gets
dma request before entering WFP state.

The errata suggests that polling until entering WFP state
as workaround and then peripherals allows to issue dma request.

[1]: https://developer.arm.com/documentation/genc008428/latest

Signed-off-by: Bumyong Lee <bumyong.lee@samsung.com>
Link: https://lore.kernel.org/r/20231219055026.118695-1-bumyong.lee@samsung.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: Return DMA_PAUSED when transaction is paused

pl330_pause() does not set anything to indicate paused condition which
causes pl330_tx_status() to return DMA_IN_PROGRESS. This breaks 8250
DMA flush after the fix in commit 57e9af7831dc ("serial: 8250_dma: Fix
DMA Rx rearm race"). The function comment for pl330_pause() claims
pause is supported but resume is not which is enough for 8250 DMA flush
to work as long as DMA status reports DMA_PAUSED when appropriate.

Add PAUSED state for descriptor and mark BUSY descriptors with PAUSED
in pl330_pause(). Return DMA_PAUSED from pl330_tx_status() when the
descriptor is PAUSED.

Reported-by: Richard Tresidder <rtresidd@electromag.com.au>
Tested-by: Richard Tresidder <rtresidd@electromag.com.au>
Fixes: 88987d2c7534 ("dmaengine: pl330: add DMA_PAUSE feature")
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/linux-serial/f8a86ecd-64b1-573f-c2fa-59f541083f1a@electromag.com.au/
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Link: https://lore.kernel.org/r/20230526105434.14959-1-ilpo.jarvinen@linux.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: rename _start to prevent build error

"_start" is used in several arches and proably should be reserved
for ARCH usage. Using it in a driver for a private symbol can cause
a build error when it conflicts with ARCH usage of the same symbol.

Therefore rename pl330's "_start" to "pl330_start_thread" so that there
is no conflict and no build error.

drivers/dma/pl330.c:1053:13: error: '_start' redeclared as different kind of symbol
1053 | static bool _start(struct pl330_thread *thrd)
| ^~~~~~
In file included from ../include/linux/interrupt.h:21,
from ../drivers/dma/pl330.c:18:
arch/riscv/include/asm/sections.h:11:13: note: previous declaration of '_start' with type 'char[]'
11 | extern char _start[];
| ^~~~~~

Fixes: b7d861d93945 ("DMA: PL330: Merge PL330 driver into drivers/dma/")
Fixes: ae43b3289186 ("ARM: 8202/1: dmaengine: pl330: Add runtime Power Management support v12")
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Jaswinder Singh <jassisinghbrar@gmail.com>
Cc: Boojin Kim <boojin.kim@samsung.com>
Cc: Krzysztof Kozlowski <krzk@kernel.org>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Vinod Koul <vkoul@kernel.org>
Cc: dmaengine@vger.kernel.org
Cc: linux-riscv@lists.infradead.org
Link: https://lore.kernel.org/r/20230524045310.27923-1-rdunlap@infradead.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: Remove unused flags

txd.flags is unused and need not be updated.

Signed-off-by: Harini Katakam <harini.katakam@amd.com>
Link: https://lore.kernel.org/r/20220802102232.17653-1-harini.katakam@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: Fix lockdep warning about non-static key

The DEFINE_SPINLOCK() macro shouldn't be used for dynamically allocated
spinlocks. The lockdep warns about this and disables locking validator.
Fix the warning by making lock static.

INFO: trying to register non-static key.
The code is fine but needs lockdep annotation, or maybe
you didn't initialize this object before use?
turning off the locking correctness validator.
Hardware name: Radxa ROCK Pi 4C (DT)
Call trace:
dump_backtrace.part.0+0xcc/0xe0
show_stack+0x18/0x6c
dump_stack_lvl+0x8c/0xb8
dump_stack+0x18/0x34
register_lock_class+0x4a8/0x4cc
__lock_acquire+0x78/0x20cc
lock_acquire.part.0+0xe0/0x230
lock_acquire+0x68/0x84
_raw_spin_lock_irqsave+0x84/0xc4
add_desc+0x44/0xc0
pl330_get_desc+0x15c/0x1d0
pl330_prep_dma_cyclic+0x100/0x270
snd_dmaengine_pcm_trigger+0xec/0x1c0
dmaengine_pcm_trigger+0x18/0x24
...

Fixes: e588710311ee ("dmaengine: pl330: fix descriptor allocation fail")
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Link: https://lore.kernel.org/r/20220520181432.149904-1-dmitry.osipenko@collabora.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

ARM: 9172/1: amba: Cleanup amba pclk operation

There is no user about amba_pclk_[un]prepare() besides pl330.c,
directly use clk_[un]prepare(). After this, all the function about
amba pclk operation, enable, disable, [un]prepare could be killed.

Acked-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>

dmaengine: pl330: fix wrong usage of spinlock flags in dma_cyclc

flags varible which is the input parameter of pl330_prep_dma_cyclic()
should not be used by spinlock_irq[save/restore] function.

Signed-off-by: Jongho Park <jongho7.park@samsung.com>
Signed-off-by: Bumyong Lee <bumyong.lee@samsung.com>
Signed-off-by: Chanho Park <chanho61.park@samsung.com>
Link: https://lore.kernel.org/r/20210507063647.111209-1-chanho61.park@samsung.com
Fixes: f6f2421c0a1c ("dmaengine: pl330: Merge dma_pl330_dmac and pl330_dmac structs")
Cc: stable@vger.kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

amba: Make the remove callback return void

All amba drivers return 0 in their remove callback. Together with the
driver core ignoring the return value anyhow, it doesn't make sense to
return a value here.

Change the remove prototype to return void, which makes it explicit that
returning an error value doesn't work as expected. This simplifies changing
the core remove callback to return void, too.

Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Acked-by: Dmitry Torokhov <dmitry.torokhov@gmail.com>
Acked-by: Krzysztof Kozlowski <krzk@kernel.org> # for drivers/memory
Acked-by: Mark Brown <broonie@kernel.org>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Suzuki K Poulose <suzuki.poulose@arm.com> # for hwtracing/coresight
Acked-By: Vinod Koul <vkoul@kernel.org> # for dmaengine
Acked-by: Guenter Roeck <linux@roeck-us.net> # for watchdog
Acked-by: Wolfram Sang <wsa@kernel.org> # for I2C
Acked-by: Takashi Iwai <tiwai@suse.de> # for sound
Acked-by: Vladimir Zapolskiy <vz@mleia.com> # for memory/pl172
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/r/20210126165835.687514-5-u.kleine-koenig@pengutronix.de
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>

dmaengine: pl330: _prep_dma_memcpy: Fix wrong burst size

Actually, burst size is equal to '1 << desc->rqcfg.brst_size'.
we should use burst size, not desc->rqcfg.brst_size.

dma memcpy performance on Rockchip RV1126
@ 1512MHz A7, 1056MHz LPDDR3, 200MHz DMA:

dmatest:

/# echo dma0chan0 > /sys/module/dmatest/parameters/channel
/# echo 4194304 > /sys/module/dmatest/parameters/test_buf_size
/# echo 8 > /sys/module/dmatest/parameters/iterations
/# echo y > /sys/module/dmatest/parameters/norandom
/# echo y > /sys/module/dmatest/parameters/verbose
/# echo 1 > /sys/module/dmatest/parameters/run

dmatest: dma0chan0-copy0: result #1: 'test passed' with src_off=0x0 dst_off=0x0 len=0x400000
dmatest: dma0chan0-copy0: result #2: 'test passed' with src_off=0x0 dst_off=0x0 len=0x400000
dmatest: dma0chan0-copy0: result #3: 'test passed' with src_off=0x0 dst_off=0x0 len=0x400000
dmatest: dma0chan0-copy0: result #4: 'test passed' with src_off=0x0 dst_off=0x0 len=0x400000
dmatest: dma0chan0-copy0: result #5: 'test passed' with src_off=0x0 dst_off=0x0 len=0x400000
dmatest: dma0chan0-copy0: result #6: 'test passed' with src_off=0x0 dst_off=0x0 len=0x400000
dmatest: dma0chan0-copy0: result #7: 'test passed' with src_off=0x0 dst_off=0x0 len=0x400000
dmatest: dma0chan0-copy0: result #8: 'test passed' with src_off=0x0 dst_off=0x0 len=0x400000

Before:

dmatest: dma0chan0-copy0: summary 8 tests, 0 failures 48 iops 200338 KB/s (0)

After this patch:

dmatest: dma0chan0-copy0: summary 8 tests, 0 failures 179 iops 734873 KB/s (0)

After this patch and increase dma clk to 400MHz:

dmatest: dma0chan0-copy0: summary 8 tests, 0 failures 259 iops 1062929 KB/s (0)

Signed-off-by: Sugar Zhang <sugar.zhang@rock-chips.com>
Link: https://lore.kernel.org/r/1605326106-55681-1-git-send-email-sugar.zhang@rock-chips.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: Remove unreachable code

_setup_req(..) never returns negative value.
Hence the condition ret < 0 is never met

Signed-off-by: Surendran K <surendran.k@samsung.com>
Link: https://lore.kernel.org/r/20201016103347.63084-1-surendran.k@samsung.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: fix argument for tasklet

Commit 59cd818763e8 ("dmaengine: fsl: convert tasklets to use new
tasklet_setup() API") converted the pl330 driver to use new tasklet
functions but missed that driver calls the tasklet function directly as
well, so update it.

Fixes: 59cd818763e8 ("dmaengine: fsl: convert tasklets to use new tasklet_setup() API")
Reported-by: kernel test robot <lkp@intel.com>
Link: https://lore.kernel.org/r/20200930121735.49699-1-vkoul@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: convert tasklets to use new tasklet_setup() API

In preparation for unconditionally passing the
struct tasklet_struct pointer to all tasklet
callbacks, switch to using the new tasklet_setup()
and from_tasklet() to pass the tasklet pointer explicitly.

Signed-off-by: Romain Perier <romain.perier@gmail.com>
Signed-off-by: Allen Pais <allen.lkml@gmail.com>
Link: https://lore.kernel.org/r/20200831103542.305571-21-allen.lkml@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: Drop local dma_parms

Since commit f458488425f1 ("amba: Initialize dma_parms for amba
devices"), struct amba_device already provides a dma_parms structure,
so we can save allocating another one.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Link: https://lore.kernel.org/r/c9e58882e33f22f9b0a6d65a5507e24004512148.1599164692.git.robin.murphy@arm.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: Simplify with dev_err_probe()

Common pattern of handling deferred probe can be simplified with
dev_err_probe(). Less code and the error value gets printed.

Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
Link: https://lore.kernel.org/r/20200828152637.16903-1-krzk@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: fix instruction dump formatting

Instruction dump uses two printk() in a row to print one instruction. Use
KERN_CONT to prevent breaking the output in the middle.

Signed-off-by: Łukasz Stelmach <l.stelmach@samsung.com>
Link: https://lore.kernel.org/r/20200813204123.19044-1-l.stelmach@samsung.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: Fix burst length if burst size is smaller than bus width

Move the burst len fixup after setting the generic value for it. This
finally enables the fixup introduced by commit 137bd11090d8 ("dmaengine:
pl330: Align DMA memcpy operations to MFIFO width"), which otherwise was
overwritten by the generic value.

Reported-by: kernel test robot <lkp@intel.com>
Fixes: 137bd11090d8 ("dmaengine: pl330: Align DMA memcpy operations to MFIFO width")
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Link: https://lore.kernel.org/r/20200825064617.16193-1-m.szyprowski@samsung.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

treewide: Use fallthrough pseudo-keyword

Replace the existing /* fall through */ comments and its variants with
the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary
fall-through markings when it is the case.

[1] https://www.kernel.org/doc/html/v5.7/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-through

Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>

dmaengine: pl330: Demote obvious misuse of kerneldoc to standard comment block

No 'struct' title is provided. Nor are any attribute descriptions.

Fixes the following W=1 kernel build warning(s):

drivers/dma/pl330.c:295: warning: cannot understand function prototype: 'struct pl330_reqcfg '

Signed-off-by: Lee Jones <lee.jones@linaro.org>
Cc: Philipp Zabel <p.zabel@pengutronix.de>
Cc: Jaswinder Singh <jassi.brar@samsung.com>
Link: https://lore.kernel.org/r/20200714111546.1755231-6-lee.jones@linaro.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: Add quirk 'arm,pl330-periph-burst'

This patch adds the qurik to use burst transfers only
for pl330 controller, even for request with a length of 1.

Although, the correct way should be: if the peripheral request
length is 1, the peripheral should use SINGLE request, and then
notify the dmac using SINGLE mode by src/dst_maxburst with 1.

For example, on the Rockchip SoCs, all the peripherals can use
SINGLE or BURST request by setting GRF registers. it is possible
that if these peripheral drivers are used only for Rockchip SoCs.
Unfortunately, it's not, such as dw uart, which is used so widely,
and we can't set src/dst_maxburst according to the SoCs' specific
to compatible with all the other SoCs.

So, for convenience, all the peripherals are set as BURST request
by default on the Rockchip SoCs. even for request with a length of 1.
the current pl330 driver will perform SINGLE transfer if the client's
maxburst is 1, which still should be working according to chapter 2.6.6
of datasheet which describe how DMAC performs SINGLE transfers for
a BURST request. Unfortunately, it's broken on the Rockchip SoCs,
which support only matching transfers, such as BURST transfer for
BURST request, SINGLE transfer for SINGLE request.

Finally, we add the quirk to specify pl330 to use burst transfers only.

Signed-off-by: Sugar Zhang <sugar.zhang@rock-chips.com>
Link: https://lore.kernel.org/r/1593439555-68130-5-git-send-email-sugar.zhang@rock-chips.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: Improve transfer efficiency for the dregs

Only the unaligned burst transfers have the dregs.
so, still use BURST transfer with a reduced size
for better performance.

Signed-off-by: Sugar Zhang <sugar.zhang@rock-chips.com>
Link: https://lore.kernel.org/r/1593439555-68130-3-git-send-email-sugar.zhang@rock-chips.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: Remove the burst limit for quirk 'NO-FLUSHP'

There is no reason to limit the performance on the 'NO-FLUSHP' SoCs,
because 'FLUSHP' instruction is broken on these platforms, so remove
the limit to improve the efficiency.

Signed-off-by: Sugar Zhang <sugar.zhang@rock-chips.com>
Link: https://lore.kernel.org/r/1593439555-68130-2-git-send-email-sugar.zhang@rock-chips.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: Make sure the debug is idle before doing DMAGO

According to the datasheet of pl330:

Example 2-1 Using DMAGO with the debug instruction registers

1. Create a program for the DMA channel
2. Store the program in a region of system memory
3. Poll the DBGSTATUS Register to ensure that the debug is idle
4. Write to the DBGINST0 Register
5. Write to the DBGINST1 Register
6. Write zero to the DBGCMD Register

so, we should make sure the debug is idle before step 4/5/6, not
only step 6. if not, there maybe a risk that fail to write DBGINST0/1.

Signed-off-by: Sugar Zhang <sugar.zhang@rock-chips.com>
Link: https://lore.kernel.org/r/1591234598-78919-1-git-send-email-sugar.zhang@rock-chips.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: Convert to the *_late and *_early system sleep callbacks

It has turned out that it's in general a good idea for dmaengines to allow
DMA requests during the entire dpm_suspend() phase. Therefore, convert the
pl330 driver into using SET_LATE_SYSTEM_SLEEP_PM_OPS.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Link: https://lore.kernel.org/r/20191205143746.24873-3-ulf.hansson@linaro.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: Drop boilerplate code for suspend/resume

Let's drop the boilerplate code in the system suspend/resume callbacks and
convert to use pm_runtime_force_suspend|resume(). This change also has a
nice side effect, as pm_runtime_force_resume() may decide to leave the
device in low power state, when that is feasible, thus avoiding to waste
both time and energy during system resume.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Link: https://lore.kernel.org/r/20191205143746.24873-2-ulf.hansson@linaro.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: use the same attributes when freeing pl330->mcode_cpu

In function dmac_alloc_resources(), pl330->mcode_cpu is allocated using
dma_alloc_attrs() but freed with dma_free_coherent().
Use the correct dma_free_attrs() function to free pl330->mcode_cpu.

Signed-off-by: Fuqian Huang <huangfq.daxian@gmail.com>
Link: https://lore.kernel.org/r/20190726105947.25342-1-huangfq.daxian@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmagengine: pl330: add code to get reset property

The DMA controller on some SoCs can be held in reset, and thus requires
the reset signal(s) to deasserted. Most SoCs will have just one reset
signal, but there are others, i.e. Arria10/Stratix10 will have an
additional reset signal, referred to as the OCP.

Add code to get the reset property from the device tree for deassert and
assert.

Signed-off-by: Dinh Nguyen <dinguyen@kernel.org>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152

Based on 1 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license as published by
the free software foundation either version 2 of the license or at
your option any later version

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-or-later

has been chosen to replace the boilerplate/reference in 3029 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Allison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190527070032.746973796@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

dmaengine: pl330: _stop: clear interrupt status

This patch kill instructs the DMAC to immediately terminate
execution of a thread. and then clear the interrupt status,
at last, stop generating interrupts for DMA_SEV. to guarantee
the next dma start is clean. otherwise, one interrupt maybe leave
to next start and make some mistake.

we can reporduce the problem as follows:

DMASEV: modify the event-interrupt resource, and if the INTEN sets
function as interrupt, the DMAC will set irq<event_num> HIGH to
generate interrupt. write INTCLR to clear interrupt.

DMA EXECUTING INSTRUCTS DMA TERMINATE
| |
| |
... _stop
| |
| spin_lock_irqsave
DMASEV |
| |
| mask INTEN
| |
| DMAKILL
| |
| spin_unlock_irqrestore

in above case, a interrupt was left, and if we unmask INTEN, the DMAC
will set irq<event_num> HIGH to generate interrupt.

to fix this, do as follows:

DMA EXECUTING INSTRUCTS DMA TERMINATE
| |
| |
... _stop
| |
| spin_lock_irqsave
DMASEV |
| |
| DMAKILL
| |
| clear INTCLR
| mask INTEN
| |
| spin_unlock_irqrestore

Signed-off-by: Sugar Zhang <sugar.zhang@rock-chips.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: introduce debugfs interface

This patch adds debugfs interface to show the relationship between
DMA threads (hardware resource for transferring data) and DMA
channel ID of DMA slave.

Typically, PL330 has many slaves than number of DMA threads.
So sometimes PL330 cannot allocate DMA threads for all slaves even
if a user specify DMA channel ID in devicetree. This interface will
be useful for checking that DMA threads are allocated or not.

Below is an output sample:

$ sudo cat /sys/kernel/debug/ff1f0000.dmac
PL330 physical channels:
THREAD: CHANNEL:
-------- -----
0 8
1 9
2 11
3 12
4 14
5 15
6 10
7 --

Signed-off-by: Katsuhiro Suzuki <katsuhiro@katsuster.net>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: drop useless LIST_HEAD

Drop LIST_HEAD where the variable it declares is never used.

The variable has not been used since the function was introduced
in 740aa95703c5 ("dmaengine: pl330: Split device_control").

The semantic patch that fixes this problem is as follows:
(http://coccinelle.lip6.fr/)

// <smpl>
@@
identifier x;
@@
- LIST_HEAD(x);
... when != x
// </smpl>

Fixes: 740aa95703c5 ("dmaengine: pl330: Split device_control")
Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: remove dma_slave_config direction usage

dma_slave_config direction was marked as deprecated quite some
time back, remove the usage from this driver so that the field
can be removed

Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: fix irq race with terminate_all

In pl330_update() when checking if a channel has been aborted, the
channel's lock is not taken, only the overall pl330_dmac lock. But in
pl330_terminate_all() the aborted flag (req_running==-1) is set under
the channel lock and not the pl330_dmac lock.

With threaded interrupts, this leads to a potential race:

pl330_terminate_all pl330_update
------------------- ------------
lock channel
entry
lock pl330
_stop channel
unlock pl330
lock pl330
check req_running != -1
req_running = -1
_start channel

Signed-off-by: John Keeping <john@metanate.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: remove set but unused variable

Compiler complains (with W=1):
drivers/dma/pl330.c: In function ‘pl330_release_channel’:
drivers/dma/pl330.c:1782:21: warning:
variable ‘pl330’ set but not used [-Wunused-but-set-variable]
struct pl330_dmac *pl330;
^~~~~

Remove the pl330 variable in pl330_release_channel as it is set but
never used.

Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: Mark expected switch fall-through

In preparation to enabling -Wimplicit-fallthrough, mark switch cases
where we are expecting to fall through.

Reviewed-by: Krzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: report BURST residue granularity

The reported residue is already calculated in BURST unit granularity, so
advertise this capability properly to other devices in the system.

Fixes: aee4d1fac887 ("dmaengine: pl330: improve pl330_tx_status() function")
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

treewide: kzalloc() -> kcalloc()

The kzalloc() function has a 2-factor argument form, kcalloc(). This
patch replaces cases of:

kzalloc(a * b, gfp)

with:
kcalloc(a * b, gfp)

as well as handling cases of:

kzalloc(a * b * c, gfp)

with:

kzalloc(array3_size(a, b, c), gfp)

as it's slightly less ugly than:

kzalloc_array(array_size(a, b), c, gfp)

This does, however, attempt to ignore constant size factors like:

kzalloc(4 * 1024, gfp)

though any constants defined via macros get caught up in the conversion.

Any factors with a sizeof() of "unsigned char", "char", and "u8" were
dropped, since they're redundant.

The Coccinelle script used for this was:

// Fix redundant parens around sizeof().
@@
type TYPE;
expression THING, E;
@@

(
kzalloc(
- (sizeof(TYPE)) * E
+ sizeof(TYPE) * E
, ...)
|
kzalloc(
- (sizeof(THING)) * E
+ sizeof(THING) * E
, ...)
)

// Drop single-byte sizes and redundant parens.
@@
expression COUNT;
typedef u8;
typedef __u8;
@@

(
kzalloc(
- sizeof(u8) * (COUNT)
+ COUNT
, ...)
|
kzalloc(
- sizeof(__u8) * (COUNT)
+ COUNT
, ...)
|
kzalloc(
- sizeof(char) * (COUNT)
+ COUNT
, ...)
|
kzalloc(
- sizeof(unsigned char) * (COUNT)
+ COUNT
, ...)
|
kzalloc(
- sizeof(u8) * COUNT
+ COUNT
, ...)
|
kzalloc(
- sizeof(__u8) * COUNT
+ COUNT
, ...)
|
kzalloc(
- sizeof(char) * COUNT
+ COUNT
, ...)
|
kzalloc(
- sizeof(unsigned char) * COUNT
+ COUNT
, ...)
)

// 2-factor product with sizeof(type/expression) and identifier or constant.
@@
type TYPE;
expression THING;
identifier COUNT_ID;
constant COUNT_CONST;
@@

(
- kzalloc
+ kcalloc
(
- sizeof(TYPE) * (COUNT_ID)
+ COUNT_ID, sizeof(TYPE)
, ...)
|
- kzalloc
+ kcalloc
(
- sizeof(TYPE) * COUNT_ID
+ COUNT_ID, sizeof(TYPE)
, ...)
|
- kzalloc
+ kcalloc
(
- sizeof(TYPE) * (COUNT_CONST)
+ COUNT_CONST, sizeof(TYPE)
, ...)
|
- kzalloc
+ kcalloc
(
- sizeof(TYPE) * COUNT_CONST
+ COUNT_CONST, sizeof(TYPE)
, ...)
|
- kzalloc
+ kcalloc
(
- sizeof(THING) * (COUNT_ID)
+ COUNT_ID, sizeof(THING)
, ...)
|
- kzalloc
+ kcalloc
(
- sizeof(THING) * COUNT_ID
+ COUNT_ID, sizeof(THING)
, ...)
|
- kzalloc
+ kcalloc
(
- sizeof(THING) * (COUNT_CONST)
+ COUNT_CONST, sizeof(THING)
, ...)
|
- kzalloc
+ kcalloc
(
- sizeof(THING) * COUNT_CONST
+ COUNT_CONST, sizeof(THING)
, ...)
)

// 2-factor product, only identifiers.
@@
identifier SIZE, COUNT;
@@

- kzalloc
+ kcalloc
(
- SIZE * COUNT
+ COUNT, SIZE
, ...)

// 3-factor product with 1 sizeof(type) or sizeof(expression), with
// redundant parens removed.
@@
expression THING;
identifier STRIDE, COUNT;
type TYPE;
@@

(
kzalloc(
- sizeof(TYPE) * (COUNT) * (STRIDE)
+ array3_size(COUNT, STRIDE, sizeof(TYPE))
, ...)
|
kzalloc(
- sizeof(TYPE) * (COUNT) * STRIDE
+ array3_size(COUNT, STRIDE, sizeof(TYPE))
, ...)
|
kzalloc(
- sizeof(TYPE) * COUNT * (STRIDE)
+ array3_size(COUNT, STRIDE, sizeof(TYPE))
, ...)
|
kzalloc(
- sizeof(TYPE) * COUNT * STRIDE
+ array3_size(COUNT, STRIDE, sizeof(TYPE))
, ...)
|
kzalloc(
- sizeof(THING) * (COUNT) * (STRIDE)
+ array3_size(COUNT, STRIDE, sizeof(THING))
, ...)
|
kzalloc(
- sizeof(THING) * (COUNT) * STRIDE
+ array3_size(COUNT, STRIDE, sizeof(THING))
, ...)
|
kzalloc(
- sizeof(THING) * COUNT * (STRIDE)
+ array3_size(COUNT, STRIDE, sizeof(THING))
, ...)
|
kzalloc(
- sizeof(THING) * COUNT * STRIDE
+ array3_size(COUNT, STRIDE, sizeof(THING))
, ...)
)

// 3-factor product with 2 sizeof(variable), with redundant parens removed.
@@
expression THING1, THING2;
identifier COUNT;
type TYPE1, TYPE2;
@@

(
kzalloc(
- sizeof(TYPE1) * sizeof(TYPE2) * COUNT
+ array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2))
, ...)
|
kzalloc(
- sizeof(TYPE1) * sizeof(THING2) * (COUNT)
+ array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2))
, ...)
|
kzalloc(
- sizeof(THING1) * sizeof(THING2) * COUNT
+ array3_size(COUNT, sizeof(THING1), sizeof(THING2))
, ...)
|
kzalloc(
- sizeof(THING1) * sizeof(THING2) * (COUNT)
+ array3_size(COUNT, sizeof(THING1), sizeof(THING2))
, ...)
|
kzalloc(
- sizeof(TYPE1) * sizeof(THING2) * COUNT
+ array3_size(COUNT, sizeof(TYPE1), sizeof(THING2))
, ...)
|
kzalloc(
- sizeof(TYPE1) * sizeof(THING2) * (COUNT)
+ array3_size(COUNT, sizeof(TYPE1), sizeof(THING2))
, ...)
)

// 3-factor product, only identifiers, with redundant parens removed.
@@
identifier STRIDE, SIZE, COUNT;
@@

(
kzalloc(
- (COUNT) * STRIDE * SIZE
+ array3_size(COUNT, STRIDE, SIZE)
, ...)
|
kzalloc(
- COUNT * (STRIDE) * SIZE
+ array3_size(COUNT, STRIDE, SIZE)
, ...)
|
kzalloc(
- COUNT * STRIDE * (SIZE)
+ array3_size(COUNT, STRIDE, SIZE)
, ...)
|
kzalloc(
- (COUNT) * (STRIDE) * SIZE
+ array3_size(COUNT, STRIDE, SIZE)
, ...)
|
kzalloc(
- COUNT * (STRIDE) * (SIZE)
+ array3_size(COUNT, STRIDE, SIZE)
, ...)
|
kzalloc(
- (COUNT) * STRIDE * (SIZE)
+ array3_size(COUNT, STRIDE, SIZE)
, ...)
|
kzalloc(
- (COUNT) * (STRIDE) * (SIZE)
+ array3_size(COUNT, STRIDE, SIZE)
, ...)
|
kzalloc(
- COUNT * STRIDE * SIZE
+ array3_size(COUNT, STRIDE, SIZE)
, ...)
)

// Any remaining multi-factor products, first at least 3-factor products,
// when they're not all constants...
@@
expression E1, E2, E3;
constant C1, C2, C3;
@@

(
kzalloc(C1 * C2 * C3, ...)
|
kzalloc(
- (E1) * E2 * E3
+ array3_size(E1, E2, E3)
, ...)
|
kzalloc(
- (E1) * (E2) * E3
+ array3_size(E1, E2, E3)
, ...)
|
kzalloc(
- (E1) * (E2) * (E3)
+ array3_size(E1, E2, E3)
, ...)
|
kzalloc(
- E1 * E2 * E3
+ array3_size(E1, E2, E3)
, ...)
)

// And then all remaining 2 factors products when they're not all constants,
// keeping sizeof() as the second factor argument.
@@
expression THING, E1, E2;
type TYPE;
constant C1, C2, C3;
@@

(
kzalloc(sizeof(THING) * C2, ...)
|
kzalloc(sizeof(TYPE) * C2, ...)
|
kzalloc(C1 * C2 * C3, ...)
|
kzalloc(C1 * C2, ...)
|
- kzalloc
+ kcalloc
(
- sizeof(TYPE) * (E2)
+ E2, sizeof(TYPE)
, ...)
|
- kzalloc
+ kcalloc
(
- sizeof(TYPE) * E2
+ E2, sizeof(TYPE)
, ...)
|
- kzalloc
+ kcalloc
(
- sizeof(THING) * (E2)
+ E2, sizeof(THING)
, ...)
|
- kzalloc
+ kcalloc
(
- sizeof(THING) * E2
+ E2, sizeof(THING)
, ...)
|
- kzalloc
+ kcalloc
(
- (E1) * E2
+ E1, E2
, ...)
|
- kzalloc
+ kcalloc
(
- (E1) * (E2)
+ E1, E2
, ...)
|
- kzalloc
+ kcalloc
(
- E1 * E2
+ E1, E2
, ...)
)

Signed-off-by: Kees Cook <keescook@chromium.org>

dmaengine: pl330: flush before wait, and add dev burst support.

Do DMAFLUSHP _before_ the first DMAWFP to ensure controller
and peripheral are in agreement about dma request state before first
transfer. Add support for burst transfers to/from peripherals. In the new
scheme, the controller does as many burst transfers as it can then
transfers the remaining dregs with either single transfers for
peripherals, or with a reduced size burst for memory-to-memory transfers.

Signed-off-by: Frank Mori Hess <fmh6jj@gmail.com>
Tested-by: Frank Mori Hess <fmh6jj@gmail.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: pl330: fix a race condition in case of threaded irqs

When booting up with "threadirqs" in command line, all irq handlers of the DMA
controller pl330 will be threaded forcedly. These threads will race for the same
list, pl330->req_done.

Before the callback, the spinlock was released. And after it, the spinlock was
taken. This opened an race window where another threaded irq handler could steal
the spinlock and be permitted to delete entries of the list, pl330->req_done.

If the later deleted an entry that was still referred to by the former, there would
be a kernel panic when the former was scheduled and tried to get the next sibling
of the deleted entry.

The scenario could be depicted as below:

Thread: T1 pl330->req_done Thread: T2
| | |
| -A-B-C-D- |
Locked | |
| | Waiting
Del A | |
| -B-C-D- |
Unlocked | |
| | Locked
Waiting | |
| | Del B
| | |
| -C-D- Unlocked
Waiting | |
|
Locked
|
get C via B
\
- Kernel panic

The kernel panic looked like as below:

Unable to handle kernel paging request at virtual address dead000000000108
pgd = ffffff8008c9e000
[dead000000000108] *pgd=000000027fffe003, *pud=000000027fffe003, *pmd=0000000000000000
Internal error: Oops: 96000044 [#1] PREEMPT SMP
Modules linked in:
CPU: 0 PID: 85 Comm: irq/59-66330000 Not tainted 4.8.24-WR9.0.0.12_standard #2
Hardware name: Broadcom NS2 SVK (DT)
task: ffffffc1f5cc3c00 task.stack: ffffffc1f5ce0000
PC is at pl330_irq_handler+0x27c/0x390
LR is at pl330_irq_handler+0x2a8/0x390
pc : [<ffffff80084cb694>] lr : [<ffffff80084cb6c0>] pstate: 800001c5
sp : ffffffc1f5ce3d00
x29: ffffffc1f5ce3d00 x28: 0000000000000140
x27: ffffffc1f5c530b0 x26: dead000000000100
x25: dead000000000200 x24: 0000000000418958
x23: 0000000000000001 x22: ffffffc1f5ccd668
x21: ffffffc1f5ccd590 x20: ffffffc1f5ccd418
x19: dead000000000060 x18: 0000000000000001
x17: 0000000000000007 x16: 0000000000000001
x15: ffffffffffffffff x14: ffffffffffffffff
x13: ffffffffffffffff x12: 0000000000000000
x11: 0000000000000001 x10: 0000000000000840
x9 : ffffffc1f5ce0000 x8 : ffffffc1f5cc3338
x7 : ffffff8008ce2020 x6 : 0000000000000000
x5 : 0000000000000000 x4 : 0000000000000001
x3 : dead000000000200 x2 : dead000000000100
x1 : 0000000000000140 x0 : ffffffc1f5ccd590

Process irq/59-66330000 (pid: 85, stack limit = 0xffffffc1f5ce0020)
Stack: (0xffffffc1f5ce3d00 to 0xffffffc1f5ce4000)
3d00: ffffffc1f5ce3d80 ffffff80080f09d0 ffffffc1f5ca0c00 ffffffc1f6f7c600
3d20: ffffffc1f5ce0000 ffffffc1f6f7c600 ffffffc1f5ca0c00 ffffff80080f0998
3d40: ffffffc1f5ce0000 ffffff80080f0000 0000000000000000 0000000000000000
3d60: ffffff8008ce202c ffffff8008ce2020 ffffffc1f5ccd668 ffffffc1f5c530b0
3d80: ffffffc1f5ce3db0 ffffff80080f0d70 ffffffc1f5ca0c40 0000000000000001
3da0: ffffffc1f5ce0000 ffffff80080f0cfc ffffffc1f5ce3e20 ffffff80080bf4f8
3dc0: ffffffc1f5ca0c80 ffffff8008bf3798 ffffff8008955528 ffffffc1f5ca0c00
3de0: ffffff80080f0c30 0000000000000000 0000000000000000 0000000000000000
3e00: 0000000000000000 0000000000000000 0000000000000000 ffffff80080f0b68
3e20: 0000000000000000 ffffff8008083690 ffffff80080bf420 ffffffc1f5ca0c80
3e40: 0000000000000000 0000000000000000 0000000000000000 ffffff80080cb648
3e60: ffffff8008b1c780 0000000000000000 0000000000000000 ffffffc1f5ca0c00
3e80: ffffffc100000000 ffffff8000000000 ffffffc1f5ce3e90 ffffffc1f5ce3e90
3ea0: 0000000000000000 ffffff8000000000 ffffffc1f5ce3eb0 ffffffc1f5ce3eb0
3ec0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
3ee0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
3f00: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
3f20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
3f40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
3f60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
3f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
3fa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
3fc0: 0000000000000000 0000000000000005 0000000000000000 0000000000000000
3fe0: 0000000000000000 0000000000000000 0000000275ce3ff0 0000000275ce3ff8
Call trace:
Exception stack(0xffffffc1f5ce3b30 to 0xffffffc1f5ce3c60)
3b20: dead000000000060 0000008000000000
3b40: ffffffc1f5ce3d00 ffffff80084cb694 0000000000000008 0000000000000e88
3b60: ffffffc1f5ce3bb0 ffffff80080dac68 ffffffc1f5ce3b90 ffffff8008826fe4
3b80: 00000000000001c0 00000000000001c0 ffffffc1f5ce3bb0 ffffff800848dfcc
3ba0: 0000000000020000 ffffff8008b15ae4 ffffffc1f5ce3c00 ffffff800808f000
3bc0: 0000000000000010 ffffff80088377f0 ffffffc1f5ccd590 0000000000000140
3be0: dead000000000100 dead000000000200 0000000000000001 0000000000000000
3c00: 0000000000000000 ffffff8008ce2020 ffffffc1f5cc3338 ffffffc1f5ce0000
3c20: 0000000000000840 0000000000000001 0000000000000000 ffffffffffffffff
3c40: ffffffffffffffff ffffffffffffffff 0000000000000001 0000000000000007
[<ffffff80084cb694>] pl330_irq_handler+0x27c/0x390
[<ffffff80080f09d0>] irq_forced_thread_fn+0x38/0x88
[<ffffff80080f0d70>] irq_thread+0x140/0x200
[<ffffff80080bf4f8>] kthread+0xd8/0xf0
[<ffffff8008083690>] ret_from_fork+0x10/0x40
Code: f2a00838 f9405763 aa1c03e1 aa1503e0 (f9000443)
---[ end trace f50005726d31199c ]---
Kernel panic - not syncing: Fatal exception in interrupt
SMP: stopping secondary CPUs
SMP: failed to stop secondary CPUs 0-1
Kernel Offset: disabled
Memory Limit: none
---[ end Kernel panic - not syncing: Fatal exception in interrupt

To fix this, re-start with the list-head after dropping the lock then
re-takeing it.

Reviewed-by: Frank Mori Hess <fmh6jj@gmail.com>
Tested-by: Frank Mori Hess <fmh6jj@gmail.com>
Signed-off-by: Qi Hou <qi.hou@windriver.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: fix descriptor allocation fail

If two concurrent threads call pl330_get_desc() when DMAC descriptor
pool is empty it is possible that allocation for one of threads will fail
with message:

kernel: dma-pl330 20078000.dma-controller: pl330_get_desc:2469 ALERT!

Here how that can happen. Thread A calls pl330_get_desc() to get
descriptor. If DMAC descriptor pool is empty pl330_get_desc() allocates
new descriptor on shared pool using add_desc() and then get newly
allocated descriptor using pluck_desc(). At the same time thread B calls
pluck_desc() and take newly allocated descriptor. In that case descriptor
allocation for thread A will fail.

Using on-stack pool for new descriptor allow avoid the issue described.
The patch modify pl330_get_desc() to use on-stack pool for allocation
new descriptors.

Signed-off-by: Alexander Kochetkov <al.kochet@gmail.com>
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: constify amba_id

amba_id are not supposed to change at runtime. All functions
working with const amba_id. So mark the non-const structs as const.

Signed-off-by: Arvind Yadav <arvind.yadav.cs@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Delete unused functions

The functions _queue_empty(), _emit_ADDH(), _emit_NOP(), _emit_STZ()
and _emit_WFE() are not used. Delete them.

Signed-off-by: Matthias Kaehlcke <mka@chromium.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: fix warning in pl330_remove

When removing a device with less than 9 IRQs (AMBA_NR_IRQS), we'll get a
big WARN_ON from devres.c because pl330_remove calls devm_free_irqs for
unallocated irqs. Similarly to pl330_probe, check that IRQ number is
present before calling devm_free_irq.

Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Add IOMMU support to slave tranfers

Wire up dma_map_resource() for slave transfers, so that we can let the
PL330 use IOMMU-backed DMA mapping ops on systems with an appropriate
IOMMU and RAM above 4GB, to avoid CPU bounce buffering.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: remove pdata based initialization

This driver is now used only on platforms which support device tree, so
it is safe to remove legacy platform data based initialization code.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Acked-by: Arnd Bergmann <arnd@arndb.de>
For plat-samsung:
Acked-by: Krzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: fix double lock

The static bug finder EBA (http://www.iagoabal.eu/eba/) reported the
following double-lock bug:

Double lock:
1. spin_lock_irqsave(pch->lock, flags) at pl330_free_chan_resources:2236;
2. call to function `pl330_release_channel' immediately after;
3. call to function `dma_pl330_rqcb' in line 1753;
4. spin_lock_irqsave(pch->lock, flags) at dma_pl330_rqcb:1505.

I have fixed it as suggested by Marek Szyprowski.

First, I have replaced `pch->lock' with `pl330->lock' in functions
`pl330_alloc_chan_resources' and `pl330_free_chan_resources'. This avoids
the double-lock by acquiring a different lock than `dma_pl330_rqcb'.

NOTE that, as a result, `pl330_free_chan_resources' executes
`list_splice_tail_init' on `pch->work_list' under lock `pl330->lock',
whereas in the rest of the code `pch->work_list' is protected by
`pch->lock'. I don't know if this may cause race conditions. Similarly
`pch->cyclic' is written by `pl330_alloc_chan_resources' under
`pl330->lock' but read by `pl330_tx_submit' under `pch->lock'.

Second, I have removed locking from `pl330_request_channel' and
`pl330_release_channel' functions. Function `pl330_request_channel' is
only called from `pl330_alloc_chan_resources', so the lock is already
held. Function `pl330_release_channel' is called from
`pl330_free_chan_resources', which already holds the lock, and from
`pl330_del'. Function `pl330_del' is called in an error path of
`pl330_probe' and at the end of `pl330_remove', but I assume that there
cannot be concurrent accesses to the protected data at those points.

Signed-off-by: Iago Abal <mail@iagoabal.eu>
Reviewed-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Make sure microcode is privileged

The PL330 is hard-wired such that instruction fetches on both the
manager and channel threads go out onto the bus with the "privileged"
bit set. This can become troublesome once there is an IOMMU or other
form of memory protection downstream, since those will typically be
programmed by the DMA mapping subsystem in the expectation of normal
unprivileged transactions (such as the PL330 channel threads' own data
accesses as currently configured by this driver).

To avoid the case of, say, an IOMMU blocking an unexpected privileged
transaction with a permission fault, use the newly-introduced
DMA_ATTR_PRIVILEGED attribute for the mapping of our microcode buffer.
That way the DMA layer can do whatever it needs to do to make things
continue to work as expected on more complex systems.

Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
[rm: remove now-redundant local variable, clarify commit message]
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>

dmaengine: pl330: Fix runtime PM support for terminated transfers

PL330 DMA engine driver is leaking a runtime reference after any terminated
DMA transactions. This patch fixes this issue by tracking runtime PM state
of the device and making additional call to pm_runtime_put() in terminate_all
callback if needed.

Fixes: ae43b3289186 ("ARM: 8202/1: dmaengine: pl330: Add runtime Power Management support v12")
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: Krzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: remove unused ‘regs’

In pl330_add(), variable ‘regs’ is initialized but never used, which
leads to warning with W=1.

drivers/dma/pl330.c: In function 'pl330_add':
drivers/dma/pl330.c:1891:16: warning: variable 'regs' set but not used [-Wunused-but-set-variable]

So remove it.

Cc: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: do not generate unaligned access

When PL330 is used with !MMU the following fault is seen:

Unhandled fault: alignment exception (0x801) at 0x8f26a002
Internal error: : 801 [#1] ARM
Modules linked in:
CPU: 0 PID: 640 Comm: dma0chan0-copy0 Not tainted 4.8.0-6a82063-clean+ #1600
Hardware name: ARM-Versatile Express
task: 8f1baa80 task.stack: 8e6fe000
PC is at _setup_req+0x4c/0x350
LR is at 0x8f2cbc00
pc : [<801ea538>] lr : [<8f2cbc00>] psr: 60000093
sp : 8e6ffdc0 ip : 00000000 fp : 00000000
r10: 00000000 r9 : 8f2cba10 r8 : 8f2cbc00
r7 : 80000013 r6 : 8f21a050 r5 : 8f21a000 r4 : 8f2ac800
r3 : 8e6ffe18 r2 : 00944251 r1 : ffffffbc r0 : 8f26a000
Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment none
Control: 00c5387c
Process dma0chan0-copy0 (pid: 640, stack limit = 0x8e6fe210)
Stack: (0x8e6ffdc0 to 0x8e700000)
fdc0: 00000001 60000093 00000000 8f2cba10 8f26a000 00000004 8f0ae000 8f2cbc00
fde0: 8f0ae000 8f2ac800 8f21a000 8f21a050 80000013 8f2cbc00 8f2cba10 00000000
fe00: 60000093 801ebca0 8e6ffe18 000013ff 40000093 00000000 00944251 8f2ac800
fe20: a0000013 8f2b1320 00001986 00000000 00000001 000013ff 8f1e4f00 8f2cba10
fe40: 8e6fff6c 801e9044 00000003 00000000 fef98c80 002faf07 8e6ffe7c 00000000
fe60: 00000002 00000000 00001986 8f1f158d 8f1e4f00 80568de4 00000002 00000000
fe80: 00001986 8f1f53ff 40000001 80580500 8f1f158d 8001e00c 00000000 cfdfdfdf
fea0: fdae2a25 00000001 00000004 8e6fe000 00000008 00000010 00000000 00000005
fec0: 8f2b1330 8f2b1334 8e6ffe80 8e6ffe8c 00001986 00000000 8f21a014 00000001
fee0: 8e6ffe60 8e6ffe78 00000002 00000000 000013ff 00000001 80568de4 8f1e8018
ff00: 0000158d 8055ec30 00000001 803f6b00 00001986 8f2cba10 fdae2a25 00000001
ff20: 8f1baca8 8e6fff24 8e6fff24 00000000 8e6fff24 ac6f3037 00000000 00000000
ff40: 00000000 8e6fe000 8f1e4f40 00000000 8f1e4f40 8f1e4f00 801e84ec 00000000
ff60: 00000000 00000000 00000000 80031714 dfdfdfcf 00000000 dfdfdfcf 8f1e4f00
ff80: 00000000 8e6fff84 8e6fff84 00000000 8e6fff90 8e6fff90 8e6fffac 8f1e4f40
ffa0: 80031640 00000000 00000000 8000f548 00000000 00000000 00000000 00000000
ffc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
ffe0: 00000000 00000000 00000000 00000000 00000013 00000000 dfdfdfcf cfdfdfdf
[<801ea538>] (_setup_req) from [<801ebca0>] (pl330_tasklet+0x41c/0x490)
[<801ebca0>] (pl330_tasklet) from [<801e9044>] (dmatest_func+0xb58/0x149c)
[<801e9044>] (dmatest_func) from [<80031714>] (kthread+0xd4/0xec)
[<80031714>] (kthread) from [<8000f548>] (ret_from_fork+0x14/0x2c)
Code: e3a03001 e3e01043 e5c03001 e59d3048 (e5802002)

This happens because _emit_{ADDH,MOV,GO) accessing to unaligned data
while writing to buffer. Fix it with writing to buffer byte by byte.

Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Handle xferred count if DMAMOV hasn't finished

After executing DMAGO it's possible that a request can come in for the
current xferred count, but if that happens too soon then DMAMOV SAR/DAR
may not have yet completed. If that happens, we should explicitly return 0
since nothing has been transferred yet.

Signed-off-by: Stephen Barber <smbarber@chromium.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Acquire dmac's spinlock in pl330_tx_status

There is a racing when accessing dmac thread in pl330_tx_status that
the pl330_update is handling active request at the same time and
changing the status of descriptors. This could cause an invalid
transferred count from BUSY descriptor added up to the residual number.
Fix the bug by using the dmac's spinlock in pl330_tx_status to protect
thread resources from changing.
Note that the nested order of holding dmac's and dma_chan's spinlock is
consistent with the rest of the driver: dma_chan first and then dmac,
so it is safe from deadlock scenario.

Signed-off-by: Hsin-Yu Chao <hychao@chromium.org>
Reviewed-by: Guenter Roeck <groeck@chromium.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: fix residual for non-running BUSY descriptors

Only one descriptor in the work list should be running at
any given time, but it's possible to have an enqueued BUSY
descriptor that has not yet transferred any data, or for
a BUSY descriptor to linger briefly before transitioning
to DONE. These cases should be handled to keep residual
calculations consistent even with the non-running BUSY
descriptors in the work list.

Signed-off-by: Stephen Barber <smbarber@chromium.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: convert callback to helper function

This is in preperation of moving to a callback that provides results to the
callback for the transaction. The conversion will maintain current behavior
and the driver must convert to new callback mechanism at a later time in
order to receive results.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: explicitly freeup irq

dmaengine device should explicitly call devm_free_irq() when using
devm_request_irq().

The irq is still ON when devices remove is executed and irq should be
quiesced before remove is completed.

Signed-off-by: Vinod Koul <vinod.koul@intel.com>
Cc: Jassi Brar <jassisinghbrar@gmail.com>
Cc: Linus Walleij <linus.walleij@linaro.org>

dmaengine: Remove site specific OOM error messages on kzalloc

If kzalloc() fails it will issue it's own error message including
a dump_stack(). So remove the site specific error messages.

Signed-off-by: Peter Griffin <peter.griffin@linaro.org>
Acked-by: Jon Hunter <jonathanh@nvidia.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: fix to support the burst mode

This patch fixes the burst mode that will break DMA uart on SoCFPGA.

In some cases, some SoCS didn't support the multi-burst
even if the devices who use the pl330 claim support the maxburst.

Fixes: commit 848e977
"dmaengine: pl330: support burst mode for dev-to-mem and mem-to-dev transmit"

Reported-by: Dinh Nguyen <dinguyen@opensource.altera.com>
Signed-off-by: Caesar Wang <wxt@rock-chips.com>
Tested-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Tested-by: Dinh Nguyen <dinguyen@opensource.altera.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: add max burst for dmaengine

This patch add max burst capability for dmaengine and
limit burst capability to one for PL330_QUIRK_BROKEN_NO_FLUSHP

Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: Caesar Wang <wxt@rock-chips.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: add quirk for broken no flushp

This patch add "arm,pl330-broken-no-flushp" quirk to avoid execute
DMAFLUSHP if Soc doesn't support it.

Signed-off-by: Addy Ke <addy.ke@rock-chips.com>
Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com>
cc: Doug Anderson <dianders@chromium.org>
cc: Heiko Stuebner <heiko@sntech.de>
cc: Olof Johansson <olof@lixom.net>

Reviewed-by: Sonny Rao <sonnyrao@chromium.org>
Signed-off-by: Caesar Wang <wxt@rock-chips.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: support burst mode for dev-to-mem and mem-to-dev transmit

This patch adds to support burst mode for dev-to-mem and
mem-to-dev transmit.

Signed-off-by: Boojin Kim <boojin.kim@samsung.com>
Signed-off-by: Addy Ke <addy.ke@rock-chips.com>
Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com>
cc: Heiko Stuebner <heiko@sntech.de>
cc: Doug Anderson <dianders@chromium.org>
cc: Olof Johansson <olof@lixom.net>

Reviewed-by: Sonny Rao <sonnyrao@chromium.org>
Signed-off-by: Caesar Wang <wxt@rock-chips.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: do not emit loop for 1 byte transfer.

When there is only one burst required do not emit loop instructions to
loop exactly once. Emit just the body of the loop.

Signed-off-by: Michal Suchanek <hramrach@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Really fix choppy sound because of wrong residue calculation

When pl330 driver was used during sound playback, after some time or
after a number of plays the sound became choppy or totally noisy. For
example on Odroid XU3 board the first four executions of aplay with
small WAVE worked fine, but fifth was unrecognizable with errors:
$ aplay /usr/share/sounds/alsa/Front_Right.wava
underrun!!! (at least 0.095 ms long)

Issue was caused by wrong residue reported by pl330 driver to
pcm_dmaengine for its cyclic dma transfers.

The pl330_tx_status(), residue reporting function, used a "last" flag in
a descriptor to indicate that there is no more data to send.

The pl330_tx_submit() iterated over descriptors trying to remove this
flag from them and then mark last descriptor as "last". However when
iterating it actually removed the flag not from descriptors but always
from last of it (and then reset it). Thus effectively once some
descriptor was marked as last, then it stayed like this forever causing
residue to be reported too low.

Signed-off-by: Krzysztof Kozlowski <k.kozlowski.k@gmail.com>
Fixes: aee4d1fac887 ("dmaengine: pl330: improve pl330_tx_status() function")
Cc: <stable@vger.kernel.org>
Reported-by: gabriel@unseen.is
Suggested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Tested-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Fix overflow when reporting residue in memcpy

During memcpy operations the residue was always set to an u32 overflowed
value.

In pl330_tx_status() function number of currently transferred bytes was
subtracted from internal "bytes_requested" field. However this
"bytes_requested" was not initialized at start to length of memcpy
buffer so transferred bytes were subtracted from 0 causing overflow.

Signed-off-by: Krzysztof Kozlowski <k.kozlowski@samsung.com>
Cc: <stable@vger.kernel.org>
Fixes: aee4d1fac887 ("dmaengine: pl330: improve pl330_tx_status() function")
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: fix wording in mcbufsz message

The kernel is not trying to increase mcbufsz. It suggests you should try
doing so. Also print the calculated required size of mcbufsz.

Signed-off-by: Michal Suchanek <hramrach@gmail.com>
Reviewed-by: Krzysztof Kozlowski <k.kozlowski@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Initialize pl330 for pl330_prep_dma_memcpy after NULL check of pch

Currently pch pointer is already dereferenced before NULL check
and thus we are getting below warning:
warn: variable dereferenced before check 'pch'

So initialize struct pl330_dmac *pl330 after NULL check
of dma_pl330_chan *pch.

Signed-off-by: Maninder Singh <maninder1.s@samsung.com>
Reviewed-by: Vaneet Narang <v.narang@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Fix hang on dmaengine_terminate_all on certain boards

The pl330 device could hang infinitely on certain boards when DMA
channels are terminated.

It was caused by lack of runtime resume when executing
pl330_terminate_all() which calls the _stop() function. _stop() accesses
device register and can loop infinitely while checking for device state.

The hang was confirmed by Dinh Nguyen on Altera SOCFPGA Cyclone V
board during boot. It can be also triggered with:

$ echo 1 > /sys/module/dmatest/parameters/iterations
$ echo dma1chan0 > /sys/module/dmatest/parameters/channel
$ echo 1 > /sys/module/dmatest/parameters/run
$ sleep 1
$ cat /sys/module/dmatest/parameters/run

Reported-by: Dinh Nguyen <dinguyen@opensource.altera.com>
Signed-off-by: Krzysztof Kozlowski <k.kozlowski@samsung.com>
Fixes: ae43b3289186 ("ARM: 8202/1: dmaengine: pl330: Add runtime Power Management support v12")
Cc: <stable@vger.kernel.org>
Tested-by: Dinh Nguyen <dinguyen@opensource.altera.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: fix return status on pending transfers

The pl330_tx_status() function returns the desc->status if the
dma_cookie_status() call does indicate the cookie completed,
however the desc->status is not look directly compatible. Sparse
throws the following warning:

pl330.c:2262:35: warning: mixing different enum types
pl330.c:2262:35: int enum desc_status versus
pl330.c:2262:35: int enum dma_status

Attempt to fix this by adding a switch statement to turn the
desc->status into a dma_status.

Note, this has only been tested with the dmatest suite.

Signed-off-by: Ben Dooks <ben.dooks@codethink.co.uk>
--
Vinod Koul <vinod.koul@intel.com>
Dan Williams <dan.j.williams@intel.com>
DMA List <dmaengine@vger.kernel.org>
Maxime Ripard <maxime.ripard@free-electrons.com>
Jassi Brar <jassisinghbrar@gmail.com>
Liviu Dudau <Liviu.Dudau@arm.com>
Linux ARM Kernel <linux-arm-kernel@lists.infradead.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: make unexported functions static

Whilst running sparse on pl330 driver it was noticed there are
two functions that are not static but not exported to any other
users in the kernel.

Fix the following warnings by making 'pl330_pause' and the
'pl330_get_current_xferred_count' static:

pl330.c:2165:5: warning: symbol 'pl330_pause' was not declared. Should it be static?
pl330.c:2206:5: warning: symbol 'pl330_get_current_xferred_count' was not declared. Should it be static?

Signed-off-by: Ben Dooks <ben.dooks@codethink.co.uk>
--
Vinod Koul <vinod.koul@intel.com>
Dan Williams <dan.j.williams@intel.com>
DMA List <dmaengine@vger.kernel.org>
Maxime Ripard <maxime.ripard@free-electrons.com>
Jassi Brar <jassisinghbrar@gmail.com>
Liviu Dudau <Liviu.Dudau@arm.com>
Linux ARM Kernel <linux-arm-kernel@lists.infradead.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: fix issues with big-endian armv7

When running Xilinx Zynq in big-endian mode the pl330 driver
fails to pass the dmatest suite. To fix this, ensure all non
byte values are written in little endian.

As a note, the documentation does not mention if it will do
big-endian descriptor fetches, only that it will swap the
data in flight.

Signed-off-by: Ben Dooks <ben.dooks@codethink.co.uk>
--
Vinod Koul <vinod.koul@intel.com>
Dan Williams <dan.j.williams@intel.com>
DMA List <dmaengine@vger.kernel.org>
Maxime Ripard <maxime.ripard@free-electrons.com>
Jassi Brar <jassisinghbrar@gmail.com>
Liviu Dudau <Liviu.Dudau@arm.com>
Linux ARM Kernel <linux-arm-kernel@lists.infradead.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: add DMA_PAUSE feature

DMA_PAUSE command is used for halting DMA transfer on chosen channel.
It can be useful when we want to safely read residue before terminating
all requests on channel. Otherwise there can be situation when some data
is transferred before channel termination but after reading residue,
which obviously results with data loss. To avoid this situation we can
pause channel, read residue and then terminate all requests.
This scenario is common, for example, in serial port drivers.

Signed-off-by: Robert Baldyga <r.baldyga@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: improve pl330_tx_status() function

This patch adds possibility to read residue of DMA transfer. It's useful
when we want to know how many bytes have been transferred before we
terminate channel. It can take place, for example, on timeout interrupt.

Signed-off-by: Lukasz Czerwinski <l.czerwinski@samsung.com>
Signed-off-by: Robert Baldyga <r.baldyga@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: fix bug that cause start the same descs in cyclic

This bug will cause NULL pointer after commit dfac17, and cause
wrong package in I2S DMA transfer before commit dfac17.

Tested on RK3288-pinky2 board.

Detail:
I2S DMA transfer(sound/core/pcm_dmaengine.c):
dmaengine_pcm_prepare_and_submit -->
dmaengine_prep_dma_cyclic -->
pl330_prep_dma_cyclic -->
the case:
1. pl330_submit_req(desc0): thrd->req[0].desc = desc0, thrd->lstenq = 0
2. pl330_submit_req(desc1): thrd->req[1].desc = desc1, thrd->lstenq = 1
3. _start(desc0) by submit_req: thrd->req_running = 0
because: idx = 1 - thrd->lstenq = 0
4. pl330_update(desc0 OK): thrd->req[0].desc = NULL, desc0 to req_done list
because: idx = active = thrd->req_running = 0
5. _start(desc1) by pl330_update: thrd->req_running = 1
because:
idx = 1 - thrd->lstenq = 0, but thrd->req[0].desc == NULL,
so:
idx = thrd->lstenq = 1
6. pl330_submit_req(desc2): thrd->req[0].desc = desc2, thrd->lstenq = 0
7. _start(desc1) by submit_req: thrd->req_running = 1
because: idx = 1 - thrd->lstenq = 1
Note: _start started the same descs
_start should start desc2 here, NOT desc1

8. pl330_update(desc1 OK): thrd->req[1].desc = NULL, desc1 to req_done list
because: idx = active = thrd->req_running = 1
9. _start(desc2) by pl330_update : thrd->req_running = 0
because: idx = 1 - thrd->lstenq = 0
10.pl330_update(desc1 OK, NOT desc2): thrd->req[0].desc = NULL,
desc2 to req_done list
because: idx = active = thrd->req_running = 0

11.pl330_submit_req(desc3): thrd->req[0].desc = desc3, thrd->lstenq = 0
12.pl330_submit_req(desc4): thrd->req[1].desc = desc4, thrd->lstenq = 1
13._start(desc3) by submit_req: thrd->req_running = 0
because: idx = 1 - thrd->lstenq = 0
14.pl330_update(desc2 OK NOT desc3): thrd->req[0].desc = NULL
desc3 to req_done list
because: idx = active = thrd->req_running = 0
15._start(desc4) by pl330_update: thrd->req_running = 1
because:
idx = 1 - thrd->lstenq = 0, but thrd->req[0].desc == NULL,
so:
idx = thrd->lstenq = 1
16.pl330_submit_req(desc5): thrd->req[0].desc = desc5, thrd->lstenq = 0
17._start(desc4) by submit_req: thrd->req_running = 1
because: idx = 1 - thrd->lstenq = 1
18.pl330_update(desc3 OK NOT desc4): thrd->req[1].desc = NULL
desc4 to req_done list
because: idx = active = thrd->req_running = 1
19._start(desc4) by pl330_update: thrd->req_running = 0
because:
idx = 1 - thrd->lstenq = 1, but thrd->req[1].desc == NULL,
so:
idx = thrd->lstenq = 0
20.pl330_update(desc4 OK): thrd->req[0].desc = NULL, desc5 to req_done list
because: idx = active = thrd->req_running = 0
21.pl330_update(desc4 OK):
1) before commit dfac17(set req_running -1 in pl330_update/mark_free()):
because: active = -1, abort
result: desc0-desc5's callback are all called,
but step 10 and step 18 go wrong.
2) before commit dfac17:
idx = active = thrd->req_runnig = 0 -->
descdone = thrd->req[0] = NULL -->
list_add_tail(&descdone->rqd, &pl330->req_done); -->
got NULL pointer!!!

Signed-off-by: Addy Ke <addy.ke@rock-chips.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Declare slave capabilities for the generic code

Now that the generic slave caps code can make use of the device assigned
capabilities, instead of relying on a callback to be implemented.

Make use of this code.

Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Split device_control

Split the device_control callback of the AMBA PL330 DMA driver to make use
of the newly introduced callbacks, that will eventually be used to retrieve
slave capabilities.

Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: Make the destination abbreviation coherent

The dmaengine header abbreviates destination as at least two different strings.
Make a coherent use of a single one.

Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: Mark Brown <broonie@kernel.org>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: update author info

Update email-id to a personal one, as I have changed employment.

Signed-off-by: Jassi Brar <jassisinghbrar@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

ARM: 8206/1: dmaengine: pl330: Add PM sleep support

Add system suspend/resume capabilities to the pl330 driver so the amba
bus clock could be also unprepared to conserve energy.

Signed-off-by: Krzysztof Kozlowski <k.kozlowski@samsung.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: 8202/1: dmaengine: pl330: Add runtime Power Management support v12

This patch adds runtime PM support to pl330 DMA engine driver.

The runtime power management for pl330 DMA driver allows gating of AMBA
clock (PDMA) in FSYS clock domain, when the device is not processing any
requests. This is necessary to enter low power modes on Exynos SoCs
(e.g. LPA on Exynos4x12 or W-AFTR on Exynos3250).

Runtime PM resuming of the device may happen in atomic context (during
call device_issue_pending()) so pm_runtime_irq_safe() is used. This will
lead only to disabling/enabling of the clock but this is sufficient for
gating the clock and for reducing energy usage.

Driver uses runtime PM callbacks from amba/bus.c driver only.

Suggested-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Krzysztof Kozlowski <k.kozlowski@samsung.com>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

dmaengine: pl330: Correct device assignment.

Commit f6f2421c0a1c removed pl330_info structure by embedding it into
pl330_dmac structure, but did not ensure that the dmac->ddma.dev
pointer gets initialised before use. When dma_alloc_coherent() gets
called on arm64 a WARN() gets triggered due to dev being NULL.

------------[ cut here ]------------
WARNING: CPU: 1 PID: 1 at arch/arm64/mm/dma-mapping.c:49 __dma_alloc_coherent+0xd0/0xe0()
Use an actual device structure for DMA allocation
Modules linked in:
CPU: 1 PID: 1 Comm: swapper/0 Not tainted 3.17.0+ #5
Call trace:
[<ffffffc000087f24>] dump_backtrace+0x0/0x130
[<ffffffc000088064>] show_stack+0x10/0x1c
[<ffffffc0004e8af8>] dump_stack+0x74/0xb8
[<ffffffc0000aa444>] warn_slowpath_common+0x8c/0xb4
[<ffffffc0000aa4b8>] warn_slowpath_fmt+0x4c/0x58
[<ffffffc000092580>] __dma_alloc_coherent+0xcc/0xe0
[<ffffffc000092734>] __dma_alloc_noncoherent+0x64/0x158
[<ffffffc000312cd8>] pl330_probe+0x650/0x8f0
[<ffffffc00030e1d4>] amba_probe+0xa0/0xc8
[<ffffffc000350240>] really_probe+0xc4/0x22c
[<ffffffc0003504b4>] __driver_attach+0xa0/0xa8
[<ffffffc00034e5fc>] bus_for_each_dev+0x54/0x98
[<ffffffc00034fd8c>] driver_attach+0x1c/0x28
[<ffffffc00034fa08>] bus_add_driver+0x14c/0x204
[<ffffffc000350b84>] driver_register+0x64/0x130
[<ffffffc00030dcf8>] amba_driver_register+0x50/0x5c
[<ffffffc0006a60d0>] pl330_driver_init+0x10/0x1c
[<ffffffc0000814ac>] do_one_initcall+0x88/0x19c
[<ffffffc00068dab8>] kernel_init_freeable+0x140/0x1e0
[<ffffffc0004e5e18>] kernel_init+0x10/0xd4
---[ end trace 76f2d47a444e523e ]---
(NULL device *): dmac_alloc_resources:1821 Can't allocate memory!
(NULL device *): Unable to create channels for DMAC

This patch will also ensure that any dev_err messages are printed
with the appropriate device name.

Signed-off-by: Liviu Dudau <Liviu.Dudau@arm.com>
Signed-off-by: Andrew Jackson <Andrew.Jackson@arm.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: Fix allocation size for PL330 data buffer depth.

The datasheet for PL330 says that the data buffer value in the CRD
register is 10bits wide. However, the value stored is "minus one",
which the driver corrects for. Maximum value that the data buffer
depth can have is 1024 lines, which requires 11 bits for storage.

While making updates I found printing the peripheral ID as a hex
value to be more useful as the datasheet shows the values that way.

Signed-off-by: Liviu Dudau <Liviu.Dudau@arm.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Limit MFIFO usage for memcpy to avoid exhausting entries

The MFIFO is shared by all channels so restrict each memcpy to it's fair
share. This is being over cautious, but without a global view of DMA
channel usage on a system it's not possible to come up with a more
optimum safe limit.

Signed-off-by: Jon Medhurst <tixy@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Align DMA memcpy operations to MFIFO width

The algorithm used for programming the DMA Controller doesn't take into
consideration the requirements of transfers that are not aligned to the
bus width. This failure may result in DMA transferring one too few MFIFO
entries (so too few bytes are copied) or the DMA trying to write one too
many MFIFO entries and hanging because this is never provided.

See "MFIFO Usage Overview" chapter in the the TRM for "CoreLink DMA
Controller DMA-330", Revision r1p1.

We work around these shortcomings by making sure we pick a burst size
and length which ensures no bursts straddle an MFIFO entry.

Signed-off-by: Jon Medhurst <tixy@linaro.org>
[squashed linker error "undefined reference to `__aeabi_uldivmod]
Reported-by: kbuild test robot <fengguang.wu@intel.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Fix NULL pointer dereference on driver unbind

Fix a NULL pointer dereference after unbinding the driver, if channel
resources were not yet allocated (no call to
pl330_alloc_chan_resources()):
$ echo 12850000.mdma > /sys/bus/amba/drivers/dma-pl330/unbind
[ 13.606533] DMA pl330_control: removing pch: eeab6800, chan: eeab6814, thread: (null)
[ 13.614472] Unable to handle kernel NULL pointer dereference at virtual address 0000000c
[ 13.622537] pgd = ee284000
[ 13.625228] [0000000c] *pgd=6e1e4831, *pte=00000000, *ppte=00000000
[ 13.631482] Internal error: Oops: 17 [#1] PREEMPT SMP ARM
[ 13.636859] Modules linked in:
[ 13.639903] CPU: 0 PID: 1 Comm: sh Not tainted 3.17.0-rc3-next-20140904-00004-g7020ffc33ca3-dirty #420
[ 13.649187] task: ee80a800 ti: ee888000 task.ti: ee888000
[ 13.654589] PC is at _stop+0x8/0x2c8
[ 13.658131] LR is at pl330_control+0x70/0x2e8
[ 13.662468] pc : [<c0206028>] lr : [<c020649c>] psr: 60000093
[ 13.662468] sp : ee889e58 ip : 00000001 fp : 000bab70
[ 13.673922] r10: eeab6814 r9 : ee16debc r8 : 00000000
[ 13.679131] r7 : eeab685c r6 : 60000013 r5 : ee16de10 r4 : eeab6800
[ 13.685641] r3 : 00000002 r2 : 00000000 r1 : 00010000 r0 : 00000000
[ 13.692153] Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment user
[ 13.699357] Control: 10c5387d Table: 6e28404a DAC: 00000015
[ 13.705085] Process sh (pid: 1, stack limit = 0xee888240)
[ 13.710466] Stack: (0xee889e58 to 0xee88a000)
[ 13.714808] 9e40: 00000002 eeab6800
[ 13.722969] 9e60: ee16de10 eeab6800 ee16de10 60000013 eeab685c c020649c 00000000 c040280c
[ 13.731128] 9e80: ee889e80 ee889e80 ee16de18 ee16de10 eeab6880 eeab6814 00200200 eeab68a8
[ 13.739287] 9ea0: 00100100 c0208048 00000000 c0409fc4 eea80800 eea808f8 c0605c44 0000000e
[ 13.747446] 9ec0: 0000000e eeb3960c eeb39600 c0203c48 eea80800 c0605c44 c0605a8c c023f694
[ 13.755605] 9ee0: ee80a800 eea80834 eea80800 c023f704 ee80a800 eea80800 c0605c44 c023e8ec
[ 13.763764] 9f00: 0000000e ee149780 ee29e580 ee889f80 ee29e580 c023e19c 0000000e c01167e4
[ 13.771923] 9f20: c01167a0 00000000 00000000 c0115e88 00000000 00000000 ee0b1a00 0000000e
[ 13.780082] 9f40: b6f48000 ee889f80 0000000e ee888000 b6f48000 c00bfadc 00000000 00000003
[ 13.788241] 9f60: 00000000 00000000 00000000 ee0b1a00 ee0b1a00 0000000e b6f48000 c00bfdf4
[ 13.796401] 9f80: 00000000 00000000 ffffffff 0000000e b6f48000 b6edc5d0 00000004 c000e7a4
[ 13.804560] 9fa0: 00000000 c000e620 0000000e b6f48000 00000001 b6f48000 0000000e 00000000
[ 13.812719] 9fc0: 0000000e b6f48000 b6edc5d0 00000004 0000000e b6f4c8c0 000c3470 000bab70
[ 13.820879] 9fe0: 00000000 bed2aa50 b6e18bdc b6e6b52c 60000010 00000001 c0c0c0c0 c0c0c0c0
[ 13.829058] [<c0206028>] (_stop) from [<c020649c>] (pl330_control+0x70/0x2e8)
[ 13.836165] [<c020649c>] (pl330_control) from [<c0208048>] (pl330_remove+0xb0/0xdc)
[ 13.843800] [<c0208048>] (pl330_remove) from [<c0203c48>] (amba_remove+0x24/0xc0)
[ 13.851272] [<c0203c48>] (amba_remove) from [<c023f694>] (__device_release_driver+0x70/0xc4)
[ 13.859685] [<c023f694>] (__device_release_driver) from [<c023f704>] (device_release_driver+0x1c/0x28)
[ 13.868971] [<c023f704>] (device_release_driver) from [<c023e8ec>] (unbind_store+0x58/0x90)
[ 13.877303] [<c023e8ec>] (unbind_store) from [<c023e19c>] (drv_attr_store+0x20/0x2c)
[ 13.885036] [<c023e19c>] (drv_attr_store) from [<c01167e4>] (sysfs_kf_write+0x44/0x48)
[ 13.892928] [<c01167e4>] (sysfs_kf_write) from [<c0115e88>] (kernfs_fop_write+0xc0/0x17c)
[ 13.901090] [<c0115e88>] (kernfs_fop_write) from [<c00bfadc>] (vfs_write+0xa0/0x1a8)
[ 13.908812] [<c00bfadc>] (vfs_write) from [<c00bfdf4>] (SyS_write+0x40/0x8c)
[ 13.915850] [<c00bfdf4>] (SyS_write) from [<c000e620>] (ret_fast_syscall+0x0/0x30)
[ 13.923392] Code: e5813010 e12fff1e e92d40f0 e24dd00c (e590200c)
[ 13.929467] ---[ end trace 10064e15a5929cf8 ]---

Terminate the thread and free channel resource only if channel resources
were allocated (thread is not NULL).

Signed-off-by: Krzysztof Kozlowski <k.kozlowski@samsung.com>
Cc: <stable@vger.kernel.org>
Fixes: b3040e40675e ("DMA: PL330: Add dma api driver")
Reviewed-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Fix NULL pointer dereference on probe failure

If dma_async_device_register() returns error and probe should clean up
and return error, a NULL pointer exception happens because of
dereference of not allocated channel thread:

Dmesg log (from early printk):
dma-pl330 12680000.pdma: unable to register DMAC
DMA pl330_control: removing pch: eeac4000, chan: eeac4014, thread: (null)
Unable to handle kernel NULL pointer dereference at virtual address 0000000c
pgd = c0004000
[0000000c] *pgd=00000000
Internal error: Oops: 5 [#1] PREEMPT SMP ARM
Modules linked in:
CPU: 2 PID: 1 Comm: swapper/0 Not tainted 3.17.0-rc3-next-20140904-00005-g6cc4c1937d90-dirty #427
task: ee80a800 ti: ee888000 task.ti: ee888000
PC is at _stop+0x8/0x2c8
LR is at pl330_control+0x70/0x2e8
pc : [<c0205dc8>] lr : [<c020623c>] psr: 60000193
sp : ee889df8 ip : 00000002 fp : 00000000
r10: eeac4014 r9 : ee0e62bc r8 : 00000000
r7 : eeac405c r6 : 60000113 r5 : ee0e6210 r4 : eeac4000
r3 : 00000002 r2 : 00000002 r1 : 00010000 r0 : 00000000
Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel
Control: 10c5387d Table: 4000404a DAC: 00000015
Process swapper/0 (pid: 1, stack limit = 0xee888240)
Stack: (0xee889df8 to 0xee88a000)
9de0: 00000002 eeac4000
9e00: ee0e6210 eeac4000 ee0e6210 60000113 eeac405c c020623c 00000000 c020725c
9e20: ee889e20 ee889e20 ee0e6210 eeac4080 00200200 00100100 eeac4014 00000020
9e40: ee0e6218 c0208374 00000000 ee9bb340 ee0e6210 00000000 00000000 c0605cd8
9e60: ee970000 c0605c84 ee9700f8 00000000 c05c4270 00000000 00000000 c0203b3c
9e80: ee970000 c06624a8 00000000 c0605c84 00000000 c023f890 ee970000 c0605c84
9ea0: ee970034 00000000 c05b23d0 c023fa3c 00000000 c0605c84 c023f9b0 c023e0d4
9ec0: ee947e78 ee9b9440 c0605c84 eea1e780 c0605acc c023f094 c0513b50 c0605c84
9ee0: c05ecbd8 c0605c84 c05ecbd8 ee11ba40 c0626500 c0240064 00000000 c05ecbd8
9f00: c05ecbd8 c0008964 c040f13c 0000009f c0626500 c057465c ee80a800 60000113
9f20: 00000000 c05efdb0 60000113 00000000 ef7fc89d c0421168 0000008f c003787c
9f40: c0573d6c 00000006 ef7fc8bb 00000006 c05efd50 ef7fc800 c05dfbc4 00000006
9f60: c05c4264 c0626500 0000008f c05c4270 c059b518 c059bcb4 00000006 00000006
9f80: c059b518 c003c08c 00000000 c040091c 00000000 00000000 00000000 00000000
9fa0: 00000000 c0400924 00000000 c000e7b8 00000000 00000000 00000000 00000000
9fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
9fe0: 00000000 00000000 00000000 00000000 00000013 00000000 c0c0c0c0 c0c0c0c0
[<c0205dc8>] (_stop) from [<c020623c>] (pl330_control+0x70/0x2e8)
[<c020623c>] (pl330_control) from [<c0208374>] (pl330_probe+0x594/0x75c)
[<c0208374>] (pl330_probe) from [<c0203b3c>] (amba_probe+0xb8/0x120)
[<c0203b3c>] (amba_probe) from [<c023f890>] (driver_probe_device+0x10c/0x22c)
[<c023f890>] (driver_probe_device) from [<c023fa3c>] (__driver_attach+0x8c/0x90)
[<c023fa3c>] (__driver_attach) from [<c023e0d4>] (bus_for_each_dev+0x54/0x88)
[<c023e0d4>] (bus_for_each_dev) from [<c023f094>] (bus_add_driver+0xd4/0x1d0)
[<c023f094>] (bus_add_driver) from [<c0240064>] (driver_register+0x78/0xf4)
[<c0240064>] (driver_register) from [<c0008964>] (do_one_initcall+0x80/0x1d0)
[<c0008964>] (do_one_initcall) from [<c059bcb4>] (kernel_init_freeable+0x108/0x1d4)
[<c059bcb4>] (kernel_init_freeable) from [<c0400924>] (kernel_init+0x8/0xec)
[<c0400924>] (kernel_init) from [<c000e7b8>] (ret_from_fork+0x14/0x3c)
Code: e5813010 e12fff1e e92d40f0 e24dd00c (e590200c)
---[ end trace c94b2f4f38dff3bf ]---

This happens because the necessary resources were not yet allocated - no
call to pl330_alloc_chan_resources().

Terminate the thread and free channel resource only if channel thread is not NULL.

Signed-off-by: Krzysztof Kozlowski <k.kozlowski@samsung.com>
Cc: <stable@vger.kernel.org>
Fixes: 0b94c5771705 ("DMA: PL330: Add check if device tree compatible")
Reviewed-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Remove unused 'regs' variable in pl330_submit_req()

The 'void __iomem *regs' is not used in pl330_submit_req() function.
Remove it.

Signed-off-by: Krzysztof Kozlowski <k.kozlowski@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Remove non-NULL check for pl330_submit_req parameters

The pl330_submit_req() checked supplied 'struct pl330_thread thrd' and
'struct dma_pl330_desc desc' parameters for non-NULL. However these
checks are useless because supplied arguments won't be NULL.

The pl330_submit_req() is called in only one place and:
1. 'desc' is already dereferenced in fill_queue() before calling
pl330_submit_req().
2. 'thrd' is always dereferenced after calling
fill_queue()->pl330_submit_req().

Removing the checks for non-NULL values fixes following warning:
drivers/dma/pl330.c:1376 pl330_submit_req() warn: variable dereferenced before check 'thrd' (see line 1367)

Signed-off-by: Krzysztof Kozlowski <k.kozlowski@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: Remove the context argument to the prep_dma_cyclic operation

The argument is always set to NULL and never used. Remove it.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Check if the DMA descriptor is NULL

Commit 6079d38 ("dmaengine: pl330: Remove useless xfer_cb indirection")
removed the __callback() function which created an unnecessary level of
indirection to execute the tranfer callback .xfer_cb

Unfortunately the commit also changed the semantics slightly since that
function used to check if the request was not NULL before attempting to
execute the callback function. Not checking this could lead to a kernel
NULL pointer dereference error.

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
Acked-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Remove pl330_chan_ctrl()

The pl330_chan_ctrl() function has 3 internal code paths which, except for the
locking, do not share any code outside of their sections. One code path is never
exercised and can be removed. The other two are mostly just forwards to the
_start() and _stop() calls. This patch modifies the code to instead of going via
pl330_chan_ctrl() to call _start() and _stop() directly. This allows to
completely remove pl330_chan_ctrl().

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Simplify marking a request as unused

Instead of storing a special instruction in the command buffer to mark a request
as currently unused just set the descriptor field to NULL.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Embed pl330_req directly into dma_pl330_desc

The pl330_req struct is embedded into the dma_pl330_desc struct. But half of the
pl330_req struct are pointers to other fields of the dma_pl330_desc struct it is
embedded to. By directly embedding the fields from the pl330_req struct into the
dma_pl330_desc struct and reworking the code to work with the dma_pl330_desc
struct those pointers can be eliminated. This slightly simplifies the code.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Merge dma_pl330_dmac and pl330_dmac structs

Both the dma_pl330_dmac and the pl330_dmac struct have the same lifetime and the
separation of them is a relict of this having been two different drivers in the
past. Merging them into one struct makes the code a bit simpler as it for
example allows to remove the pointers going back and forth between the two
structs.

While we are at it also directly embed the pl330_info struct into the
pl330_dmac struct as this allows to remove some more redundant fields.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Simplify is_manager()

Since we keep a pointer to the manager thread it is fairly easy to check if a
thread is the manager thread.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Remove uneccessary ccr validation

We know that we do not create invalid ccr settings in this driver. There is no
need to validate them.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Change type pl330_chid from void * to struct pl330_thread *

The pl330_chid field of the dma_pl330_chan struct always holds a pointer to the
thread that is associated with the channel. Changing its type form void * to
struct pl330_thread makes things more type safe and removes the need for
unnecessary typecasts. While we are at it also rename the field from the cryptic
pl330_chid to thread.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Remove useless xfer_cb indirection

The xfer_cb callback of the pl330_req struct is always set to the same function.
This adds an unnecessary step of indirection. Instead just call the callback
function directly.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Remove unused mc_len field from _pl330_req struct

The mc_len is initialized but its value is never read again, so remove it.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Remove unused next field form pl330_xfer struct

The next field is always NULL, so we can remove it.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Remove unused client_data field form pl330_info

The field is completely unused, remove it.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Remove unused dmac_reset callback

The dmac_reset() callaback of the pl330_info struct is always set to NULL, so
remove it.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Remove unused pl330_chanstatus struct

The pl330_chanstatus struct is completely unused, so remove it.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Remove duplicated cachecontrol enum

The settings for destination and source cache control are exactly the same. This
patch removes the duplicated enum and uses the same for both destination and
source cache control.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: Use dma_transfer_direction instead of custom pl330_reqtype

The pl330 driver has the custom pl330_reqtype enum which has the same possible
settings as the generic dma_transfer_direction enum. Switching over to the
generic enum internally makes it possible to directly initialize it from the
transfer request direction.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: pl330: Use dma_get_slave_channel() in the of xlate callback

Currently the driver uses dma_request_channel() with a custom filter function to
find the requested channel. This will loop over all available channels until the
one we want has been found, but we already know which channel we want to
request, so we can dma_get_slave_channel(). This also makes the code a bit
shorter cleaner.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: pl330: Differentiate between submitted and issued descriptors

The pl330 dmaengine driver currently does not differentiate between submitted
and issued descriptors. It won't start transferring a newly submitted
descriptor until issue_pending() is called, but only if it is idle. If it is
active and a new descriptor is submitted before it goes idle it will happily
start the newly submitted descriptor once all earlier submitted descriptors have
been completed. This is not a 100% correct with regards to the dmaengine
interface semantics. A descriptor is not supposed to be started until the next
issue_pending() call after the descriptor has been submitted. This patch adds a
second per channel list that keeps track of the submitted descriptors. Once
issue_pending() is called the submitted descriptors are moved to the working
list and only descriptors on the working list are started.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: pl330: Set residue_granularity

The pl330 driver currently does not support residue reporting, so set the
residue granularity to DMA_RESIDUE_GRANULARITY_DESCRIPTOR.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Mark Brown <broonie@linaro.org>

dma: pl330: ensure DMA descriptors are zero-initialised

I see the following splat with 3.13-rc1 when attempting to perform DMA:

[ 253.004516] Alignment trap: not handling instruction e1902f9f at [<c0204b40>]
[ 253.004583] Unhandled fault: alignment exception (0x221) at 0xdfdfdfd7
[ 253.004646] Internal error: : 221 [#1] PREEMPT SMP ARM
[ 253.004691] Modules linked in: dmatest(+) [last unloaded: dmatest]
[ 253.004798] CPU: 0 PID: 671 Comm: kthreadd Not tainted 3.13.0-rc1+ #2
[ 253.004864] task: df9b0900 ti: df03e000 task.ti: df03e000
[ 253.004937] PC is at dmaengine_unmap_put+0x14/0x34
[ 253.005010] LR is at pl330_tasklet+0x3c8/0x550
[ 253.005087] pc : [<c0204b44>] lr : [<c0207478>] psr: a00e0193
[ 253.005087] sp : df03fe48 ip : 00000000 fp : df03bf18
[ 253.005178] r10: bf00e108 r9 : 00000001 r8 : 00000000
[ 253.005245] r7 : df837040 r6 : dfb41800 r5 : df837048 r4 : df837000
[ 253.005316] r3 : dfdfdfcf r2 : dfb41f80 r1 : df837048 r0 : dfdfdfd7
[ 253.005384] Flags: NzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel
[ 253.005459] Control: 30c5387d Table: 9fb9ba80 DAC: fffffffd
[ 253.005520] Process kthreadd (pid: 671, stack limit = 0xdf03e248)

This is due to desc->txd.unmap containing garbage (uninitialised memory).

Rather than add another dummy initialisation to _init_desc, instead
ensure that the descriptors are zero-initialised during allocation and
remove the dummy, per-field initialisation.

Cc: Andriy Shevchenko <andriy.shevchenko@intel.com>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

dma: pl330: Alloc dma_parms for the dma device

In order to be able to set a maximum segment size for the device we need to
allocate a dma_parameters struct for the device first.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: prepare for generic 'unmap' data

Add a hook for a common dma unmap implementation to enable removal of
the per driver custom unmap code. (A reworked version of Bartlomiej
Zolnierkiewicz's patches to remove the custom callbacks and the size
increase of dma_async_tx_descriptor for drivers that don't care about
raid).

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
[bzolnier: prepare pl330 driver for adding missing unmap while at it]
Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>

dma: pl330: silence a compile warning

On 64 bit systems GCC warns that:

drivers/dma/pl330.c: In function ‘pl330_filter’:
drivers/dma/pl330.c:2317:21: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]

It's harmless and I have casted it away.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: pl330: off by one in pl330_probe()

There are only AMBA_NR_IRQS (2) elements in adev->irq[]. This code
maybe works if the there is a zero directly after the array.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

DMA-API: dma: pl330: add dma_set_mask_and_coherent() call

The DMA API requires drivers to call the appropriate dma_set_mask()
functions before doing any DMA mapping. Add this required call to
the AMBA PL330 driver.

Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

dma: pl330: Remove unnecessary amba_set_drvdata()

Driver core clears the driver data to NULL after device_release
or on probe failure, so just remove it from here.

Driver core change:
"device-core: Ensure drvdata = NULL when no driver is bound"
(sha1: 0998d0631001288a5974afc0b2a5f568bcdecb4d)

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: pl330: Support per channel irq allocation

Some pl330 have per channel irq and it is necessary
to allocate all of them. Loop over irq assigned for this
device to support these pl330 IPs.

For example this IP is available on Xilinx Zynq platform.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Tested-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: pl330: Simplify irq allocation

Use devm_request_irq function.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: use dma_set_max_seg_size to set the sg limit

Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: pl330: Fix handling of TERMINATE_ALL while processing completed descriptors

The pl330 DMA driver is broken in regard to handling a terminate all request
while it is processing the list of completed descriptors. This is most visible
when calling dmaengine_terminate_all() from within the descriptors callback for
cyclic transfers. In this case the TERMINATE_ALL transfer will clear the
work_list and stop the transfer. But after all callbacks for all completed
descriptors have been handled the descriptors will be re-enqueued into the (now
empty) work_list. So the next time dma_async_issue_pending() is called for the
channel these descriptors will be transferred again which will cause data
corruption. Similar issues can occur if dmaengine_terminate_all() is not called
from within the descriptor callback but runs on a different CPU at the same time
as the completed descriptor list is processed.

This patch introduces a new per channel list which will hold the completed
descriptors. While processing the list the channel's lock will be held to avoid
racing against dmaengine_terminate_all(). The lock will be released when calling
the descriptors callback though. Since the list of completed descriptors might
be modified (e.g. by calling dmaengine_terminate_all() from the callback) we can
not use the normal list iterator macros. Instead we'll need to check for each
loop iteration again if there are still items in the list. The drivers
TERMINATE_ALL implementation is updated to move descriptors from both the
work_list as well the new completed_list back to the descriptor pool. This makes
sure that none of the descripts finds its way back into the work list and also
that we do not call any futher complete callbacks after
dmaengine_terminate_all() has been called.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: pl330: split off common code to give back descriptors

This patch adds __pl330_giveback_descs which give back descriptors when fails
allocating descriptors. It requires to eliminate duplication for
pl330_prep_dma_sg which will be added later.

Signed-off-by: Chanho Park <chanho61.park@samsung.com>
Acked-by : Jassi Brar <jassisinghbrar@gmail.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: use dev_get_platdata()

Use the wrapper function for retrieving the platform data instead of
accessing dev->platform_data directly.

Signed-off-by: Jingoo Han <jg1.han@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: pl330: Fix cyclic transfers

Allocate a descriptor for each period of a cyclic transfer, not just the first.
Also since the callback needs to be called for each finished period make sure to
initialize the callback and callback_param fields of each descriptor in a cyclic
transfer.

Cc: stable@vger.kernel.org
Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: pl330: Implement device_slave_caps

Implement the device_slave_caps() callback for the pl330 driver. This allows
dmaengine users like the generic ALSA dmaengine PCM driver to query the
capabilities of the driver. The PL330 supports all buswidths and both
mem-to-dev as well as dev-to-mem transfers. In theory there is no limit on the
number of segments that can be transferred (in practice you'll run out of memory
eventually) and the number of bytes per segment is limited by the size of the
PL330 program buffer. Due to the nature of the PL330 the maximum number of bytes
per segment depends on the burstsize, the driver sets it to the value for a
1-byte burstsize, since it is the smallest.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: pl330: use dma_addr_t for describing bus addresses

The microcode bus address (pl330_dmac.mcode_bus) is currently a u32,
which fails to compile when building on a system with 64-bit bus
addresses.

This patch uses dma_addr_t to represent the address instead.

Cc: Jassi Brar <jaswinder.singh@linaro.org>
Cc: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Acked-by: Jassi Brar <jaswinder.singh@linaro.org>
Acked-by: Grant Likely <grant.likely@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dma: pl330: rip out broken, redundant ID probing

The PL330 driver probes the peripheral and primecell IDs of the device to
make sure that it is indeed an AMBA PL330. However, it does this by
making byte accesses to a device mapping of the word-aligned ID
registers, which is either UNPREDICTABLE or generates an alignment fault
(depending on the presence of the virtualisation extensions).

Rather than fix this code, we can actually rip most of it out and let
the AMBA bus driver correctly do the probing for us.

Cc: Jassi Brar <jaswinder.singh@linaro.org>
Cc: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Acked-by: Jassi Brar <jaswinder.singh@linaro.org>
Acked-by: Grant Likely <grant.likely@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

drivers/dma/pl330.c: fix locking in pl330_free_chan_resources()

tasklet_kill() may sleep so call it before taking pch->lock.

Fixes following lockup:

BUG: scheduling while atomic: cat/2383/0x00000002
Modules linked in:
unwind_backtrace+0x0/0xfc
__schedule_bug+0x4c/0x58
__schedule+0x690/0x6e0
sys_sched_yield+0x70/0x78
tasklet_kill+0x34/0x8c
pl330_free_chan_resources+0x24/0x88
dma_chan_put+0x4c/0x50
[...]
BUG: spinlock lockup suspected on CPU#0, swapper/0/0
lock: 0xe52aa04c, .magic: dead4ead, .owner: cat/2383, .owner_cpu: 1
unwind_backtrace+0x0/0xfc
do_raw_spin_lock+0x194/0x204
_raw_spin_lock_irqsave+0x20/0x28
pl330_tasklet+0x2c/0x5a8
tasklet_action+0xfc/0x114
__do_softirq+0xe4/0x19c
irq_exit+0x98/0x9c
handle_IPI+0x124/0x16c
gic_handle_irq+0x64/0x68
__irq_svc+0x40/0x70
cpuidle_wrap_enter+0x4c/0xa0
cpuidle_enter_state+0x18/0x68
cpuidle_idle_call+0xac/0xe0
cpu_idle+0xac/0xf0

Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Cc: Vinod Koul <vinod.koul@linux.intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

dma: pl330: Convert to devm_ioremap_resource()

Use the newly introduced devm_ioremap_resource() instead of
devm_request_and_ioremap() which provides more consistent error handling.

Signed-off-by: Sachin Kamat <sachin.kamat@linaro.org>
Reviewed-by: Thierry Reding <thierry.reding@avionic-design.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

DMA: PL330: allow submitting 2 requests at a time

Fix the logic to allow mc programming of second transfer after first
has been done, by removing immediate return upon success and iterating
until we detect QFull or DMAC dying.

Reported-by: Alvaro Moran <dirac3000@gmail.com>
Tested-by: Alvaro Moran <dirac3000@gmail.com>
Signed-off-by: Jassi Brar <jaswinder.singh@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

DMA: PL330: Add check if device tree compatible

This patch register the dma controller with generic dma helpers only
in DT case. This also adds some extra error handling in the driver.

Signed-off-by: Padmavathi Venna <padma.v@samsung.com>
Reported-by: Sachin Kamat <sachin.kamat@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

DMA: PL330: Register the DMA controller with the generic DMA helpers

This patch registers the pl330 dma controller driver with the generic
device tree dma helper functions.

Signed-off-by: Padmavathi Venna <padma.v@samsung.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

DMA: PL330: Add xlate function

Add xlate to translate the device-tree binding information into
the appropriate format. The filter function requires the dma
controller device and dma channel number as filter_params.

Signed-off-by: Padmavathi Venna <padma.v@samsung.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

DMA: PL330: Add new pl330 filter for DT case.

This patch adds a new pl330_dt_filter for DT case to filter the
required channel based on the new filter params and modifies the
old filter only for non-DT case as suggested by Arnd Bergmann.

Signed-off-by: Padmavathi Venna <padma.v@samsung.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

DMA: PL330: Use devm_* functions

devm_* functions are device managed and make the code and error
handling a bit simpler.

Cc: Jassi Brar <jassisinghbrar@gmail.com>
Signed-off-by: Sachin Kamat <sachin.kamat@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

Drivers: dma: remove __dev* attributes.

CONFIG_HOTPLUG is going away as an option. As a result, the __dev*
markings need to be removed.

This change removes the use of __devinit, __devexit_p, __devinitconst,
and __devexit from these drivers.

Based on patches originally written by Bill Pemberton, but redone by me
in order to handle some of the coding style issues better, by hand.

Cc: Bill Pemberton <wfp5p@virginia.edu>
Cc: Viresh Kumar <viresh.linux@gmail.com>
Cc: Dan Williams <djbw@fb.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Barry Song <baohua.song@csr.com>
Cc: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Cc: Alexander Duyck <alexander.h.duyck@intel.com>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Linus Walleij <linus.walleij@linaro.org>
Cc: Jassi Brar <jassisinghbrar@gmail.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Bill Pemberton <wfp5p@virginia.edu>
Cc: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

dma: remove use of __devinit

CONFIG_HOTPLUG is going away as an option so __devinit is no longer
needed.

Signed-off-by: Bill Pemberton <wfp5p@virginia.edu>
Cc: Li Yang <leoli@freescale.com>
Cc: Zhang Wei <zw@zh-kernel.org>
Cc: Barry Song <baohua.song@csr.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

dmaengine: Pass flags via device_prep_dma_cyclic() callback

Change the parameter list of device_prep_dma_cyclic() so the DMA drivers
can receive the flags coming from clients.
This feature can be used during audio operation to disable all audio
related interrupts when the DMA_PREP_INTERRUPT is cleared from the flags.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: Nicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: Shawn Guo <shawn.guo@linaro.org>
Acked-by: Vinod Koul <vinod.koul@linux.intel.com>
Signed-off-by: Mark Brown <broonie@opensource.wolfsonmicro.com>

DMA: PL330: Check the pointer returned by kzalloc

kzalloc could return NULL. Hence add a check to avoid
NULL pointer dereference.

Signed-off-by: Sachin Kamat <sachin.kamat@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Cc: Stable <stable@vger.kernel.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

DMA: PL330: Fix potential NULL pointer dereference in pl330_submit_req()

'r->cfg' is being checked for NULL. However, it is dereferenced
in the previous statements. Thus moving those statements within
the check.

Signed-off-by: Sachin Kamat <sachin.kamat@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Cc: Stable <stable@vger.kernel.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

DMA: PL330: return ENOMEM instead of 0 from pl330_alloc_chan_resources

Since 0 is not considered as error at dmaengine level, return ENOMEM
from pl330_alloc_chan_resources in case of failure.

Signed-off-by: Inderpal Singh <inderpal.singh@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Acked-by: Kukjin Kim <kgene.kim@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

DMA: PL330: Remove redundant runtime_suspend/resume functions

The driver's runtime_suspend/resume functions just disable/enable
the clock which is already being managed at AMBA bus level
runtime_suspend/resume functions.

Hence, remove the driver's runtime_suspend/resume functions.

Signed-off-by: Inderpal Singh <inderpal.singh@linaro.org>
Tested-by: Chander Kashyap <chander.kashyap@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

DMA: PL330: Remove controller clock enable/disable

The controller clock is being enabled/disabled in AMBA bus
infrastructre in probe/remove functions. Hence, its not required
at driver level probe/remove.

Signed-off-by: Inderpal Singh <inderpal.singh@linaro.org>
Tested-by: Chander Kashyap <chander.kashyap@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

DMA: PL330: Set the capability of pdm0 and pdm1 as DMA_PRIVATE

DMA clients pdma0 and pdma1 are internal to the SoC and are used only
by dedicated peripherals. Since they cannot be used for generic
purpose, their capability should be set as DMA_PRIVATE.

CC: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Tushar Behera <tushar.behera@linaro.org>
Acked-by: Kukjin Kim <kgene.kim@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dma: fix comments

Correct spelling typo in drivers/dma.

Signed-off-by: Masanari Iida <standby24x7@gmail.com>
Signed-off-by: Jiri Kosina <jkosina@suse.cz>

DMA: PL330: Fix racy mutex unlock

pl330_update() stores a pointer to the thrd->req that finished, which
contains a pointer to the corresponding pl330_req. This is done with
the pl330_lock held. Then, it iterates through the req_done list,
calling the callback for each of the requests that are done. The
problem is that the driver releases the lock before calling the
callback for each of the callbacks. pl330_submit_req() running in
another processor can then acquire the lock and insert another request
in one of the thrd->req that hasn't been processed yet, replacing the
pointer to pl330_req there. When the callback returns in
pl330_update() and the next rqdone is popped from the list, it
dereferences the pl330_req pointer to the just scheduled pl330_req,
instead of the one that has finished, calling pl330 with the wrong r.

This patch fixes this by storing the pointer to pl330_req directly in
the list.

Signed-off-by: Javi Merino <javi.merino@arm.com>
Cc: Jassi Brar <jaswinder.singh@linaro.org>
Acked-by: Jassi Brar <jaswinder.singh@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

DMA: PL330: Add missing static storage class specifier

Fixes the following sparse warning:
drivers/dma/pl330.c:2542:5: warning: symbol 'add_desc' was not declared. Should it be static?

Signed-off-by: Sachin Kamat <sachin.kamat@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: pl330: dont complete descriptor for cyclic dma

Commit eab215855803 ("dmaengine: pl330: dont complete descriptor for
cyclic dma") wrongly completes descriptor for cyclic dma, hence following
BUG_ON is still hit with cyclic DMA operations.

kernel BUG at drivers/dma/dmaengine.h:53!

Signed-off-by: Tushar Behera <tushar.behera@linaro.org>
Acked-by: Jassi Brar <jaswinder.singh@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>
Cc: stable <stable@vger.kernel.org>

DMA: PL330: Remove duplicate header file inclusion

Removes <linux/interrupt.h> file which was included twice.

Signed-off-by: Sachin Kamat <sachin.kamat@linaro.org>
Acked-by: Jassi Brar <jaswinder.singh@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: pl330: dont complete descriptor for cyclic dma

the cookie updates completed the cyclic dma descriptor wrongly. This caused the
BUG_ON to be hit as submit is called for completed descriptor

Fix this by not marking the cyclic descriptor as complete

Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>
Tested-by: Thomas Abraham <thomas.abraham@linaro.org>

dma: pl330: fix a couple of compilation warnings

Move a couple of tests and do a minor refactor to avoid:

drivers/dma/pl330.c: In function 'pl330_probe':
drivers/dma/pl330.c:2929:215: warning: comparison of distinct pointer types lacks a cast [enabled by default]
drivers/dma/pl330.c: In function 'pl330_tasklet':
drivers/dma/pl330.c:2250:8: warning: 'pch' may be used uninitialized in this function [-Wuninitialized]
drivers/dma/pl330.c:2228:25: note: 'pch' was declared here
drivers/dma/pl330.c:2277:130: warning: 'pch' may be used uninitialized in this function [-Wuninitialized]
drivers/dma/pl330.c:2260:25: note: 'pch' was declared here

Signed-off-by: Olof Johansson <olof@lixom.net>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: add context parameter to prep_slave_sg and prep_dma_cyclic

Add context parameter to device_prep_slave_sg() and device_prep_dma_cyclic()
interfaces to allow passing client/target specific information associated
with the data transfer.
Modify all affected DMA engine drivers.

Signed-off-by: Alexandre Bounine <alexandre.bounine@idt.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Felipe Balbi <balbi@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

ARM: 7362/1: AMBA: Add module_amba_driver() helper macro for amba_driver

For simple modules that contain a single amba_driver without any
additional setup code then ends up being a block of duplicated
boilerplate. This patch adds a new macro, module_amba_driver(),
which replaces the module_init()/module_exit() registrations with
template functions.

Signed-off-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

dmaengine: pl330: fix the pl330 build after cookie cleanup

drivers/dma/pl330.c: In function 'pl330_control':
drivers/dma/pl330.c:342: error: 'struct dma_pl330_chan' has no member named
'completed'

Reported by: Jassi Brar <jassisinghbrar@gmail.com>
Suggested by: Jassi Brar <jassisinghbrar@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: consolidate initialization of cookies

Provide a common function to initialize a channels cookie values.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tested-by: Linus Walleij <linus.walleij@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: consolidate tx_status functions

Now that we have the completed cookie in the dma_chan structure, we
can consolidate the tx_status functions by providing a function to set
the txstate structure and returning the DMA status. We also provide
a separate helper to set the residue for cookies which are still in
progress.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tested-by: Linus Walleij <linus.walleij@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: provide a common function for completing a dma descriptor

Provide a common function to do the cookie mechanics for completing
a DMA descriptor.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tested-by: Linus Walleij <linus.walleij@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: consolidate assignment of DMA cookies

Everyone deals with assigning DMA cookies in the same way (it's part of
the API so they should be), so lets consolidate the common code into a
helper function to avoid this duplication.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tested-by: Linus Walleij <linus.walleij@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: add private header file

Add a local private header file to contain definitions and declarations
which should only be used by DMA engine drivers.

We also fix linux/dmaengine.h to use LINUX_DMAENGINE_H to guard against
multiple inclusion.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tested-by: Linus Walleij <linus.walleij@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: move last completed cookie into generic dma_chan structure

Every DMA engine implementation declares a last completed dma cookie
in their private dma channel structures. This is pointless, and
forces driver specific code. Move this out into the common dma_chan
structure.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tested-by: Linus Walleij <linus.walleij@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

DMA: PL330: Support MEMTOMEM transmit w/o RMB, WMB

The DMAC PL330 r1p0 version fixed the lockup error being on r0p0.
This patch supports the DMA transmission without memory barrier
operation when the revision of DMAC PL330 is the next of r0p0.

Cc: Jassi Brar <jassisinghbrar@gmail.com>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Boojin Kim <boojin.kim@samsung.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

DMA: PL330: Removes useless function

Cc: Jassi Brar <jassisinghbrar@gmail.com>
Cc: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Boojin Kim <boojin.kim@samsung.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

DMA: PL330: Merge PL330 driver into drivers/dma/

Currently there were two part of DMAC PL330 driver for
support old styled s3c-pl330 which has been merged into
drivers/dma/pl330.c driver. Actually, there is no reason
to separate them now.

Basically this patch merges arch/arm/common/pl330.c into
drivers/dma/pl330.c driver and removes useless exported
symbol, externed function and so on.

The newer pl330 driver tested on SMDKV310 and SMDK4212 boards

Cc: Jassi Brar <jassisinghbrar@gmail.com>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Boojin Kim <boojin.kim@samsung.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

drivers/dma/pl330.c: add missing iounmap

Add missing iounmap in error handling code, in a case where the function
already preforms iounmap on some other execution path.

This patch additionally adds calls to clk_disable and clk_put.

A simplified version of the semantic match that finds this problem is as
follows: (http://coccinelle.lip6.fr/)

// <smpl>
@@
expression e;
statement S,S1;
int ret;
@@
e = \(ioremap\|ioremap_nocache\)(...)
... when != iounmap(e)
if (<+...e...+>) S
... when any
when != iounmap(e)
*if (...)
{ ... when != iounmap(e)
return ...; }
... when any
iounmap(e);
// </smpl>

Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

DMA: PL330: Add device tree support

For PL330 dma controllers instantiated from device tree, the channel
lookup is based on phandle of the dma controller and dma request id
specified by the client node. During probe, the private data of each
channel of the controller is set to point to the device node of the
dma controller. The 'chan_id' of the each channel is used as the
dma request id.

Client driver requesting dma channels specify the phandle of the
dma controller and the request id. The pl330 filter function
converts the phandle to the device node pointer and matches that
with channel's private data. If a match is found, the request id
from the client node and the 'chan_id' of the channel is matched.
A channel is found if both the values match.

Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Acked-by: Boojin Kim <boojin.kim@samsung.com>
Signed-off-by: Thomas Abraham <thomas.abraham@linaro.org>
Reviewed-by: Rob Herring <rob.herring@calxeda.com>
Acked-by: Grant Likely <grant.likely@secretlab.ca>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>

DMA: PL330: Infer transfer direction from transfer request instead of platform data

The transfer direction for a channel can be inferred from the transfer
request and the need for specifying transfer direction in platfrom data
can be eliminated. So the structure definition 'struct dma_pl330_peri'
is no longer required.

The channel's private data is set to point to a channel id specified in
the platform data (instead of an instance of type 'struct dma_pl330_peri').
The filter function is correspondingly modified to match the channel id.

With the 'struct dma_pl330_peri' removed from platform data, the dma
controller transfer capabilities cannot be inferred any more. Hence,
the dma controller capabilities is specified using platform data.

Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Acked-by: Boojin Kim <boojin.kim@samsung.com>
Signed-off-by: Thomas Abraham <thomas.abraham@linaro.org>
Acked-by: Grant Likely <grant.likely@secretlab.ca>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>

DMA: PL330: move filter function into driver

The dma channel selection filter function is moved from plat-samsung
into the pl330 driver. In additon to that, a check is added in the
filter function to ensure that the channel on which the filter has
been invoked is pl330 channel instance (and avoid any incorrect
access of chan->private in a system with multiple types of DMA
drivers).

Suggested-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Thomas Abraham <thomas.abraham@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Acked-by: Grant Likely <grant.likely@secretlab.ca>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>

DMA: PL330: Remove pm_runtime_xxx calls from pl330 probe/remove

amba_probe() now calls pm_runtime_get_noresume() and pm_runtime_enable()
for the devices before the device probe is called. Hence we don't need
to call pm_runtime_get_xxx and pm_runtime_enable() in device probe again.
In the same way, since amba_remove() calls the respective pm_runtime
functions, those functions need not be called from device remove.

This patch fixes following run time error with pl330 driver.

dma-pl330 dma-pl330.0: Unbalanced pm_runtime_enable!
dma-pl330 dma-pl330.0: failed to get runtime pm

Signed-off-by: Giridhar Maruthy <giridhar.maruthy@linaro.org>
Signed-off-by: Tushar Behera <tushar.behera@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: pl330: Enable module alias autogeneration for AMBA drivers

Signed-off-by: Dave Martin <dave.martin@linaro.org>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Acked-by: Vinod Koul <vinod.koul@intel.com>

DMA: PL330: Fix build warning

This patch adds to fix the build warning as following.

drivers/dma/pl330.c: In function 'pl330_probe':
drivers/dma/pl330.c:859: warning: comparison of distinct pointer types lacks a cast

Signed-off-by: Boojin Kim <boojin.kim@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: move drivers to dma_transfer_direction

fixup usage of dma direction by introducing dma_transfer_direction,
this patch moves dma/drivers/* to use new enum

Cc: Jassi Brar <jaswinder.singh@linaro.org>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Viresh Kumar <viresh.kumar@st.com>
Cc: Linus Walleij <linus.walleij@linaro.org>
Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Cc: Mika Westerberg <mika.westerberg@iki.fi>
Cc: H Hartley Sweeten <hartleys@visionengravers.com>
Cc: Li Yang <leoli@freescale.com>
Cc: Zhang Wei <zw@zh-kernel.org>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Cc: Shawn Guo <shawn.guo@freescale.com>
Cc: Yong Wang <yong.y.wang@intel.com>
Cc: Tomoya MORINAGA <tomoya-linux@dsn.lapis-semi.com>
Cc: Boojin Kim <boojin.kim@samsung.com>
Cc: Barry Song <Baohua.Song@csr.com>
Acked-by: Mika Westerberg <mika.westerberg@iki.fi>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Viresh Kumar <viresh.kumar@st.com>
Acked-by: Nicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: Vinod Koul <vinod.koul@linux.intel.com>

dmaengine: delete redundant chan_id and chancnt initialization in dma drivers

dma_async_device_register will re-init chan_id and chancnt,
so whatever chan_id and chancnt are set in drivers, they will
be re-written by dma_async_device_register.

Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Cc: Viresh Kumar <viresh.kumar@st.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Piotr Ziecik <kosmo@semihalf.com>
Cc: Yong Wang <yong.y.wang@intel.com>
Cc: Jaswinder Singh <jassi.brar@samsung.com>
Cc: Pelagicore AB <info@pelagicore.com>
Signed-off-by: Barry Song <Baohua.Song@csr.com>
Acked-by: Viresh Kumar <viresh.kumar@st.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

DMA: PL330: Add DMA_CYCLIC capability

This patch adds DMA_CYCLIC capability that is used for audio driver.
DMA driver activated with it reuses the dma requests that were submitted
through tx_submit().

Signed-off-by: Boojin Kim <boojin.kim@samsung.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

DMA: PL330: Remove the start operation for handling DMA_TERMINATE_ALL command

Original code carries out the start operation after flush operation.
But start operation is not required for DMA_TERMINATE_ALL command.
So, this patch removes the unnecessary start operation and only carries out
the flush operation for handling DMA_TERMINATE_ALL command.

Signed-off-by: Boojin Kim <boojin.kim@samsung.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>
[Fixed typos in changelog]
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

DMA: PL330: Support DMA_SLAVE_CONFIG command

Signed-off-by: Boojin Kim <boojin.kim@samsung.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

DMA: PL330: Update PL330 DMA API driver

This patch updates following 3 items.
1. Removes unneccessary code.
2. Add AMBA, PL330 configuration
3. Change the meaning of 'peri_id' variable
from PL330 event number to specific dma id by user.

Signed-off-by: Boojin Kim <boojin.kim@samsung.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

DMA: PL330: Add support runtime PM for PL330 DMAC

Signed-off-by: Boojin Kim <boojin.kim@samsung.com>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Vinod Koul <vinod.koul@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Kukjin Kim <kgene.kim@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

dmaengine: pl330: make platform data optional

The pl330 needs platform data for describing peripheral connections, but
some platforms may only support memory to memory dma channels. In this
case, we can probe for how many channels there are and don't need the
platform data.

As memcpy requests don't need channel private data to hold peripheral
info, allow private data to be NULL in this case.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Cc: Jassi Brar <jassisinghbrar@gmail.com>
Cc: Vinod Koul <vkoul@infradead.org>
Cc: Dan Williams <dan.j.williams@intel.com>
Acked-by: Jassi Brar <jassisinghbrar@gmail.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

ARM: amba: make probe() functions take const id tables

Make Primecell driver probe functions take a const pointer to their
ID tables. Drivers should never modify their ID tables in their
probe handler.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

DMA: PL330: Add dma api driver

Add DMA Engine API driver for the PL330 DMAC.
This driver is supposed to be reusable by various
platforms that have one or more PL330 DMACs.
Atm, DMA_SLAVE and DMA_MEMCPY capabilities have been
implemented.

Signed-off-by: Jassi Brar <jassi.brar@samsung.com>
Reviewed-by: Linus Walleij <linus.walleij@stericsson.com>
[dan.j.williams@intel.com: missing slab.h and ->device_control() fixups]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>