dmaengine: ti: k3-udma: Report short packet errors

Propagate the TR response status to the device using BCDMA
split-channels. For example CSI-RX driver should be able to check if a
frame was not transferred completely (short packet) and needs to be
discarded.

Fixes: 25dcb5dd7b7c ("dmaengine: ti: New driver for K3 UDMA")
Signed-off-by: Jai Luthra <j-luthra@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20240103-tr_resp_err-v1-1-2fdf6d48ab92@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Add PSIL threads for AM62P and J722S

Add PSIL thread information and enable UDMA support for AM62P
and J722S SoC. J722S SoC family is a superset of AM62P, thus
common PSIL thread ID map is reused for both devices.

For those interested, more details about the SoC can be found
in the Technical Reference Manual here:
AM62P - https://www.ti.com/lit/pdf/spruj83
J722S - https://www.ti.com/lit/zip/sprujb3

Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
Signed-off-by: Bryan Brattlof <bb@ti.com>
Signed-off-by: Vaishnav Achath <vaishnav.a@ti.com>
Reviewed-by: Jai Luthra <j-luthra@ti.com>
Link: https://lore.kernel.org/r/20231213081318.26203-1-vaishnav.a@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: Explicitly include correct DT includes

The DT of_device.h and of_platform.h date back to the separate
of_platform_bus_type before it as merged into the regular platform bus.
As part of that merge prepping Arm DT support 13 years ago, they
"temporarily" include each other. They also include platform_device.h
and of.h. As a result, there's a pretty much random mix of those include
files used throughout the tree. In order to detangle these headers and
replace the implicit includes with struct declarations, users need to
explicitly include the correct includes.

Signed-off-by: Rob Herring <robh@kernel.org>
Link: https://lore.kernel.org/r/20230718143138.1066177-1-robh@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Add support for J721S2 CSI BCDMA instance

J721S2 has dedicated BCDMA instance for Camera Serial Interface RX
and TX. The BCDMA instance supports RX and TX channels but block copy
channels are not present, add support for the same.

Signed-off-by: Vaishnav Achath <vaishnav.a@ti.com>
Link: https://lore.kernel.org/r/20230505143929.28131-3-vaishnav.a@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: annotate pm function with __maybe_unused

We get a warning when PM is not set:

../drivers/dma/ti/k3-udma.c:5552:12: warning: 'udma_pm_resume' defined but not used [-Wunused-function]
5552 | static int udma_pm_resume(struct device *dev)
| ^~~~~~~~~~~~~~
../drivers/dma/ti/k3-udma.c:5530:12: warning: 'udma_pm_suspend' defined but not used [-Wunused-function]
5530 | static int udma_pm_suspend(struct device *dev)
| ^~~~~~~~~~~~~~~

Fix this by annotating pm function with __maybe_unused

Fixes: fbe05149e40b ("dmaengine: ti: k3-udma: Add system suspend/resume support")
Reported-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Tested-by: Randy Dunlap <rdunlap@infradead.org> # build-tested
Link: https://lore.kernel.org/r/20230516174311.117264-1-vkoul@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-psil: Add PSI-L thread support for J784s4

Add psil thread IDs for J784s4 and include J784s4 in the set of
"k3_soc_devices" in k3-psil.c.

Signed-off-by: Apurva Nandan <a-nandan@ti.com>
Signed-off-by: Hari Nagalla <hnagalla@ti.com>
Signed-off-by: Siddharth Vadapalli <s-vadapalli@ti.com>
Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
[vaishnav.a@ti.com: add MCSPI-TX and 3rd CSI2RX instance entries]
Signed-off-by: Vaishnav Achath <vaishnav.a@ti.com>
[j-choudhary@ti.com: add sa2ul entries, minor cleanups]
Signed-off-by: Jayesh Choudhary <j-choudhary@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20230308201513.116638-1-j-choudhary@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Workaround errata i2234

Per [1], UDMA TR15 transactions may hang if ICNT0 is less than 64B
Work around is to set EOL flag is to 1 for ICNT0.

Since, there is no performance penalty / side effects of setting EOL
flag event ICNTO > 64B, just set the flag for all UDMAP TR15
descriptors.

[1] https://www.ti.com/lit/er/sprz455a/sprz455a.pdf
Errata doc for J721E DRA829/TDA4VM Processors Silicon Revision 1.1/1.0
(Rev. A)

Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
[j-choudhary@ti.com: minor cleanups]
Signed-off-by: Jayesh Choudhary <j-choudhary@ti.com>
Link: https://lore.kernel.org/r/20230323120107.27638-1-j-choudhary@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Add system suspend/resume support

The K3 platforms configure the DMA resources with the
help of the TI's System Firmware's Device Manager(DM)
over TISCI. The group of DMA related Resource Manager[1]
TISCI messages includes: INTA, RINGACC, UDMAP, and PSI-L.
This configuration however, does not persist in the DM
after leaving from Suspend-to-RAM state. We have to restore
the DMA channel configuration over TISCI for all configured
channels when returning from suspend.

The TISCI resource management calls for each DMA type (UDMA,
PKTDMA, BCDMA) happen in device_free_chan_resources() and
device_alloc_chan_resources(). In pm_suspend() we store
the current udma_chan_config for channels that still have
attached clients and call device_free_chan_resources().
In pm_resume() restore the udma_channel_config from backup
and call device_alloc_chan_resources() for those channels.

Drivers like CPSW that use k3-udma-glue already do their own
DMA resource management so use the late system suspend/resume hooks.

[1] https://software-dl.ti.com/tisci/esd/latest/2_tisci_msgs/index.html#resource-management-rm

Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
[g-vlaev@ti.com: Add patch description and config backup]
[g-vlaev@ti.com: Supend only channels with clients]
Signed-off-by: Georgi Vlaev <g-vlaev@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20230329155349.2566010-1-vigneshr@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: remove non-fatal probe deferral log

Drop the non-fatal probe deferral log for getting MSI domain.
This makes the kernel log clean and we do not get recurring logs
stating: "Failed to get MSI domain".

Signed-off-by: Jayesh Choudhary <j-choudhary@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20230117051855.29644-1-j-choudhary@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Add support for BCDMA CSI RX

BCDMA CSI RX present on AM62Ax SoC is a dedicated DMA for servicing
Camera Serial Interface (CSI) IP. Add support for the same.

Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20221213164304.1126945-6-vigneshr@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Add support for DMAs on AM62A SoC

AM62A SoC has a BCDMA and PKTDMA as systems DMAs for service various
peripherals similar to AM64 SoC. Add support for the same.

Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20221213164304.1126945-5-vigneshr@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Fix BCDMA for case w/o BCHAN

Reusing loop iterator fails if BCHAN is not present as iterator is
uninitialized

Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20221213164304.1126945-3-vigneshr@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Do conditional decrement of UDMA_CHAN_RT_PEER_BCNT_REG

PSIL_EP_NATIVE endpoints may not have PEER registers for BCNT and thus
udma_decrement_byte_counters() should not try to decrement these counters.
This fixes the issue of crypto IPERF testing where the client side (EVM)
hangs without transfer of packets to the server side, seen since this
function was added.

Fixes: 7c94dcfa8fcf ("dmaengine: ti: k3-udma: Reset UDMA_CHAN_RT byte counters to prevent overflow")
Signed-off-by: Jayesh Choudhary <j-choudhary@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20221128085005.489964-1-j-choudhary@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: convert k3-udma to module

Currently k3-udma driver is built as separate platform drivers with a
shared probe and identical code path, just differnet platform data.

To enable to build as module, convert the separate platform driver
into a single module_platform_driver with the data selection done via
compatible string and of_match. The separate of_match tables are also
combined into a single table to avoid the multiple calls to
of_match_node()

Since all modern TI platforms using this are DT enabled, the removal
of separate platform_drivers should have no functional change.

Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Signed-off-by: Kevin Hilman <khilman@baylibre.com>
Link: https://lore.kernel.org/r/20220929234820.940048-3-khilman@baylibre.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Respond TX done if DMA_PREP_INTERRUPT is not requested

If the DMA consumer driver does not expect the callback for TX done, then
we need not perform the channel RT byte counter calculations and estimate
the completion but return complete on first attempt itself.This assumes
that the consumer who did not request DMA_PREP_INTERRUPT has its own
mechanism for understanding TX completion, example: MCSPI EOW interrupt
can be used as TX completion signal for a SPI transaction.

Signed-off-by: Vaishnav Achath <vaishnav.a@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20220914110049.5842-1-vaishnav.a@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Reset UDMA_CHAN_RT byte counters to prevent overflow

UDMA_CHAN_RT_*BCNT_REG stores the real-time channel bytecount statistics.
These registers are 32-bit hardware counters and the driver uses these
counters to monitor the operational progress status for a channel, when
transferring more than 4GB of data it was observed that these counters
overflow and completion calculation of a operation gets affected and the
transfer hangs indefinitely.

This commit adds changes to decrease the byte count for every complete
transaction so that these registers never overflow and the proper byte
count statistics is maintained for ongoing transaction by the RT counters.

Earlier uc->bcnt used to maintain a count of the completed bytes at driver
side, since the RT counters maintain the statistics of current transaction
now, the maintenance of uc->bcnt is not necessary.

Signed-off-by: Vaishnav Achath <vaishnav.a@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20220802054835.19482-1-vaishnav.a@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

lib/bitmap: change type of bitmap_weight to unsigned long

bitmap_weight() doesn't return negative values, so change it's type
to unsigned long. It may help compiler to generate better code and
catch bugs.

Signed-off-by: Yury Norov <yury.norov@gmail.com>

dmaengine: ti: k3-udma: Add AM62x DMSS support

Attribute AM64x soc data to AM62x as well as the DMSS IP is similar b/w
these two SoCs

Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20220219083220.489420-2-vigneshr@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Add SoC dependent data for J721S2 SoC

Add SYSFW defined rchan_oes_offset number for J721S2 SoC in soc data.

Signed-off-by: Aswath Govindraju <a-govindraju@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20211119132315.15901-2-a-govindraju@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

soc: ti: ti_sci_inta_msi: Get rid of ti_sci_inta_msi_get_virq()

Just use the core function msi_get_virq().

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Nishanth Menon <nm@ti.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Vinod Koul <vkoul@kernel.org>
Acked-by: Nishanth Menon <nm@ti.com>
Link: https://lore.kernel.org/r/20211210221815.269468319@linutronix.de

device: Move MSI related data into a struct

The only unconditional part of MSI data in struct device is the irqdomain
pointer. Everything else can be allocated on demand. Create a data
structure and move the irqdomain pointer into it. The other MSI specific
parts are going to be removed from struct device in later steps.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Michael Kelley <mikelley@microsoft.com>
Tested-by: Nishanth Menon <nm@ti.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Link: https://lore.kernel.org/r/20211210221813.617178827@linutronix.de

dmaengine: ti: k3-udma: Fix smatch warnings

Smatch reports below warnings [1] wrt dereferencing rm_res when it can
potentially be ERR_PTR(). This is possible when entire range is
allocated to Linux
Fix this case by making sure, there is no deference of rm_res when its
ERR_PTR().

[1]:
drivers/dma/ti/k3-udma.c:4524 udma_setup_resources() error: 'rm_res' dereferencing possible ERR_PTR()
drivers/dma/ti/k3-udma.c:4537 udma_setup_resources() error: 'rm_res' dereferencing possible ERR_PTR()
drivers/dma/ti/k3-udma.c:4681 bcdma_setup_resources() error: 'rm_res' dereferencing possible ERR_PTR()
drivers/dma/ti/k3-udma.c:4696 bcdma_setup_resources() error: 'rm_res' dereferencing possible ERR_PTR()
drivers/dma/ti/k3-udma.c:4711 bcdma_setup_resources() error: 'rm_res' dereferencing possible ERR_PTR()
drivers/dma/ti/k3-udma.c:4848 pktdma_setup_resources() error: 'rm_res' dereferencing possible ERR_PTR()
drivers/dma/ti/k3-udma.c:4861 pktdma_setup_resources() error: 'rm_res' dereferencing possible ERR_PTR()

Reported-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20211209180957.29036-1-vigneshr@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Set r/tchan or rflow to NULL if request fail

udma_get_*() checks if rchan/tchan/rflow is already allocated by checking
if it has a NON NULL value. For the error cases, rchan/tchan/rflow will
have error value and udma_get_*() considers this as already allocated
(PASS) since the error values are NON NULL. This results in NULL pointer
dereference error while de-referencing rchan/tchan/rflow.

Reset the value of rchan/tchan/rflow to NULL if a channel request fails.

CC: stable@vger.kernel.org
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Link: https://lore.kernel.org/r/20211031032411.27235-3-kishon@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Set bchan to NULL if a channel request fail

bcdma_get_*() checks if bchan is already allocated by checking if it
has a NON NULL value. For the error cases, bchan will have error value
and bcdma_get_*() considers this as already allocated (PASS) since the
error values are NON NULL. This results in NULL pointer dereference
error while de-referencing bchan.

Reset the value of bchan to NULL if a channel request fails.

CC: stable@vger.kernel.org
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Link: https://lore.kernel.org/r/20211031032411.27235-2-kishon@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Fix fall-through warning for Clang

Fix the following fallthrough warning:

drivers/dma/ti/k3-udma.c:4951:2: warning: unannotated fall-through between switch labels [-Wimplicit-fallthrough]

Reported-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>

dmaengine: ti: k3-udma: Do not initialize ret in tisci channel config functions

The ret does not need to be initialized to 0 in the tisci channel config
functions.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20210113114923.9231-4-peter.ujfalusi@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Add support for burst_size configuration for mem2mem

The UDMA and BCDMA can provide higher throughput if the burst_size of the
channel is changed from it's default (which is 64 bytes) for Ultra-high
and high capacity channels.

This performance benefit is even more visible when the buffers are aligned
with the burst_size configuration.

The am654 does not have a way to change the burst size, but it is using
64 bytes burst, so increasing the copy_align from 8 bytes to 64 (and
clients taking that into account) can increase the throughput as well.

Numbers gathered on j721e:
echo 8000000 > /sys/module/dmatest/parameters/test_buf_size
echo 2000 > /sys/module/dmatest/parameters/timeout
echo 50 > /sys/module/dmatest/parameters/iterations
echo 1 > /sys/module/dmatest/parameters/max_channels

Prior this patch: ~1.3 GB/s
After this patch: ~1.8 GB/s
with 1 byte alignment: ~1.7 GB/s

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Tested-by: Kishon Vijay Abraham I <kishon@ti.com>
Link: https://lore.kernel.org/r/20210113114923.9231-3-peter.ujfalusi@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Set rflow count for BCDMA split channels

BCDMA RX channels have one flow per channel, therefore set the rflow_cnt
to rchan_cnt.

Without this patch, request for BCDMA RX channel allocation fails as
rflow_cnt is 0 thus fails to reserve a rflow for the channel.

Fixes: 8844898028d4 ("dmaengine: ti: k3-udma: Add support for BCDMA channel TPL handling")
Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20210112141403.30286-1-vigneshr@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Fix a resource leak in an error handling path

In 'dma_pool_create()', we return -ENOMEM, but don't release the resources
already allocated, as in all the other error handling paths.

Go to 'err_res_free' instead of returning directly.

Fixes: 017794739702 ("dmaengine: ti: k3-udma: Initial support for K3 BCDMA")
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20210124070923.724479-1-christophe.jaillet@wanadoo.fr
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Fix pktdma rchan TPL level setup

Instead of initializing the rchan_tpl the initial commit re-initialized
the tchan_tpl.

Fixes: d2abc982333c0 ("dmaengine: ti: k3-udma: Initial support for K3 PKTDMA")
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201216154833.20821-1-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Initial support for K3 PKTDMA

One of the DMAs introduced with AM64 is the Packet DMA (PKTDMA).
It serves similar purpose as K3 UDMAP channels in packet mode, but with
notable differences, like tflow support and channels being allocated to
service specific peripherals.
The rings for the PKTDMA is integrated within the DMA itself instead of
using rings from the general purpose ringacc.

PKTDMA can be used to service PSI-L peripherals, similarly to
K3 UDMA channels.

Most of the driver code can be reused for PKTDMA tchan/rchan support but
new setup and allocation functions are needed to handle the differences
between the DMAs.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-20-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Add support for BCDMA channel TPL handling

Unlike UDMAP the BCDMA defines the channel TPL levels per channel type.
In UDMAP the number of high and ultra-high channels applies to both tchan
and rchan.

BCDMA defines the TPL per channel types: bchan, tchan and rchan can have
different number of high and ultra-high channels.

In order to support BCDMA channel TPL we need to move the tpl information
as per channel type property for the DMAs.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-19-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Initial support for K3 BCDMA

One of the DMAs introduced with AM64 is the Block Copy DMA (BCDMA).
It serves similar purpose as K3 UDMAP channels in TR mode.

The rings for the BCDMA is integrated within the DMA itself instead of
using rings from the general purpose ringacc.

A BCDMA have two different type of channels:
- Block Copy Channels (bchan)
- Split Channels (tchan and rchan)

tchan and rchan can be used to service PSI-L peripherals, similarly to
K3 UDMA channels.

bchan can be only used for block copy operation (TR type15) like the
paired K3 UDMA tchan/rchan configured in block copy mode.
bchans can be also used to service peripherals directly if an external
trigger is selected for the channel.

Most of the driver code can be reused for BCDMA bchan/tchan/rchan support
but new setup and allocation functions are needed to handle the
differences between the DMAs.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-18-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Add support for second resource range from sysfw

Resource allocation via sysfw can use up to two ranges per resource subtype
to support more complex resource assignment, mainly for DMA channels.

Take the second range also into consideration when setting up the maps for
available resources.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-4-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Wait for peer teardown completion if supported

Set the TDTYPE if it is supported on the platform (j721e) which will cause
UDMAP to wait for the remote peer to finish the teardown before returning
the teardown completed message.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Tested-by: Keerthy <j-keerthy@ti.com>
Reviewed-by: Grygorii Strashko <grygorii.strashko@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-3-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Correct normal channel offset when uchan_cnt is not 0

According to different sections of the TRM, the hchan_cnt of CAP3 includes
the number of uchan in UDMA, thus the start offset of the normal channels
are hchan_cnt.

Fixes: daf4ad0499aa4 ("dmaengine: ti: k3-udma: Query throughput level information from hardware")
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-2-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: remove redundant irqsave and irqrestore in hardIRQ

Running in hardIRQ, disabling IRQ is redundant since hardIRQ has disabled
IRQ. This patch removes the irqsave and irqstore to save some instruction
cycles.

Signed-off-by: Barry Song <song.bao.hua@hisilicon.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201027215252.25820-3-song.bao.hua@hisilicon.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: use devm_platform_ioremap_resource_byname

Use the devm_platform_ioremap_resource_byname() helper instead of
calling platform_get_resource_byname() and devm_ioremap_resource()
separately.

Signed-off-by: Zhang Qilong <zhangqilong3@huawei.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200921093701.102208-1-zhangqilong3@huawei.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: k3-udma: convert tasklets to use new tasklet_setup() API

In preparation for unconditionally passing the
struct tasklet_struct pointer to all tasklet
callbacks, switch to using the new tasklet_setup()
and from_tasklet() to pass the tasklet pointer explicitly.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200831103542.305571-36-allen.lkml@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Use soc_device_match() for SoC dependent parameters

Use separate data for SoC dependent parameters. These parameters depends
on the DMA integration (either in HW or in SYSFW), the DMA controller
itself remains compatible with either the am654 or j721e variant.

j7200 have the same DMA as j721e with different number of channels, which
can be queried from HW, but SYSFW defines different rchan_oes_offset
number for j7200 (0x80) compared to j721e (0x400).

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200910124329.21206-1-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Update rchan_oes_offset for am654 SYSFW ABI 3.0

SYSFW ABI 3.0 has changed the rchan_oes_offset value for am654 to support
SR2.

Since the kernel now needs SYSFW API 3.0 to work because the merged irqchip
update, we need to also update the am654 rchan_oes_offset.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200831091019.25273-1-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Remove redundant is_slave_direction() checks

The direction has been already validated in the main callback and there is
no need to check it again in the TR mode handlers for slave_sg and cyclic.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200824120120.9270-1-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Fix the TR initialization for prep_slave_sg

The TR which needs to be initialized for the next sg entry is indexed by
tr_idx and not by the running i counter.

In case any sub element in the SG needs more than one TR, the code would
corrupt an already configured TR.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200824120108.9178-1-peter.ujfalusi@ti.com
Fixes: 6cf668a4ef829 ("dmaengine: ti: k3-udma: Use the TR counter helper for slave_sg and cyclic")
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Query throughput level information from hardware

The CAP3 register contains information about the number of
HCHAN (High Capacity) and UCHAN (Ultra High Capacity) channels in UDMAP.

Based on this information the start indexes of the levels can be calculated
without a need of a table in the match data.

On am654 the CAP3 does not contain information about the number different
channels. Set up the tpl information based on the available documentation.

This change will allow to use the same compatible for different SoCs where
the only difference is the number of channel types.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200717120903.8774-3-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Use defines for capabilities register parsing

Add defines for the CAP register fields to make the code self explaining.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200717120903.8774-2-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Switch to k3_ringacc_request_rings_pair

We only request ring pairs via K3 DMA driver, switch to use the new
k3_ringacc_request_rings_pair() to simplify the code.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
Acked-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>

dmaengine: ti: k3-udma: Use struct_size() in kzalloc()

Make use of the struct_size() helper instead of an open-coded version
in order to avoid any potential type mistakes.

This code was detected with the help of Coccinelle and, audited and
fixed manually.

Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Acked-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Addresses-KSPP-ID: https://github.com/KSPP/linux/issues/83
Link: https://lore.kernel.org/r/20200619224334.GA7857@embeddedor
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Use udma_chan instead of tchan/rchan for IO functions

Move the uc->tchan/rchan checks to the IO wrappers itself instead of
calling the functions with tchan/rchan directly.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: Grygorii Strashko <grygorii.strashko@ti.com>
Link: https://lore.kernel.org/r/20200707102352.28773-6-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Use common defines for TCHANRT/RCHANRT registers

The register offsets and functions are the same among TCHAN and RCHAN.
Use generic, common names for them.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: Grygorii Strashko <grygorii.strashko@ti.com>
Link: https://lore.kernel.org/r/20200707102352.28773-4-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Do not use ring_get_occ in udma_pop_from_ring

The ring_get_occ is redundant as the k3_ringacc_ring_pop() is also
checking the occ of the ring.

With removing the ring_get_occ, the function can be simplified as well.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: Grygorii Strashko <grygorii.strashko@ti.com>
Link: https://lore.kernel.org/r/20200707102352.28773-3-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Remove dma_sync_single calls for descriptors

The descriptors are allocated via wither dma_pool or dma_alloc_coherent.

There is no need for the dma_sync_singel_* calls.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: Grygorii Strashko <grygorii.strashko@ti.com>
Link: https://lore.kernel.org/r/20200707102352.28773-2-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Fix delayed_work usage for tx drain workaround

INIT_DELAYED_WORK_ONSTACK() must be used with on-stack delayed work, which
is not the case here.
Use normal delayed_work for the channels instead.

Fixes: 25dcb5dd7b7c ("dmaengine: ti: New driver for K3 UDMA")
Reported-by: Tomi Valkeinen <tomi.valkeinen@ti.com>
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200618114004.6268-1-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Fix the running channel handling in alloc_chan_resources

In the unlikely case when the channel is running (RT enabled) during
alloc_chan_resources then we should use udma_reset_chan() and not
udma_stop() as the later is trying to initiate a teardown on the channel,
which is not valid at this point.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200527070612.636-3-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Fix cleanup code for alloc_chan_resources

Some of the earlier errors should be sent to the error cleanup path to
make sure that the uchan struct is reset, the dma_pool (if allocated) is
released and memcpy channel pairs are released in a correct way.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200527070612.636-2-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Use correct node to read "ti,udma-atype"

The "ti,udma-atype" property is expected in the UDMA node and not in the
parent navss node.

Fixes: 0ebcf1a274c5 ("dmaengine: ti: k3-udma: Implement support for atype (for virtualization)")
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200527065357.30791-1-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: Replace zero-length array with flexible-array

There is a regular need in the kernel to provide a way to declare having a
dynamically sized set of trailing elements in a structure. Kernel code should
always use “flexible array members”[1] for these cases. The older style of
one-element or zero-length arrays should no longer be used[2].

[1] https://en.wikipedia.org/wiki/Flexible_array_member
[2] https://github.com/KSPP/linux/issues/21

Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>

dmaengine: ti: k3-udma: Use proper return code in alloc_chan_resources

In udma_alloc_chan_resources() if the channel is not willing to stop then
the function should return with error code.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200512134519.5642-1-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Remove udma_chan.in_ring_cnt

The in_ring_cnt is not used for anything, it can be removed.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200512134611.6015-1-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Add missing dma_sync call for rx flush descriptor

The TR mode rx flush descriptor did not had a dma_sync_single_for_device()
call to make sure that the DMA see the correct information.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200512134544.5839-1-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Fix TR mode flags for slave_sg and memcpy

cppi5_tr_csf_set() clears previously set Configuration Specific Flags.
Setting the EOP flag clears the SUPR_EVT flag for the last TR which is not
desirable as we do not want to have events from the TR.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200512134531.5742-1-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Use PTR_ERR_OR_ZERO() to simplify code

Fixes coccicheck warnings:

drivers/dma/ti/k3-udma.c:1294:1-3: WARNING: PTR_ERR_OR_ZERO can be used
drivers/dma/ti/k3-udma.c:1311:1-3: WARNING: PTR_ERR_OR_ZERO can be used
drivers/dma/ti/k3-udma.c:1376:1-3: WARNING: PTR_ERR_OR_ZERO can be used

Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Samuel Zou <zou_wei@huawei.com>
Link: https://lore.kernel.org/r/1588757146-38858-1-git-send-email-zou_wei@huawei.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Disable memcopy via MCU NAVSS on am654

Trace of a test for DMA memcpy domains slipped into the glue layer commit.
The memcpy support should be disabled on the MCU UDMAP.

Fixes: d702419134133 ("dmaengine: ti: k3-udma: Add glue layer for non DMAengine users")
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200327144228.11101-1-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Implement custom dbg_summary_show for debugfs

With the custom dbg_summary_show the driver can show useful information
about the used channels.

dma0 (285c0000.dma-controller): number of channels: 24

dma1 (31150000.dma-controller): number of channels: 84
dma1chan0 | 2b00000.mcasp:tx (MEM_TO_DEV, tchan16 [0x1010 -> 0xc400], PDMA[ ACC32 BURST ], TR mode)
dma1chan1 | 2b00000.mcasp:rx (DEV_TO_MEM, rchan16 [0x4400 -> 0x9010], PDMA[ ACC32 BURST ], TR mode)
dma1chan2 | 2ba0000.mcasp:tx (MEM_TO_DEV, tchan17 [0x1011 -> 0xc507], PDMA[ ACC32 BURST ], TR mode)
dma1chan3 | 2ba0000.mcasp:rx (DEV_TO_MEM, rchan17 [0x4507 -> 0x9011], PDMA[ ACC32 BURST ], TR mode)
dma1chan4 | in-use (MEM_TO_MEM, chan0 pair [0x1000 -> 0x9000], PSI-L Native, TR mode)
dma1chan5 | in-use (MEM_TO_MEM, chan1 pair [0x1001 -> 0x9001], PSI-L Native, TR mode)
dma1chan6 | in-use (MEM_TO_MEM, chan4 pair [0x1004 -> 0x9004], PSI-L Native, TR mode)
dma1chan7 | in-use (MEM_TO_MEM, chan5 pair [0x1005 -> 0x9005], PSI-L Native, TR mode)

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200306142839.17910-3-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Implement support for atype (for virtualization)

The DT for virtualized hosts have dma-cells == 2 where the second parameter
is the ATYPE for the channel.

In case of dma-cells == 1 we can configure the ATYPE as 0 (reset value).

The ATYPE defined for j721e are:
0: pointers are physical addresses (no translation)
1: pointers are intermediate addresses (PVU)
2: pointers are virtual addresses (SMMU)

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200218143126.11361-3-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Fix terminated transfer handling

When we receive back the descriptor of the terminated transfer the cookie
must be marked as completed to make sure that the accounting is correct.

In udma_tx_status() the status should be marked as completed if the channel
is no longer running (it can only happen if the channel is not yet started
for the first time, or after a channel termination).

Fixes: 25dcb5dd7b7ce ("dmaengine: ti: New driver for K3 UDMA")
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200214091441.27535-7-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Use the channel direction in pause/resume functions

It should be possible to pause, resume and check the pause state of a
channel even if we do not have active transfer.

udma_is_chan_paused() can trigger NULL pointer reference in it's current
form when the status is checked while uc->desc is NULL.

Fixes: 25dcb5dd7b7ce ("dmaengine: ti: New driver for K3 UDMA")
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200214091441.27535-6-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Use the TR counter helper for slave_sg and cyclic

Use the generic TR setup function to get the TR counters for both cyclic
and slave_sg transfers.
This way the period_size for cyclic and sg_dma_len() for slave_sg can be
as large as (SZ_64K - 1) * (SZ_64K - 1) and we can handle cases when the
length is >SZ_64K and a prime number.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200214091441.27535-5-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Move the TR counter calculation to helper function

Move the TR counter parameter configuration code out from the prep_memcpy
callback to a helper function to allow a generic re-usable code for other
TR based transfers.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200214091441.27535-4-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Workaround for RX teardown with stale data in peer

When a channel is asked to be stopped (teardown) and we do not have active
descriptor to receive stale data buffered on the remote side then the
teardown will not complete as UDMA needs a descriptor to be able to flush
out the DMA pipe.
The peer is trying to push the data to UDMA in teardown, but UDMA is
pushing back because it has no descriptor which would allow it to drain the
data.

The workaround is to create 1K 'trashcan' to receive the discarded data and
set up descriptors for packet and TR mode channels.
When a channel is stopped and there is no active descriptor then a
descriptor is pushed to the ring for UDMA before the teardown is initiated.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200214091441.27535-3-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Use ktime/usleep_range based TX completion check

In some cases (McSPI for example) the jiffie and delayed_work based
workaround can cause big throughput drop.

Switch to use ktime/usleep_range based implementation to be able
to sustain speed for PDMA based peripherals.

Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200214091441.27535-2-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: fix spelling mistake "limted" -> "limited"

There are spelling mistakes in dev_err messages. Fix them.

Signed-off-by: Colin Ian King <colin.king@canonical.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200122093818.2800743-1-colin.king@canonical.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: k3-udma: Add glue layer for non DMAengine users

Certain users can not use right now the DMAengine API due to missing
features in the core. Prime example is Networking.

These users can use the glue layer interface to avoid misuse of DMAengine
API and when the core gains the needed features they can be converted to
use generic API.

The most prominent features the glue layer clients are depending on:

- most PSI-L native peripheral use extra rflow ranges on a receive channel
and depending on the peripheral's configuration packets from a single
free descriptor ring is going to be received to different receive ring
- it is also possible to have different free descriptor rings per rflow
and an rflow can also support 4 additional free descriptor ring based
on the size of the incoming packet
- out of order completion of descriptors on a channel
- when we have several queues to handle different priority packets the
descriptors will be completed 'out-of-order'
- the notion of prep_slave_sg is not matching with what the streaming type
of operation is demanding for networking
- Streaming type of operation
- Ability to fill the free descriptor ring with descriptors in
anticipation of incoming traffic and when a packet arrives UDMAP will
form a packet and gives it to the client driver
- the descriptors are not backed with exact size data buffers as we don't
know the size of the packet we will receive, but as a generic pool of
buffers to be used by the receive channel
- NAPI type of operation (polling instead of interrupt driven transfer)
- without this we can not sustain gigabit speeds and we need to support NAPI
- not to limit this to networking, but other high performance operations

Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Tested-by: Keerthy <j-keerthy@ti.com>
Link: https://lore.kernel.org/r/20191223110458.30766-12-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

dmaengine: ti: New driver for K3 UDMA

Split patch for review containing: defines, structs, io and low level
functions and interrupt callbacks.

DMA driver for
Texas Instruments K3 NAVSS Unified DMA – Peripheral Root Complex (UDMA-P)

The UDMA-P is intended to perform similar (but significantly upgraded) functions
as the packet-oriented DMA used on previous SoC devices. The UDMA-P module
supports the transmission and reception of various packet types. The UDMA-P is
architected to facilitate the segmentation and reassembly of SoC DMA data
structure compliant packets to/from smaller data blocks that are natively
compatible with the specific requirements of each connected peripheral. Multiple
Tx and Rx channels are provided within the DMA which allow multiple segmentation
or reassembly operations to be ongoing. The DMA controller maintains state
information for each of the channels which allows packet segmentation and
reassembly operations to be time division multiplexed between channels in order
to share the underlying DMA hardware. An external DMA scheduler is used to
control the ordering and rate at which this multiplexing occurs for Transmit
operations. The ordering and rate of Receive operations is indirectly controlled
by the order in which blocks are pushed into the DMA on the Rx PSI-L interface.

The UDMA-P also supports acting as both a UTC and UDMA-C for its internal
channels. Channels in the UDMA-P can be configured to be either Packet-Based or
Third-Party channels on a channel by channel basis.

The initial driver supports:
- MEM_TO_MEM (TR mode)
- DEV_TO_MEM (Packet / TR mode)
- MEM_TO_DEV (Packet / TR mode)
- Cyclic (Packet / TR mode)
- Metadata for descriptors

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Tested-by: Keerthy <j-keerthy@ti.com>
Reviewed-by: Grygorii Strashko <grygorii.strashko@ti.com>
Link: https://lore.kernel.org/r/20191223110458.30766-11-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>