memory: tegra: Make CPU cluster BW request a multiple of MC channels

Make CPU cluster's bandwidth (BW) request a multiple of MC channels.
CPU OPP tables have BW info per MC channel. But, the actual BW depends
on the number of MC channels which can change as per the boot config.
Get the number of MC channels which are actually enabled in current
boot configuration and multiply the BW request from a CPU cluster with
the number of enabled MC channels. This is not required to be done for
other MC clients.

Signed-off-by: Sumit Gupta <sumitg@nvidia.com>
Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Signed-off-by: Thierry Reding <treding@nvidia.com>

memory: tegra: Add MC error logging on Tegra186 onward

Add support for logging memory controller errors on Tegra186, Tegra194
and Tegra234. On these SoCs, interrupts can occur on multiple channels.
Add support required to read the status of interrupts across multiple
channels, log and clear them.

Also add new interrupts supported on these SoCs.

Reviewed-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Ashish Mhetre <amhetre@nvidia.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20220506132312.3910637-5-thierry.reding@gmail.com
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>

memory: tegra: Add Tegra234 support

The memory controller and external memory controller found on Tegra234
is similar to the version found on earlier SoCs but supports a number of
new memory clients.

Add initial memory client definitions for the Tegra234 so that the SMMU
stream ID override registers can be properly programmed at boot time.

Signed-off-by: Thierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20220506132312.3910637-2-thierry.reding@gmail.com
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>

memory: tegra: Split Tegra194 data into separate file

Keep the directory structure consistent by splitting the Tegra194 data
into a separate file.

Signed-off-by: Thierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20210602163302.120041-13-thierry.reding@gmail.com
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>

memory: tegra: Unify drivers

The Tegra210 (and earlier) driver now supports all the functionality
that the Tegra186 (and later) driver does, so they can be unified.

Note that previously the Tegra186 (and later) driver could be unloaded,
even if that was perhaps not very useful. Older chips don't support that
yet, but once they do this code can be reenabled.

Signed-off-by: Thierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20210602163302.120041-11-thierry.reding@gmail.com
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>

memory: tegra: Parameterize interrupt handler

Tegra20 requires a slightly different interrupt handler than Tegra30 and
later, so parameterize the handler, so that each SoC implementation can
provide its own.

Signed-off-by: Thierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20210602163302.120041-8-thierry.reding@gmail.com
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>

memory: tegra: Extract setup code into callback

Separate the setup code for Tegra30 and later into a ->setup() callback
and set it for all applicable chips.

Signed-off-by: Thierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20210602163302.120041-7-thierry.reding@gmail.com
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>

memory: tegra20: Add debug statistics

Add debug statistics collection support. The statistics is available
via debugfs in '/sys/kernel/debug/mc/stats', it shows percent of memory
controller utilization for each memory client. This information is
intended to help with debugging of memory performance issues, it already
was proven to be useful by helping to improve memory bandwidth management
of the display driver.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
Link: https://lore.kernel.org/r/20210319130933.23261-1-digetx@gmail.com

memory: tegra-mc: Add interconnect framework

Add common SoC-agnostic ICC framework which turns Tegra Memory Controller
into a memory interconnection provider. This allows us to use interconnect
API for tuning of memory configurations.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Tested-by: Peter Geis <pgwipeout@gmail.com>
Tested-by: Nicolas Chauvet <kwizart@gmail.com>
Link: https://lore.kernel.org/r/20201104164923.21238-33-digetx@gmail.com
Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>

memory: tegra: Add EMC scaling support code for Tegra210

This is the initial patch for Tegra210 EMC frequency scaling. It has the
code to program various aspects of the EMC that are standardized, but it
does not yet include the specific programming sequence needed for clock
scaling.

The driver is designed to support LPDDR4 SDRAM. Devices that use LPDDR4
need to perform training of the RAM before it can be used. Firmware will
perform this training during early boot and pass a table of supported
frequencies to the kernel via device tree.

For the frequencies above 800 MHz, periodic retraining is needed to
compensate for changes in timing. This periodic training will have to be
performed until the frequency drops back to or below 800 MHz.

This driver provides helpers used during this runtime retraining that
will be used by the sequence specific code in a follow-up patch.

Based on work by Peter De Schrijver <pdeschrijver@nvidia.com>.

Signed-off-by: Joseph Lo <josephl@nvidia.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>

memory: tegra: Consolidate registers definition into common header

The Memory Controller registers definition is sparse and duplicated,
let's consolidate everything into a common place for consistency.

Acked-by: Peter De Schrijver <pdeschrijver@nvidia.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>

memory: tegra: Ensure timing control debug features are disabled

Timing control debug features should be disabled at a boot time, but you
never now and hence it's better to disable them explicitly because some of
those features are crucial for the driver to do a proper thing.

Acked-by: Peter De Schrijver <pdeschrijver@nvidia.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>

memory: tegra: Introduce Tegra30 EMC driver

Introduce driver for the External Memory Controller (EMC) found on Tegra30
chips, it controls the external DRAM on the board. The purpose of this
driver is to program memory timing for external memory on the EMC clock
rate change.

Acked-by: Peter De Schrijver <pdeschrijver@nvidia.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Tested-by: Peter Geis <pgwipeout@gmail.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500

Based on 2 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation #

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-only

has been chosen to replace the boilerplate/reference in 4122 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Enrico Weigelt <info@metux.net>
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Allison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

memory: tegra: Properly spell "tegra"

Rename all occurrences of "terga" to "tegra". It's an easy typo to make
and a difficult one to spot.

Reviewed-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>

memory: tegra: Use relaxed versions of readl/writel

There is no need for inserting of memory barriers to access registers of
Memory Controller. Hence use the relaxed versions of the accessors.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Acked-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

memory: tegra: Adapt to Tegra20 device-tree binding changes

The tegra20-mc device-tree binding has been changed, GART has been
squashed into Memory Controller and now the clock property is mandatory
for Tegra20, the DT compatible has been changed as well. Adapt driver to
the DT changes.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Acked-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

memory: tegra: Introduce memory client hot reset

In order to reset busy HW properly, memory controller needs to be
involved, otherwise it is possible to get corrupted memory or hang machine
if HW was reset during DMA. Introduce memory client 'hot reset' that will
be used for resetting of busy HW.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>

memory: tegra: Squash tegra20-mc into common tegra-mc driver

Tegra30+ has some minor differences in registers / bits layout compared
to Tegra20. Let's squash Tegra20 driver into the common tegra-mc driver
in a preparation for the upcoming MC hot reset controls implementation,
avoiding code duplication.

Note that this currently doesn't report the value of MC_GART_ERROR_REQ
because it is located within the GART register area and cannot be safely
accessed from the MC driver (this happens to work only by accident). The
proper solution is to integrate the GART driver with the MC driver, much
like is done for the Tegra SMMU, but that is an invasive change and will
be part of a separate patch series.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>

memory: tegra: Apply interrupts mask per SoC

Currently we are enabling handling of interrupts specific to Tegra124+
which happen to overlap with previous generations. Let's specify
interrupts mask per SoC generation for consistency and in a preparation
of squashing of Tegra20 driver into the common one that will enable
handling of GART faults which may be undesirable by newer generations.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>

memory: tegra: Add Tegra210 support

Add the table of memory clients and SWGROUPs for Tegra210 to enable SMMU
support for this new SoC.

Signed-off-by: Thierry Reding <treding@nvidia.com>

memory: tegra: Add Tegra132 support

The memory controller on Tegra132 is very similar to the one found on
Tegra124. But the Denver CPUs don't have an outer cache, so dcache
maintenance is done slightly differently.

Signed-off-by: Thierry Reding <treding@nvidia.com>

memory: Add NVIDIA Tegra memory controller support

The memory controller on NVIDIA Tegra exposes various knobs that can be
used to tune the behaviour of the clients attached to it.

Currently this driver sets up the latency allowance registers to the HW
defaults. Eventually an API should be exported by this driver (via a
custom API or a generic subsystem) to allow clients to register latency
requirements.

This driver also registers an IOMMU (SMMU) that's implemented by the
memory controller. It is supported on Tegra30, Tegra114 and Tegra124
currently. Tegra20 has a GART instead.

The Tegra SMMU operates on memory clients and SWGROUPs. A memory client
is a unidirectional, special-purpose DMA master. A SWGROUP represents a
set of memory clients that form a logical functional unit corresponding
to a single device. Typically a device has two clients: one client for
read transactions and one client for write transactions, but there are
also devices that have only read clients, but many of them (such as the
display controllers).

Because there is no 1:1 relationship between memory clients and devices
the driver keeps a table of memory clients and the SWGROUPs that they
belong to per SoC. Note that this is an exception and due to the fact
that the SMMU is tightly integrated with the rest of the Tegra SoC. The
use of these tables is discouraged in drivers for generic IOMMU devices
such as the ARM SMMU because the same IOMMU could be used in any number
of SoCs and keeping such tables for each SoC would not scale.

Acked-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Thierry Reding <treding@nvidia.com>