iommu/omap: Convert to generic_single_device_group()

Use the new helper.

For some reason omap will probe its driver even if it doesn't load an
iommu driver. Keep this working by keeping a bool to track if the iommu
driver was started.

Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Link: https://lore.kernel.org/r/7-v1-c869a95191f2+5e8-iommu_single_grp_jgg@nvidia.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: add support for late attachment of iommu devices

Current implementation of OMAP IOMMU enforces strict ordering of device
probe, initiated by iommu and followed by remoteproc later. This doesn't
work too well with the new setup done with ti-sysc changes which may
have the devices probed at pretty much any order. To overcome this limitation,
if iommu has not been probed yet when a consumer tries to attach to it,
add the device to orphan device list which will be parsed during iommu
probe to see if any orphan devices should be attached.

Signed-off-by: Tero Kristo <t-kristo@ti.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: add logic to save/restore locked TLBs

The MMUs provide a mechanism to lock TLB entries to avoid
eviction and fetching of frequently used page table entries.
These TLBs lose context when the MMUs are turned OFF. Add the
logic to save and restore these locked TLBS during suspend
and resume respectively. There are no locked TLBs during
initial power ON, and they need not be saved during final
shutdown.

Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: fix boot issue on remoteprocs with AMMU/Unicache

Support has been added to the OMAP IOMMU driver to fix a boot hang
issue on OMAP remoteprocs with AMMU/Unicache, caused by an improper
AMMU/Unicache state upon initial deassertion of the processor reset.
The issue is described in detail in the next three paragraphs.

All the Cortex M3/M4 IPU processor subsystems in OMAP SoCs have a
AMMU/Unicache IP that dictates the memory attributes for addresses
seen by the processor cores. The AMMU/Unicache is configured/enabled
by the SCACHE_CONFIG.BYPASS bit - a value of 1 enables the cache and
mandates all addresses accessed by M3/M4 be defined in the AMMU. This
bit is not programmable from the host processor. The M3/M4 boot
sequence starts out with the AMMU/Unicache in disabled state, and
SYS/BIOS programs the AMMU regions and enables the Unicache during
one of its initial boot steps. This SCACHE_CONFIG.BYPASS bit is
however enabled by default whenever a RET reset is applied to the IP,
irrespective of whether it was previously enabled or not. The AMMU
registers lose their context whenever this reset is applied. The reset
is effective as long as the MMU portion of the subsystem is enabled
and clocked. This behavior is common to all the IPU and DSP subsystems
that have an AMMU/Unicache.

The IPU boot sequence involves enabling and programming the MMU, and
loading the processor and releasing the reset(s) for the processor.
The PM setup code currently sets the target state for most of the
power domains to RET. The L2 MMU can be enabled, programmed and
accessed properly just fine with the domain in hardware supervised
mode, while the power domain goes through a RET->ON->RET transition
during the programming sequence. However, the ON->RET transition
asserts a RET reset, and the SCACHE_CONFIG.BYPASS bit gets auto-set.
An AMMU fault is thrown immediately when the M3/M4 core's reset is
released since the first instruction address itself will not be
defined in any valid AMMU regions. The ON->RET transition happens
automatically on the power domain after enabling the iommu due to
the hardware supervised mode.

This patch adds and invokes the .set_pwrdm_constraint pdata ops, if
present, during the OMAP IOMMU enable and disable functions to resolve
the above boot hang issue. The ops will allow to invoke a mach-omap2
layer API pwrdm_set_next_pwrst() in a multi-arch kernel environment.
The ops also returns the current power domain state while enforcing
the constraint so that the driver can store it and use it to set back
the power domain state while releasing the constraint. The pdata ops
implementation restricts the target power domain to ON during enable,
and back to the original power domain state during disable, and thereby
eliminating the conditions for the boot issue. The implementation is
effective only when the original power domain state is either RET or
OFF, and is a no-op when it is ON or INACTIVE.

The .set_pwrdm_constraint ops need to be plugged in pdata-quirks
for the affected remote processors to be able to boot properly.

Note that the current issue is seen only on kernels with the affected
power domains programmed to enter RET. For eg., IPU1 on DRA7xx is in a
separate domain and is susceptible to this bug, while the IPU2 subsystem
is within CORE power domain, and CORE RET is not supported on this SoC.
IPUs on OMAP4 and OMAP5 are also susceptible since they are in CORE power
domain, and CORE RET is a valid power target on these SoCs.

Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500

Based on 2 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation #

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-only

has been chosen to replace the boilerplate/reference in 4122 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Enrico Weigelt <info@metux.net>
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Allison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

iommu/omap: Add support to program multiple iommus

A client user instantiates and attaches to an iommu_domain to
program the OMAP IOMMU associated with the domain. The iommus
programmed by a client user are bound with the iommu_domain
through the user's device archdata. The OMAP IOMMU driver
currently supports only one IOMMU per IOMMU domain per user.

The OMAP IOMMU driver has been enhanced to support allowing
multiple IOMMUs to be programmed by a single client user. This
support is being added mainly to handle the DSP subsystems on
the DRA7xx SoCs, which have two MMUs within the same subsystem.
These MMUs provide translations for a processor core port and
an internal EDMA port. This support allows both the MMUs to
be programmed together, but with each one retaining it's own
internal state objects. The internal EDMA block is managed by
the software running on the DSPs, and this design provides
on-par functionality with previous generation OMAP DSPs where
the EDMA and the DSP core shared the same MMU.

The multiple iommus are expected to be provided through a
sentinel terminated array of omap_iommu_arch_data objects
through the client user's device archdata. The OMAP driver
core is enhanced to loop through the array of attached
iommus and program them for all common operations. The
sentinel-terminated logic is used so as to not change the
omap_iommu_arch_data structure.

NOTE:
1. The IOMMU group and IOMMU core registration is done only for
the DSP processor core MMU even though both MMUs are represented
by their own platform device and are probed individually. The
IOMMU device linking uses this registered MMU device. The struct
iommu_device for the second MMU is not used even though memory
for it is allocated.
2. The OMAP IOMMU debugfs code still continues to operate on
individual IOMMU objects.

Signed-off-by: Suman Anna <s-anna@ti.com>
[t-kristo@ti.com: ported support to 4.13 based kernel]
Signed-off-by: Tero Kristo <t-kristo@ti.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Use DMA-API for performing cache flushes

The OMAP IOMMU driver was using ARM assembly code directly for
flushing the MMU page table entries from the caches. This caused
MMU faults on OMAP4 (Cortex-A9 based SoCs) as L2 caches were not
handled due to the presence of a PL310 L2 Cache Controller. These
faults were however not seen on OMAP5/DRA7 SoCs (Cortex-A15 based
SoCs).

The OMAP IOMMU driver is adapted to use the DMA Streaming API
instead now to flush the page table/directory table entries from
the CPU caches. This ensures that the devices always see the
updated page table entries. The outer caches are now addressed
automatically with the usage of the DMA API.

Signed-off-by: Josue Albarran <j-albarran@ti.com>
Acked-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Add iommu-group support

Support for IOMMU groups will become mandatory for drivers,
so add it to the omap iommu driver.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
[s-anna@ti.com: minor error cleanups]
Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Make use of 'struct iommu_device'

Modify the driver to register individual iommus and
establish links between devices and iommus in sysfs.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
[s-anna@ti.com: fix some cleanup issues during failures]
Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Store iommu_dev pointer in arch_data

Instead of finding the matching IOMMU for a device using
string comparision functions, store the pointer to the
iommu_dev in arch_data during the omap_iommu_add_device
callback and reset it during the omap_iommu_remove_device
callback functions.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
[s-anna@ti.com: few minor cleanups]
Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Move data structures to omap-iommu.h

The internal data-structures are scattered over various
header and C files. Consolidate them in omap-iommu.h.

While at this, add the kerneldoc comment for the missing
iommu domain variable and revise the iommu_arch_data name.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
[s-anna@ti.com: revise kerneldoc comments]
Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Add support for configuring dsp iommus on DRA7xx

The DSP MMUs on DRA7xx SoC requires configuring an additional
MMU_CONFIG register present in the DSP_SYSTEM sub module. This
setting dictates whether the DSP Core's MDMA and EDMA traffic
is routed through the respective MMU or not. Add the support
to the OMAP iommu driver so that the traffic is not bypassed
when enabling the MMUs.

The MMU_CONFIG register has two different bits for enabling
each of these two MMUs present in the DSP processor sub-system
on DRA7xx. An id field is added to the OMAP iommu object to
identify and enable each IOMMU. The id information and the
DSP_SYSTEM.MMU_CONFIG register programming is achieved through
the processing of the optional "ti,syscon-mmuconfig" property.
A proper value is assigned to the id field only when this
property is present.

Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Use BIT(x) macros in omap-iommu.h

Switch to using the BIT(x) macros in omap-iommu.h where
possible. This eliminates the following checkpatch check
warning:
"CHECK: Prefer using the BIT macro"

A couple of the warnings were ignored for better readability
of the bit-shift for the different values.

Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Remove unused union fields

There are couple of unions defined in the structures
iotlb_entry and cr_regs. There are no usage/references
to some of these union fields in the code, so clean
them up and simplify the structures.

Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Move debugfs functions to omap-iommu-debug.c

The main OMAP IOMMU driver file has some helper functions used
by the OMAP IOMMU debugfs functionality, and there is already a
dedicated source file omap-iommu-debug.c dealing with these debugfs
routines. Move all these functions to the omap-iommu-debug.c file,
so that all the debugfs related routines are in one place.

The move required exposing some new functions and moving some
definitions to the internal omap-iommu.h header file.

Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Do not export unneeded functions

The following functions were exported previously for usage by
the OMAP IOMMU debug module:
omap_iommu_dump_ctx()
omap_dump_tlb_entries()
omap_iopgtable_store_entry()

These functions need not be exported anymore as the OMAP IOMMU
debugfs code is integrated with the OMAP IOMMU driver, and
there won't be external users for these functions. So, remove
the EXPORT_SYMBOL_GPL on these. The omap_iopgtable_store_entry()
is also made internal only, after making the 'pagetable' debugfs
entry read-only.

Signed-off-by: Suman Anna <s-anna@ti.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Remove couple of unused exported functions

The exported functions omap_foreach_iommu_device() and
omap_iotlb_cr_to_e() have been deleted, as they are no
longer needed.

The function omap_foreach_iommu_device() is not required
after the consolidation of the OMAP IOMMU debug module,
and the function omap_iotlb_cr_to_e() is not required
after making the debugfs entry 'pagetable' read-only.

Signed-off-by: Suman Anna <s-anna@ti.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Integrate omap-iommu-debug into omap-iommu

The debugfs support for OMAP IOMMU is currently implemented
as a module, warranting certain OMAP-specific IOMMU API to
be exported. The OMAP IOMMU, when enabled, can only be built-in
into the kernel, so integrate the OMAP IOMMU debug module
into the OMAP IOMMU driver. This helps in eliminating the
need to export most of the current OMAP IOMMU API.

The following are the main changes:
- The debugfs directory and entry creation logic is reversed,
the calls are invoked by the OMAP IOMMU driver now.
- The current iffy circular logic of adding IOMMU archdata
to the IOMMU devices itself to get a pointer to the omap_iommu
object in the debugfs support code is replaced by directly
using the omap_iommu structure while creating the debugfs
entries.
- The debugfs root directory is renamed from the generic name
"iommu" to a specific name "omap_iommu".
- Unneeded headers have also been cleaned up while at this.
- There will no longer be a omap-iommu-debug.ko module after
this patch.
- The OMAP_IOMMU_DEBUG Kconfig option is converted to boolean
only, the OMAP IOMMU debugfs support is built alongside the
OMAP IOMMU driver only when this option is enabled.

Signed-off-by: Suman Anna <s-anna@ti.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Consolidate OMAP IOMMU modules

The OMAP IOMMU driver was originally designed as modules, and split
into a core module and a thin arch-specific module through the OMAP
arch-specific struct iommu_functions, to scale for both OMAP1 and
OMAP2+ IOMMU variants. The driver can only be built for OMAP2+
platforms currently, and also can only be built-in after the
adaptation to generic IOMMU API. The OMAP1 variant was never added
and will most probably be never added (the code for the only potential
user, its parent, DSP processor has already been cleaned up). So,
consolidate the OMAP2 specific omap-iommu2 module into the core OMAP
IOMMU driver - this eliminates the arch-specific ops structure and
simplifies the driver into a single module that only implements the
generic IOMMU API's iommu_ops.

The following are the main changes:
- omap-iommu2 module is completely eliminated, with the common
definitions moved to the internal omap-iommu.h, and the ops
implementations moved into omap-iommu.c
- OMAP arch-specific struct iommu_functions is also eliminated,
with the ops implementations directly absorbed into the calling
functions
- iotlb_alloc_cr() is no longer inlined and defined only when
PREFETCH_IOTLB is defined
- iotlb_dump_cr() is similarly defined only when CONFIG_OMAP_IOMMU_DEBUG
is defined
- Elimination of the OMAP IOMMU exported functions to register the
arch ops, omap_install_iommu_arch() & omap_uninstall_iommu_arch()
- Any stale comments about OMAP1 are also cleaned up

Signed-off-by: Suman Anna <s-anna@ti.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Remove omap_iommu_arch_version() and version field

The function omap_iommu_arch_version() is not used anymore,
and is not required either, so remove it. The .version field
in struct iommu_functions that this function uses is also
removed, as it is not really an ops to retrieve a version and
there won't be any usage for this field either.

Signed-off-by: Suman Anna <s-anna@ti.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Remove conditional definition of dev_to_omap_iommu()

The dev_to_omap_iommu() is local to the OMAP IOMMU modules, and
need not be defined conditionally. The CONFIG_IOMMU_API dependency
check was added in the past to fix a compilation issue back when
the header resided in the arch/arm layers, and is no longer
needed.

While at this, fix the header against double inclusion as well.

Signed-off-by: Suman Anna <s-anna@ti.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Remove duplicate declarations

The omap_iommu_save_ctx() and omap_iommu_restore_ctx() declarations
are defined in include/linux/omap-iommu.h and do not belong in the
internal drivers/iommu/omap-iommu.h header, so remove them.

Signed-off-by: Suman Anna <s-anna@ti.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Remove unused isr_priv field from omap_iommu

The isr_priv field is a left-over from before the IOMMU API
adaptation, this was used to store the callback data. This is
no longer relevant, so remove it.

Signed-off-by: Suman Anna <s-anna@ti.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Remove refcount field from omap_iommu object

The refcount field in omap_iommu object is primarily used to check
if an IOMMU device has already been enabled, but this is already
implicit in the omap_iommu_attach_dev() which ensures that only
a single device can attach to an IOMMU. This field is redundant,
and so has been cleaned up.

Signed-off-by: Suman Anna <s-anna@ti.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Remove omap_iommu unused owner field

The owner field is never set. Remove it.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Remove virtual memory manager

The OMAP3 ISP driver was the only user of the OMAP IOVMM API. Now that
is has been ported to the DMA API, remove the unused virtual memory
manager.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>

iommu/omap: Enable bus-error back on supported iommus

The remoteproc MMUs in OMAP4+ SoCs have some additional debug
registers that can give out the PC value in addition to the
MMU fault address. The PC value can be extracted properly only
on the DSP cores, and is not available on the ARM processors
within the IPU sub-systems. Instead, the MMUs have been enhanced
to throw a bus-error response back to the IPU processors.

This functionality is programmable through the MMU_GP_REG register.
The cores are simply stalled if the MMU_GP_REG.BUS_ERR_BACK_EN bit
is not set. When set, a bus-error exception is raised allowing the
processor to handle it as a bus fault and provide additional debug
information. This feature is turned on by default by the driver on
iommus supporting it.

Signed-off-by: Subramaniam Chanderashekarapuram <subramaniam.ca@ti.com>
Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>

iommu/omap: Adapt to runtime pm

Use runtime PM functionality interfaced with hwmod enable/idle
functions, to replace direct clock operations and sysconfig
handling.

Due to reset sequence, pm_runtime_[get|put]_sync must be used, to
avoid possible operations with the module under reset. Because of
this and given that the driver uses spin_locks to protect their
critical sections, we must use pm_runtime_irq_safe in order for the
runtime ops to be happy, otherwise might_sleep_if checks in runtime
framework will complain.

The remaining pm_runtime out of iommu_enable and iommu_disable
corresponds to paths that can be accessed through debugfs, some of
them doesn't work if the module is not enabled first, but in future
if the mmu is idled withouth freeing, these are needed to debug.

Signed-off-by: Omar Ramirez Luna <omar.luna@linaro.org>
Tested-by: Ohad Ben-Cohen <ohad@wizery.com>
Acked-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Joerg Roedel <joro@8bytes.org>

ARM: OMAP2+: Make some definitions local

Move some of the definitions in omap-iommu.h that can be made local to
either drivers/iommu.

Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Mauro Carvalho Chehab <mchehab@infradead.org>
Cc: Omar Ramirez Luna <omar.luna@linaro.org>
Signed-off-by: Ido Yariv <ido@wizery.com>
Acked-by: Ohad Ben-Cohen <ohad@wizery.com>
Acked-by: Joerg Roedel <joro@8bytes.org>
[tony@atomide.com: updated for header changes in the series]
Signed-off-by: Tony Lindgren <tony@atomide.com>

ARM: OMAP2+: Move iommu2 to drivers/iommu/omap-iommu2.c

This file should not be in arch/arm. Move it to drivers/iommu
to allow making most of the header local to drivers/iommu.

This is needed as we are removing plat and mach includes
from drivers for ARM common zImage support.

Cc: Ido Yariv <ido@wizery.com>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Mauro Carvalho Chehab <mchehab@infradead.org>
Cc: Omar Ramirez Luna <omar.luna@linaro.org>
Cc: linux-media@vger.kernel.org
Acked-by: Ohad Ben-Cohen <ohad@wizery.com>
Acked-by: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Tony Lindgren <tony@atomide.com>