PCI: dwc: Add dw_pcie_ep_{read,write}_dbi[2] helpers

The current code calculated some dbi[2] registers' offset by calling
dw_pcie_ep_get_dbi[2]_offset() in each function. To improve the code
readability, add dw_pcie_ep_{read,write}_dbi[2} and some data-width
related helpers.

Link: https://lore.kernel.org/linux-pci/20231220053829.1921187-5-yoshihiro.shimoda.uh@renesas.com
Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: Rename .func_conf_select to .get_dbi_offset in struct dw_pcie_ep_ops

Since the struct member .func_conf_select makes the intentions behind it
difficult to ascertain from its name alone, rename it to .get_dbi_offset
to make the intended usage more obvious.

[kwilczynski: commmit log]
Suggested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Link: https://lore.kernel.org/linux-pci/20231220053829.1921187-4-yoshihiro.shimoda.uh@renesas.com
Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>

PCI: dwc: Rename .ep_init to .init in struct dw_pcie_ep_ops

Since the name of the dw_pcie_ep_ops struct makes it obvious that it's
for the PCIe Endpoint, rename the struct member .ep_init to .init.

[kwilczynski: commit log]
Suggested-by: Serge Semin <fancer.lancer@gmail.com>
Suggested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Link: https://lore.kernel.org/linux-pci/20231220053829.1921187-3-yoshihiro.shimoda.uh@renesas.com
Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Srikanth Thokala <srikanth.thokala@intel.com>
Acked-by: Jesper Nilsson <jesper.nilsson@axis.com>
Acked-by: Kunihiko Hayashi <hayashi.kunihiko@socionext.com>

PCI: dwc: Drop host prefix from struct dw_pcie_host_ops members

Since the name of the dw_pcie_host_ops struct makes it obvious that it's
for the PCIe Host, drop the host prefix from the struct members.

[kwilczynski: commit log]
Suggested-by: Serge Semin <fancer.lancer@gmail.com>
Link: https://lore.kernel.org/linux-pci/20231220053829.1921187-2-yoshihiro.shimoda.uh@renesas.com
Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: Heiko Stuebner <heiko@sntech.de>
Acked-by: Jesper Nilsson <jesper.nilsson@axis.com>
Acked-by: Kunihiko Hayashi <hayashi.kunihiko@socionext.com>
Acked-by: Lei Chuanhua <lchuanhua@maxlinear.com>
Acked-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>

PCI: dwc: Rename dw_pcie_ep_raise_legacy_irq()

Rename the function dw_pcie_ep_raise_legacy_irq() of the Designware
endpoint controller driver to dw_pcie_ep_raise_intx_irq() to match the
name of the PCI_IRQ_INTX macro.

Link: https://lore.kernel.org/r/20231122060406.14695-10-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

PCI: endpoint: Drop PCI_EPC_IRQ_XXX definitions

linux/pci.h defines the IRQ flags PCI_IRQ_INTX, PCI_IRQ_MSI and
PCI_IRQ_MSIX. Let's use these flags directly instead of the endpoint
definitions provided by enum pci_epc_irq_type. This removes the need
for defining this enum type completely.

Link: https://lore.kernel.org/r/20231122060406.14695-3-dlemoal@kernel.org
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Acked-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: endpoint: Introduce .pre_init() and .deinit()

Renesas R-Car Gen4 PCIe controllers require vendor-specific
initialization before .init().

To use dw->dbi and dw->num-lanes in the initialization code,
introduce .pre_init() into struct dw_pcie_ep_ops. While at it,
also introduce .deinit() to disable the controller by using
vendor-specific de-initialization.

Note that the ep_init in the struct dw_pcie_ep_ops should be
renamed to init later.

Link: https://lore.kernel.org/linux-pci/20231018085631.1121289-9-yoshihiro.shimoda.uh@renesas.com
Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>

PCI: dwc: Add EDMA_UNROLL capability flag

Renesas R-Car Gen4 PCIe controllers have an unexpected register value in
the eDMA CTRL register.

So, add a new capability flag "EDMA_UNROLL" which would force the unrolled
eDMA mapping for the problematic device.

Suggested-by: Serge Semin <fancer.lancer@gmail.com>
Link: https://lore.kernel.org/linux-pci/20231018085631.1121289-7-yoshihiro.shimoda.uh@renesas.com
Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>

PCI: dwc: endpoint: Add multiple PFs support for dbi2

The commit 24ede430fa49 ("PCI: designware-ep: Add multiple PFs support
for DWC") added .func_conf_select() to get the configuration space of
different PFs and assumed that the offsets between dbi and dbi2 would
be the same.

However, Renesas R-Car Gen4 PCIe controllers have different offsets of
function 1: dbi (+0x1000) and dbi2 (+0x800). To get the offset for dbi2,
add .get_dbi2_offset() and dw_pcie_ep_get_dbi2_offset().

Note:
- .func_conf_select() should be renamed later.
- dw_pcie_ep_get_dbi2_offset() will call .func_conf_select()
if .get_dbi2_offset() doesn't exist for backward compatibility.
- dw_pcie_writeX_{dbi/dbi2} APIs accepted the func_no argument,
so that these offset calculations are contained in the API
definitions itself as it should.

Link: https://lore.kernel.org/linux-pci/20231018085631.1121289-6-yoshihiro.shimoda.uh@renesas.com
Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: Add host_post_init() callback

This callback can be used by the platform drivers to do configuration
once all the devices are scanned. Like changing LNKCTL of all downstream
devices to enable ASPM etc...

Link: https://lore.kernel.org/linux-pci/20231010155914.9516-2-manivannan.sadhasivam@linaro.org
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org>

PCI: dwc: Implement generic suspend/resume functionality

Introduce an helper function (dw_pcie_get_ltssm()) to retrieve
SMLH_LTSS_STATE.

Add common dw_pcie_suspend(resume)_noirq() API to implement the DWC
controller generic suspend/resume functionality.

Add a controller specific callback to send the PME_Turn_Off message
(ie .pme_turn_off) for controller platform specific PME handling.

Link: https://lore.kernel.org/r/20230821184815.2167131-3-Frank.Li@nxp.com
Signed-off-by: Frank Li <Frank.Li@nxp.com>
[lpieralisi@kernel.org: commit log]
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Acked-by: Manivannan Sadhasivam <mani@kernel.org>

Revert "PCI: dwc: Wait for link up only if link is started"

This reverts commit da56a1bfbab55189595e588f1d984bdfb5cf5924.

Bjorn Andersson, Fabio Estevam, Xiaolei Wang, and Jon Hunter reported that
da56a1bfbab5 ("PCI: dwc: Wait for link up only if link is started") broke
controller probing by returning an error in case the link does not come up
during host initialisation, for example when the slot is empty.

As explained in commit 886a9c134755 ("PCI: dwc: Move link handling into
common code") and as indicated by the comment "Ignore errors, the link may
come up later" in the code, waiting for link up and ignoring errors is the
intended behaviour:

Let's standardize this to succeed as there are usecases where devices
(and the link) appear later even without hotplug. For example, a
reconfigured FPGA device.

Reverting the offending commit specifically fixes a regression on Qualcomm
platforms like the Lenovo ThinkPad X13s which no longer reach the
interconnect sync state if a slot does not have a device populated (e.g. an
optional modem).

Note that enabling asynchronous probing by default as was done for Qualcomm
platforms by commit c0e1eb441b1d ("PCI: qcom: Enable async probe by
default"), should take care of any related boot time concerns.

Finally, note that the intel-gw driver is the only driver currently not
providing a .start_link() callback and instead starts the link in its
.host_init() callback, which may avoid an additional one-second timeout
during probe by making the link-up wait conditional. If anyone cares, that
can be done in a follow-up patch with a proper motivation.

[bhelgaas: add Fabio Estevam, Xiaolei Wang, Jon Hunter reports]
Fixes: da56a1bfbab5 ("PCI: dwc: Wait for link up only if link is started")
Link: https://lore.kernel.org/r/20230706082610.26584-1-johan+linaro@kernel.org
Reported-by: Bjorn Andersson <quic_bjorande@quicinc.com>
Signed-off-by: Johan Hovold <johan+linaro@kernel.org>
Reported-by: Fabio Estevam <festevam@gmail.com>
Link: https://lore.kernel.org/r/20230704122635.1362156-1-festevam@gmail.com/
Reported-by: Xiaolei Wang <xiaolei.wang@windriver.com>
Link: https://lore.kernel.org/r/20230705010624.3912934-1-xiaolei.wang@windriver.com/
Reported-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://lore.kernel.org/r/6ca287a1-6c7c-7b90-9022-9e73fb82b564@nvidia.com
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Cc: Sajid Dalvi <sdalvi@google.com>
Cc: Ajay Agarwal <ajayagarwal@google.com>

PCI: dwc: Wait for link up only if link is started

In dw_pcie_host_init() regardless of whether the link has been
started or not, the code waits for the link to come up. Even in
cases where start_link() is not defined the code ends up spinning
in a loop for 1 second. Since in some systems dw_pcie_host_init()
gets called during probe, this one second loop for each pcie
interface instance ends up extending the boot time.

Wait for the link up in only if the start_link() is defined.

Link: https://lore.kernel.org/r/20230412093425.3659088-1-ajayagarwal@google.com
Tested-by: Will McVicker <willmcvicker@google.com>
Signed-off-by: Sajid Dalvi <sdalvi@google.com>
Signed-off-by: Ajay Agarwal <ajayagarwal@google.com>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>

PCI: dwc: Add Root Port and Endpoint controller eDMA engine support

Since the DW eDMA core now supports eDMA controllers embedded in locally
accessible DW PCIe Root Ports and Endpoints, register these controllers
when possible.

To do that the DW PCIe core driver needs to perform some preparations
first. First of all, it needs to find the eDMA controller CSRs base
address, whether they are accessible over the Port Logic or iATU unrolled
space. Afterwards it can try to auto-detect the eDMA controller
availability and number of read/write channels. If none are found the
procedure silently returns without error.

Secondly, the platform is supposed to provide either combined or
per-channel IRQ signals. If no valid IRQs set is found, the procedure
returns without error to be backward compatible with platforms where DW
PCIe controllers have eDMA but lack the IRQ description.

Finally, before actually probing the eDMA device we need to allocate LLP
items buffers. After that the DW eDMA can be registered. If registration is
successful, a message regarding the number of detected Read/Write eDMA
channels will be printed to the system as is done for the iATU settings.

Note: the DW PCI controller driver (either host or endpoint mode) is
currently always built-in, so if the DW eDMA core is built as a module
(CONFIG_DW_EDMA=m), eDMA controllers will not be registered even if the
dw-edma module is later loaded.

Link: https://lore.kernel.org/r/20230113171409.30470-28-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>

PCI: dwc: Introduce generic platform clocks and resets

Currently almost each platform driver uses its own resets and clocks
naming in order to get the corresponding descriptors. It makes the code
harder to maintain and comprehend especially seeing the DWC PCIe core main
resets and clocks signals set hasn't changed much for about at least one
major IP-core release. So in order to organize things around these signals
we suggest to create a generic interface for them in accordance with the
naming introduced in the DWC PCIe IP-core reference manual:

Application clocks:
- "dbi" - data bus interface clock (on some DWC PCIe platforms it's
referred as "pclk", "pcie", "sys", "ahb", "cfg", "iface",
"gio", "reg", "pcie_apb_sys");
- "mstr" - AXI-bus master interface clock (some DWC PCIe glue drivers
refer to this clock as "port", "bus", "pcie_bus",
"bus_master/master_bus/axi_m", "pcie_aclk");
- "slv" - AXI-bus slave interface clock (also called as "port", "bus",
"pcie_bus", "bus_slave/slave_bus/axi_s", "pcie_aclk",
"pcie_inbound_axi").

Core clocks:
- "pipe" - core-PCS PIPE interface clock coming from external PHY (it's
normally named by the platform drivers as just "pipe");
- "core" - primary clock of the controller (none of the platform drivers
declare such a clock but in accordance with the ref. manual
the devices may have it separately specified);
- "aux" - auxiliary PMC domain clock (it is named by some platforms as
"pcie_aux" and just "aux");
- "ref" - Generic reference clock (it is a generic clock source, which
can be used as a signal source for multiple interfaces, some
platforms call it as "ref", "general", "pcie_phy",
"pcie_phy_ref").

Application resets:
- "dbi" - Data-bus interface reset (it's CSR interface clock and is
normally called as "apb" though technically it's not APB but
DWC PCIe-specific interface);
- "mstr" - AXI-bus master reset (some platforms call it as "port", "apps",
"bus", "axi_m");
- "slv" - ABI-bus slave reset (some platforms call it as "port", "apps",
"bus", "axi_s").

Core resets:
- "non-sticky" - non-sticky CSR flags reset;
- "sticky" - sticky CSR flags reset;
- "pipe" - PIPE-interface (Core-PCS) logic reset (some platforms
call it just "pipe");
- "core" - controller primary reset (resets everything except PMC
module, some platforms refer to this signal as "soft",
"pci");
- "phy" - PCS/PHY block reset (strictly speaking it is normally
connected to the input of an external block, but the
reference manual says it must be available for the PMC
working correctly, some existing platforms call it
"pciephy", "phy", "link");
- "hot" - PMC hot reset signal (also called as "sleep");
- "pwr" - cold reset signal (can be referred as "pwr", "turnoff").

Bus reset:
- "perst" - PCIe standard signal used to reset the PCIe peripheral
devices.

As you can see each platform uses it's own naming for basically the same
set of the signals. In the framework of this commit we suggest to add a
set of the clocks and reset signals resources, corresponding names and
identifiers for each denoted entity. At current stage the platforms will
be able to use the provided infrastructure to automatically request all
these resources and manipulate with them in the Host/EP init callbacks.
Alas it isn't that easy to create a common cold/hot reset procedure due to
too many platform-specifics in the procedure, like the external flags
exposure and the delays requirement.

Link: https://lore.kernel.org/r/20221113191301.5526-20-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>

PCI: dwc: Introduce generic resources getter

Currently the DW PCIe Root Port and Endpoint CSR spaces are retrieved in
the separate parts of the DW PCIe core driver. It doesn't really make
sense since the both controller types have identical set of the core CSR
regions: DBI, DBI CS2 and iATU/eDMA. Thus we can simplify the DW PCIe Host
and EP initialization methods by moving the platform-specific registers
space getting and mapping into a common method. It gets to be even more
justified seeing the CSRs base address pointers are preserved in the
common DW PCIe descriptor. Note all the OF-based common DW PCIe settings
initialization will be moved to the new method too in order to have a
single function for all the generic platform properties handling in single
place.

A nice side-effect of this change is that the pcie-designware-host.c and
pcie-designware-ep.c drivers are cleaned up from all the direct dw_pcie
storage modification, which makes the DW PCIe core, Root Port and Endpoint
modules more coherent.

Link: https://lore.kernel.org/r/20221113191301.5526-18-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Reviewed-by: Rob Herring <robh@kernel.org>

PCI: dwc: Introduce generic controller capabilities interface

Since in addition to the already available iATU unrolled mapping we are
about to add a few more DW PCIe platform-specific capabilities (CDM-check
and generic clocks/resets resources) let's add a generic interface to set
and get the flags indicating their availability. The new interface shall
improve maintainability of the platform-specific code.

Link: https://lore.kernel.org/r/20221113191301.5526-17-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: Introduce dma-ranges property support for RC-host

In accordance with the generic PCIe Root Port DT-bindings the "dma-ranges"
property has the same format as the "ranges" property. The only difference
is in their semantics. The "dma-ranges" property describes the PCIe-to-CPU
memory mapping in opposite to the CPU-to-PCIe mapping of the "ranges"
property. Even though the DW PCIe controllers are normally equipped with
the internal Address Translation Unit which inbound and outbound tables
can be used to implement both properties semantics, it was surprising for
me to discover that the host-related part of the DW PCIe driver currently
supports the "ranges" property only while the "dma-ranges" windows are
just ignored. Having the "dma-ranges" supported in the driver would be
very handy for the platforms, that don't tolerate the 1:1 CPU-PCIe memory
mapping and require a customized PCIe memory layout. So let's fix that by
introducing the "dma-ranges" property support.

First of all we suggest to rename the dw_pcie_prog_inbound_atu() method to
dw_pcie_prog_ep_inbound_atu() and create a new version of the
dw_pcie_prog_inbound_atu() function. Thus we'll have two methods for the
RC and EP controllers respectively in the same way as it has been
developed for the outbound ATU setup methods.

Secondly aside with the memory window index and type the new
dw_pcie_prog_inbound_atu() function will accept CPU address, PCIe address
and size as its arguments. These parameters define the PCIe and CPU memory
ranges which will be used to setup the respective inbound ATU mapping. The
passed parameters need to be verified against the ATU ranges constraints
in the same way as it is done for the outbound ranges.

Finally the DMA-ranges detected for the PCIe controller need to be
converted to the inbound ATU entries during the host controller
initialization procedure. It will be done in the framework of the
dw_pcie_iatu_setup() method. Note before setting the inbound ranges up we
need to disable all the inbound ATU entries in order to prevent unexpected
PCIe TLPs translations defined by some third party software like
bootloaders.

Link: https://lore.kernel.org/r/20221113191301.5526-16-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: Drop dependency on ZONE_DMA32

Re-work the msi_msg DMA allocation logic to use dmam_alloc_coherent() which
uses the coherent DMA mask to try to return an allocation within the DMA
mask limits. With that, we now can drop the msi_page parameter in struct
dw_pcie_rp. This allows kernel configurations that disable ZONE_DMA32 to
continue supporting a 32-bit DMA mask. Without this patch, the PCIe host
device will fail to probe when ZONE_DMA32 is disabled.

Link: https://lore.kernel.org/r/20220825235404.4132818-2-willmcvicker@google.com
Fixes: 35797e672ff0 ("PCI: dwc: Fix MSI msi_msg DMA mapping")
Reported-by: Isaac J. Manjarres <isaacmanjarres@google.com>
Signed-off-by: Will McVicker <willmcvicker@google.com>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Reviewed-by: Rob Herring <robh@kernel.org>
Acked-by: Jingoo Han <jingoohan1@gmail.com>

PCI: dwc: Convert struct pcie_port.msi_irq to an array

The Qualcomm DWC PCIe controller supports more than 32 MSI interrupts, but
they are routed to separate interrupts in groups of 32 vectors. To support
this configuration, change the msi_irq field to an array. Let the DWC core
handle all interrupts that were set in this array.

[bhelgaas: reorder, drop "irq" temporary to make patch cleaner]
Link: https://lore.kernel.org/r/20220707134733.2436629-3-dmitry.baryshkov@linaro.org
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Johan Hovold <johan+linaro@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: Fix MSI msi_msg DMA mapping

As of 07940c369a6b ("PCI: dwc: Fix MSI page leakage in suspend/resume"),
the PCIe designware host driver has been using the driver data allocation
for the msi_msg DMA mapping which can result in a DMA_MAPPING_ERROR due to
the DMA overflow check in dma_direct_map_page() when the address is greater
than 32 bits (reported in [1]). The commit was trying to address a memory
leak on suspend/resume by moving the MSI mapping to dw_pcie_host_init(),
but subsequently dropped the page allocation thinking it wasn't needed.

To fix the DMA mapping issue as well as make msi_msg DMA'able, switch back
to allocating a 32-bit page for the msi_msg. To avoid the suspend/resume
leak, allocate the page in dw_pcie_host_init() since that shouldn't be
called during suspend/resume.

[1] https://lore.kernel.org/all/Yo0soniFborDl7+C@google.com/

Signed-off-by: Will McVicker <willmcvicker@google.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>

PCI: dwc: Check iATU in/outbound range setup status

Make the DWC PCIe RC/EP safer and more verbose for invalid or failed
inbound and outbound iATU window setups. Silently ignoring iATU regions
setup errors may cause unpredictable errors. For instance if a cfg or IO
window fails to be activated, then any CFG/IO requested won't reach target
PCIe devices and the corresponding accessors will return platform-specific
random values.

[bhelgaas: trim commit log]
Link: https://lore.kernel.org/r/20220624143947.8991-15-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: Validate iATU outbound mappings against hardware constraints

Make __dw_pcie_prog_outbound_atu() check the requested region base and size
against what the hardware can support. Return error if the region is not
correctly aligned or of a supported size.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20220624143947.8991-14-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: Add iATU regions size detection procedure

The DWC PCIe RC/EP/DM IP core configuration parameters determine the number
of inbound and outbound iATU windows, alignment requirements (which is also
the minimum window size), minimum and maximum sizes. If internal ATU is
enabled, the former settings are determined by CX_ATU_MIN_REGION_SIZE; the
latter are determined by CX_ATU_MAX_REGION_SIZE.

Determine the required alignment and maximum size supported by the
controller and log it to help verify whether the requested inbound or
outbound memory mappings can be fully created.

Note 1. The extended iATU regions have been supported since DWC PCIe
v4.60a. There is no need in testing the upper limit register availability
for the older cores.

Note 2. The regions alignment is determined with using the fls() method
since the lower four bits of the ATU Limit register can be occupied with
the Circular Buffer Increment setting, which can be initialized with zeros.

Link: https://lore.kernel.org/r/20220624143947.8991-13-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: Simplify in/outbound iATU setup methods

Previously __dw_pcie_prog_outbound_atu() duplicated a lot of code between
the iatu_unroll_enabled version and the PCIE_ATU_VIEWPORT version:

__dw_pcie_prog_outbound_atu
if (iatu_unroll_enabled)
dw_pcie_prog_outbound_atu_unroll
dw_pcie_writel_ob_unroll(PCIE_ATU_UNR_LOWER_BASE, ...)
dw_pcie_writel_ob_unroll(PCIE_ATU_UNR_UPPER_BASE, ...)
...
return
dw_pcie_writel_dbi(PCIE_ATU_VIEWPORT, ...)
dw_pcie_writel_dbi(PCIE_ATU_LOWER_BASE, ...)
dw_pcie_writel_dbi(PCIE_ATU_UPPER_BASE, ...)
...

Unify those by pushing the unroll address computation and viewport
selection down into dw_pcie_writel_atu() so we can use the same
dw_pcie_writel_atu_ob() accessor for both paths:

__dw_pcie_prog_outbound_atu
dw_pcie_writel_atu_ob(PCIE_ATU_LOWER_BASE, ...)
dw_pcie_writel_atu
dw_pcie_select_atu # new
if (iatu_unroll_enabled)
return pci->atu_base + PCIE_ATU_UNROLL_BASE(...)
dw_pcie_writel_dbi(PCIE_ATU_VIEWPORT, ...)
return pci->atu_base
dw_pcie_write(base + reg)
dw_pcie_writel_atu_ob(PCIE_ATU_UPPER_BASE, ...)
...

In the non-unroll case, this does involve more MMIO writes to
PCIE_ATU_VIEWPORT, but it's mainly in initialization paths and the code
simplification is significant.

[bhelgaas: commit log, simplify dw_pcie_select_atu()]
Link: https://lore.kernel.org/r/20220624143947.8991-12-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: dwc: Drop enum dw_pcie_region_type in favor of PCIE_ATU_REGION_DIR_IB/OB

Previously callers of dw_pcie_disable_atu() supplied enum
dw_pcie_region_type (DW_PCIE_REGION_INBOUND, DW_PCIE_REGION_OUTBOUND),
which dw_pcie_disable_atu() converted to the PCIE_ATU_REGION_DIR_IB or
PCIE_ATU_REGION_DIR_OB values needed to program the ATU registers.

Simplify the code by dropping the dw_pcie_region_type enum and passing
PCIE_ATU_REGION_DIR_IB or PCIE_ATU_REGION_DIR_OB directly.

Reorder dw_pcie_disable_atu() arguments to (dir, index) since "index"
indicates an ATU window in the regions of the corresponding direction.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20220624143947.8991-11-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: Drop enum dw_pcie_as_type in favor of PCIE_ATU_TYPE_MEM/IO

Previously dw_pcie_ep_set_bar() converted the BAR PCI_BASE_ADDRESS_SPACE
bit to the internal dw_pcie_as_type enum (DW_PCIE_AS_MEM, DW_PCIE_AS_IO)
and passed it down to dw_pcie_prog_inbound_atu(), which converted the enum
to the PCIE_ATU_TYPE_MEM/PCIE_ATU_TYPE_IO values needed to program the ATU
registers.

Simplify the code by dropping the dw_pcie_as_type enum and passing
PCIE_ATU_TYPE_MEM or PCIE_ATU_TYPE_IO directly.

Reorder inbound ATU function arguments to match the outbound functions,
with address-related parameters at the end.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20220624143947.8991-10-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: Add dw_pcie_ops.host_deinit() callback

dw_pcie_host_init() calls the dw_pcie_ops.host_init() callback to do
platform-specific host initialization.

Add a dw_pcie_ops.host_deinit() callback to perform the corresponding
cleanups in dw_pcie_host_deinit() and in dw_pcie_host_init() failure paths.

Link: https://lore.kernel.org/r/20220624143947.8991-9-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: Add macros to compare Synopsys IP core versions

Add macros to compare DWC IP core versions:

dw_pcie_ver_is()
dw_pcie_ver_is_ge()
dw_pcie_ver_type_is()
dw_pcie_ver_type_is_ge()

These are along the lines of DWC3_VER_IS() and dw_spi_ver_is().

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20220624143947.8991-6-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>

PCI: dwc: Read DWC IP core version from register

Since DWC PCIe v4.70a, the controller version and version type can be read
from the PORT_LOGIC.PCIE_VERSION_OFF and PORT_LOGIC.PCIE_VERSION_TYPE_OFF
registers respectively.

Read the version from those registers and warn if if's different from the
version we got from the device tree.

We can only read the version after platform-specific drivers have done any
DBI-related initialization, such as reference clock activation.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20220624143947.8991-5-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: Use native DWC IP core version representation

Save the DWC IP core version in the same format as the
PORT_LOGIC.PCIE_VERSION_OFF register, similar to what other drivers for DWC
IP do (dw_spi_hw_init(), dwc3_core_is_valid(), stmmac_hwif_init()).

[bhelgaas: trim commit log]
Link: https://lore.kernel.org/r/20220624143947.8991-4-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: tegra194: Find RAS DES PCIe capability offset

Find RAS DES PCIe capability offset instead of hardcoding the offset
for each controller.

Link: https://lore.kernel.org/r/20220721142052.25971-10-vidyas@nvidia.com
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: qcom: Add IPQ60xx support

IPQ60xx series of SoCs have one port of PCIe gen 3. Add support for that
platform.

The code is based on downstream[1] Codeaurora kernel v5.4 (branch
win.linuxopenwrt.2.0).

Split out the DBI registers access part from .init into .post_init. DBI
registers are only accessible after phy_power_on().

[1] https://source.codeaurora.org/quic/qsdk/oss/kernel/linux-ipq-5.4/

Link: https://lore.kernel.org/r/f7f848653c99abbf9a0f877949a44e52329543ae.1655799816.git.baruch@tkos.co.il
Tested-by: Robert Marko <robert.marko@sartura.hr>
Signed-off-by: Selvam Sathappan Periakaruppan <quic_speriaka@quicinc.com>
Signed-off-by: Baruch Siach <baruch.siach@siklu.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Johan Hovold <johan+linaro@kernel.org>
Acked-by: Stanimir Varbanov <svarbanov@mm-sol.com>

PCI: dwc: Move GEN3_RELATED DBI definitions to common header

These are common dwc macros that will be used for other platforms.

Link: https://lore.kernel.org/r/1c2d5a7a139be81fa15f356b2380163dbdebdc09.1655799816.git.baruch@tkos.co.il
Signed-off-by: Baruch Siach <baruch.siach@siklu.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>

PCI: dwc: Rename struct pcie_port to dw_pcie_rp

All of the DW PCIe core driver entities except the pcie_port struct have
names with the "dw_" prefix to distinguish local and common PCIe name
spaces, and endpoint-related entities have an "_ep" suffix.

Rename struct pcie_port to dw_pcie_rp to make it more consistent with other
names.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20220624143428.8334-16-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Neil Armstrong <narmstrong@baylibre.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Jesper Nilsson <jesper.nilsson@axis.com>

PCI: dwc: Move io_cfg_atu_shared to struct pcie_port

The io_cfg_atu_shared flag is set if there is an outbound iATU window used
for both config space accesses and IO port transfers.

Since the flag semantic is purely Root Port specific, it's not used in
either the DW PCIe common code or in the DW PCIe Endpoint driver. Move it
to the struct pcie_port and rename to cfg0_io_shared.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20220624143428.8334-15-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: Add start_link/stop_link inlines

Factor out this pattern:

if (!pci->ops || !pci->ops->start_link)
return -EINVAL;

return pci->ops->start_link(pci);

into a new dw_pcie_start_link() wrapper and do the same for the stop_link()
method.

Note that dw_pcie_ep_start() previously returned -EINVAL if there was no
platform start_link() method, which didn't make much sense since that is
not an error. It will now return 0 in that case.

As a side-effect, drop the empty start_link() and dummy dw_pcie_ops
instances from the generic DW PCIe and Layerscape EP platform drivers.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20220624143428.8334-14-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI: dwc: Move iATU detection earlier

dw_pcie_ep_init() depends on the detected iATU region numbers to allocate
the in/outbound window management bitmap. It fails after 281f1f99cf3a
("PCI: dwc: Detect number of iATU windows").

Move the iATU region detection into a new function, move the detection to
the very beginning of dw_pcie_host_init() and dw_pcie_ep_init(). Also
remove it from the dw_pcie_setup(), since it's more like a software
initialization step than hardware setup.

Link: https://lore.kernel.org/r/20210125044803.4310-1-Zhiqiang.Hou@nxp.com
Link: https://lore.kernel.org/linux-pci/20210407131255.702054-1-dmitry.baryshkov@linaro.org
Link: https://lore.kernel.org/r/20210413142219.2301430-1-dmitry.baryshkov@linaro.org
Fixes: 281f1f99cf3a ("PCI: dwc: Detect number of iATU windows")
Tested-by: Kunihiko Hayashi <hayashi.kunihiko@socionext.com>
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
[DB: moved dw_pcie_iatu_detect to happen after host_init callback]
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Cc: stable@vger.kernel.org # v5.11+
Cc: Marek Szyprowski <m.szyprowski@samsung.com>

PCI: dwc: Add upper limit address for outbound iATU

The size parameter is unsigned long type which can accept size > 4GB. In
that case, the upper limit address must be programmed. Add support to
program the upper limit address and set INCREASE_REGION_SIZE in case size >
4GB.

Link: https://lore.kernel.org/r/1612250918-19610-1-git-send-email-shradha.t@samsung.com
Signed-off-by: Shradha Todi <shradha.t@samsung.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Pankaj Dubey <pankaj.dubey@samsung.com>
Reviewed-by: Rob Herring <robh@kernel.org>

PCI: dwc: Change size to u64 for EP outbound iATU

Since outbound iATU permits size to be greater than 4GB for which the
support is also available, allow EP function to send u64 size instead of
truncating to u32.

Link: https://lore.kernel.org/r/1609929900-19082-1-git-send-email-shradha.t@samsung.com
Signed-off-by: Shradha Todi <shradha.t@samsung.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Pankaj Dubey <pankaj.dubey@samsung.com>

PCI: dwc: Work around ECRC configuration issue

DesignWare core has a TLP digest (TD) override bit in one of the control
registers of ATU. This bit also needs to be programmed for proper ECRC
functionality. This is currently identified as an issue with DesignWare
IP version 4.90a.

[bhelgaas: fix typos/grammar errors]
Link: https://lore.kernel.org/r/20201230165723.673-1-vidyas@nvidia.com
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: dwc: Detect number of iATU windows

Currently the number of inbound and outbound iATU windows are determined
from DT properties. Unfortunately, there's 'num-viewport' for RC mode
and 'num-ib-windows' and 'num-ob-windows' for EP mode, yet the number of
windows is not mode dependent. Also, 'num-viewport' is not clear whether
that's inbound, outbound or both. We can probably assume it's outbound
windows as that's all RC mode uses.

However, using DT properties isn't really needed as the number of
regions can be detected at runtime by poking the iATU registers. The
basic algorithm is just writing a target address and reading back what
we wrote. In the unrolled ATU case, we have to take care not to go
past the mapped region.

With this, we can drop num_viewport in favor of num_ob_windows instead.

Link: https://lore.kernel.org/r/20201105211159.1814485-17-robh@kernel.org
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Jonathan Hunter <jonathanh@nvidia.com>
Cc: linux-tegra@vger.kernel.org

PCI: dwc: Move inbound and outbound windows to common struct

The number of inbound and outbound windows are defined by the h/w and
apply to both RC and EP modes, so move them to the appropriate struct.

Link: https://lore.kernel.org/r/20201105211159.1814485-16-robh@kernel.org
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>

PCI: dwc: Move dw_pcie_msi_init() into core

The host drivers which call dw_pcie_msi_init() are all the ones using
the built-in MSI controller, so let's move it into the common DWC code.

Link: https://lore.kernel.org/r/20201105211159.1814485-12-robh@kernel.org
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Jingoo Han <jingoohan1@gmail.com>
Cc: Kishon Vijay Abraham I <kishon@ti.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Kukjin Kim <kgene@kernel.org>
Cc: Krzysztof Kozlowski <krzk@kernel.org>
Cc: Richard Zhu <hongxing.zhu@nxp.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Shawn Guo <shawnguo@kernel.org>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
Cc: Fabio Estevam <festevam@gmail.com>
Cc: NXP Linux Team <linux-imx@nxp.com>
Cc: Yue Wang <yue.wang@Amlogic.com>
Cc: Kevin Hilman <khilman@baylibre.com>
Cc: Neil Armstrong <narmstrong@baylibre.com>
Cc: Jerome Brunet <jbrunet@baylibre.com>
Cc: Martin Blumenstingl <martin.blumenstingl@googlemail.com>
Cc: Jesper Nilsson <jesper.nilsson@axis.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Xiaowei Song <songxiaowei@hisilicon.com>
Cc: Binghui Wang <wangbinghui@hisilicon.com>
Cc: Stanimir Varbanov <svarbanov@mm-sol.com>
Cc: Andy Gross <agross@kernel.org>
Cc: Bjorn Andersson <bjorn.andersson@linaro.org>
Cc: Pratyush Anand <pratyush.anand@gmail.com>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Jonathan Hunter <jonathanh@nvidia.com>
Cc: Kunihiko Hayashi <hayashi.kunihiko@socionext.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: linux-omap@vger.kernel.org
Cc: linux-samsung-soc@vger.kernel.org
Cc: linux-amlogic@lists.infradead.org
Cc: linux-arm-kernel@axis.com
Cc: linux-arm-msm@vger.kernel.org
Cc: linux-tegra@vger.kernel.org

PCI: dwc: Rework MSI initialization

There are 3 possible MSI implementations for the DWC host. The first is
using the built-in DWC MSI controller. The 2nd is a custom MSI
controller as part of the PCI host (keystone only). The 3rd is an
external MSI controller (typically GICv3 ITS). Currently, the last 2
are distinguished with a .msi_host_init() hook with the 3rd option using
an empty function. However we can detect the 3rd case with the presence
of 'msi-parent' or 'msi-map' properties, so let's do that instead and
remove the empty functions.

Link: https://lore.kernel.org/r/20201105211159.1814485-10-robh@kernel.org
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Jingoo Han <jingoohan1@gmail.com>
Cc: Murali Karicheri <m-karicheri2@ti.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Minghuan Lian <minghuan.Lian@nxp.com>
Cc: Mingkai Hu <mingkai.hu@nxp.com>
Cc: Roy Zang <roy.zang@nxp.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: linuxppc-dev@lists.ozlabs.org

PCI: dwc: Drop the .set_num_vectors() host op

There's no reason for the .set_num_vectors() host op. Drivers needing a
non-default value can just initialize pcie_port.num_vectors directly.

Link: https://lore.kernel.org/r/20201105211159.1814485-8-robh@kernel.org
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Jonathan Hunter <jonathanh@nvidia.com>
Cc: linux-tegra@vger.kernel.org

PCI: dwc: Add support to program ATU for >4GB memory

Add support to program the ATU to enable translations for >4GB sizes of
the prefetchable memory apertures.

Link: https://lore.kernel.org/r/20201118144626.32189-3-vidyas@nvidia.com
Tested-by: Thierry Reding <treding@nvidia.com>
Tested-by: Jon Hunter <jonathanh@nvidia.com>
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Acked-by: Jingoo <jingoohan1@gmail.com>

PCI: dwc: Support multiple ATU memory regions

The current ATU setup only supports a single memory resource which
isn't sufficient if there are also prefetchable memory regions. In order
to support multiple memory regions, we need to move away from fixed ATU
slots and rework the assignment. As there's always an ATU entry for
config space, let's assign index 0 to config space. Then we assign
memory resources to index 1 and up. Finally, if we have an I/O region
and slots remaining, we assign the I/O region last. If there aren't
remaining slots, we keep the same config and I/O space sharing.

Link: https://lore.kernel.org/r/20201026181652.418729-1-robh@kernel.org
Tested-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Vidya Sagar <vidyas@nvidia.com>
Acked-by: Jingoo Han <jingoohan1@gmail.com>
Cc: Vidya Sagar <vidyas@nvidia.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>

PCI: dwc: Fix MSI page leakage in suspend/resume

Currently, dw_pcie_msi_init() allocates and maps page for msi, then
program the PCIE_MSI_ADDR_LO and PCIE_MSI_ADDR_HI. The Root Complex
may lose power during suspend-to-RAM, so when we resume, we want to
redo the latter but not the former. If designware based driver (for
example, pcie-tegra194.c) calls dw_pcie_msi_init() in resume path, the
msi page will be leaked.

As pointed out by Rob and Ard, there's no need to allocate a page for
the MSI address, we could use an address in the driver data.

To avoid map the MSI msg again during resume, we move the map MSI msg
from dw_pcie_msi_init() to dw_pcie_host_init().

Suggested-by: Rob Herring <robh@kernel.org>
Link: https://lore.kernel.org/r/20201009155505.5a580ef5@xhacker.debian
Signed-off-by: Jisheng Zhang <Jisheng.Zhang@synaptics.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Rob Herring <robh@kernel.org>

PCI: designware-ep: Modify MSI and MSIX CAP way of finding

Each PF of EP device should have its own MSI or MSIX capabitily
struct, so create a dw_pcie_ep_func struct and move the msi_cap
and msix_cap to this struct from dw_pcie_ep, and manage the PFs
via a list.

Link: https://lore.kernel.org/r/20200918080024.13639-5-Zhiqiang.Hou@nxp.com
Signed-off-by: Xiaowei Bao <xiaowei.bao@nxp.com>
Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

PCI: designware-ep: Add the doorbell mode of MSI-X in EP mode

Add the doorbell mode of MSI-X in DWC EP driver.

Link: https://lore.kernel.org/r/20200918080024.13639-3-Zhiqiang.Hou@nxp.com
Signed-off-by: Xiaowei Bao <xiaowei.bao@nxp.com>
Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Andrew Murray <andrew.murray@arm.com>

PCI: designware-ep: Add multiple PFs support for DWC

Add multiple PFs support for DWC, due to different PF have different
config space, we use func_conf_select callback function to access
the different PF's config space, the different chip company need to
implement this callback function when use the DWC IP core and intend
to support multiple PFs feature.

Link: https://lore.kernel.org/r/20200918080024.13639-2-Zhiqiang.Hou@nxp.com
Signed-off-by: Xiaowei Bao <xiaowei.bao@nxp.com>
Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>

PCI: dwc: Move N_FTS setup to common setup

The Designware controller has common registers to set number of fast
training sequence ordered sets. The Artpec6, Intel, and Tegra driver
initialize these register fields. Let's move the initialization to the
common setup code and drivers just have to provide the value.

There's a slight change in that the common clock mode N_FTS field is
now initialized. Previously only the Intel driver set this. It's not
clear from the code if common clock mode is used in the Artpec6 or Tegra
driver. It depends on the DWC configuration. Given the field is not
initialized while the others are, it seems unlikely common clock mode
is used.

Link: https://lore.kernel.org/r/20200821035420.380495-40-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Jesper Nilsson <jesper.nilsson@axis.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Jonathan Hunter <jonathanh@nvidia.com>
Cc: linux-tegra@vger.kernel.org

PCI: dwc: Centralize link gen setting

keystone would force gen2 if no DT property. Now it relies on the
PCI_EXP_LNKCAP value.

Link: https://lore.kernel.org/r/20200821035420.380495-35-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Kishon Vijay Abraham I <kishon@ti.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Richard Zhu <hongxing.zhu@nxp.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Shawn Guo <shawnguo@kernel.org>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
Cc: Fabio Estevam <festevam@gmail.com>
Cc: NXP Linux Team <linux-imx@nxp.com>
Cc: Murali Karicheri <m-karicheri2@ti.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Stanimir Varbanov <svarbanov@mm-sol.com>
Cc: Andy Gross <agross@kernel.org>
Cc: Bjorn Andersson <bjorn.andersson@linaro.org>
Cc: Pratyush Anand <pratyush.anand@gmail.com>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Jonathan Hunter <jonathanh@nvidia.com>
Cc: linux-omap@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-arm-msm@vger.kernel.org
Cc: linux-tegra@vger.kernel.org

PCI: dwc: Make ATU accessors private

The ATU registers are only accessed in pcie-designware.c and can be private
to it.

Link: https://lore.kernel.org/r/20200821035420.380495-34-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>

PCI: dwc: Remove read_dbi2 code

The DBI2 appears to be write-only and there's no read accesses in the code
anyways, so let's remove all the read_dbi2 related code.

Link: https://lore.kernel.org/r/20200821035420.380495-33-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Murali Karicheri <m-karicheri2@ti.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>

PCI: dwc/tegra: Use common Designware port logic register definitions

The Tegra driver has its own defines for common Designware Port Logic
registers. Convert it to use the standard register definitions.

Link: https://lore.kernel.org/r/20200821035420.380495-32-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Jonathan Hunter <jonathanh@nvidia.com>
Cc: linux-tegra@vger.kernel.org

PCI: dwc: Ensure FAST_LINK_MODE is cleared

"Fast Link Mode" is a simulation environment speed up setting which should
never be set and the default is not set. However some Amlogic platforms
have it set (by firmware presumably). See commit 87dccf09323f ("PCI:
amlogic: meson: Don't use FAST_LINK_MODE to set up link") for more
information. Let's clear it in core DWC code so we can drop some vendor
specific code.

Link: https://lore.kernel.org/r/20200821035420.380495-25-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>

PCI: dwc: Add a 'num_lanes' field to struct dw_pcie

Add a 'num_lanes' field to allow drivers to provide a the number of lanes
if not in DT or using a custom DT property. A driver can provide a
non-zero value which is used if the DT doesn't have a 'num-lanes'
property.

Link: https://lore.kernel.org/r/20200821035420.380495-24-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>

PCI: dwc: Simplify config space handling

The config space is divided in half for type 0 and type 1 accesses, but
this is pointless as there's only one iATU window which is
reconfigured on each access.

The only platform doing something custom is TI Keystone (surprise!).
It does its own mapping of the config space to avoid spliting the
config space and never actually uses va_cfg1_base as it has its own
config space accessors. With the splitting removed, Keystone can use the
default mapping of config space.

Link: https://lore.kernel.org/r/20200821035420.380495-20-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Murali Karicheri <m-karicheri2@ti.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>

PCI: dwc: Remove storing of PCI resources

The PCI bridge resources are stored in pci_host_bridge.windows, so
there's no need to store them in a DWC specific struct. There's also no
need to parse the resources and store them a 2nd time as they are mainly
used for one time setup of iATU windows.

Link: https://lore.kernel.org/r/20200821035420.380495-19-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Murali Karicheri <m-karicheri2@ti.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jonathan Chocron <jonnyc@amazon.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>

PCI: dwc: Remove root_bus pointer

The pci_host_bridge struct already has a pointer to its pci_bus, so
let's convert the one user to use the bridge struct and remove the
private 'root_bus' pointer.

Link: https://lore.kernel.org/r/20200821035420.380495-18-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Jonathan Hunter <jonathanh@nvidia.com>
Cc: linux-tegra@vger.kernel.org

PCI: dwc: Convert to use pci_host_probe()

Now that there are no more .scan_bus() callbacks, we can remove it and just
use pci_host_probe().

Link: https://lore.kernel.org/r/20200821035420.380495-17-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>

PCI: dwc: Remove dwc specific config accessor ops

Now that all the drivers needing custom config accessors have been
converted to define their own pci_ops, we can remove the DWC specific
function callbacks {rd,wr}_{own,other}_conf.

Link: https://lore.kernel.org/r/20200821035420.380495-13-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>

PCI: dwc: Add a default pci_ops.map_bus for root port

The Designware root port config space is memory mapped accesses via the
DBI space by default. Add a common implementation
dw_pcie_own_conf_map_bus() for platforms to use.

Link: https://lore.kernel.org/r/20200821035420.380495-5-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>

PCI: dwc: Allow overriding bridge pci_ops

In preparation to allow drivers to set their own root and child pci_ops
instead of using the DWC specific config space ops, we need to make
the pci_host_bridge pointer available and move setting the bridge->ops
and bridge->child_ops pointer to before the .host_init() hook.

Link: https://lore.kernel.org/r/20200821035420.380495-4-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>

PCI: designware: Use pci_is_root_bus() to check if bus is root bus

Use pci_is_root_bus() rather than tracking the root bus number to
determine if the bus is the root bus or not. This removes storing
duplicated data as well as the need for the host bridge driver to have
to care about the bus numbers in most cases.

Link: https://lore.kernel.org/r/20200722022514.1283916-6-robh@kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Richard Zhu <hongxing.zhu@nxp.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Shawn Guo <shawnguo@kernel.org>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
Cc: Fabio Estevam <festevam@gmail.com>
Cc: NXP Linux Team <linux-imx@nxp.com>
Cc: Murali Karicheri <m-karicheri2@ti.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>

PCI: Replace http:// links with https://

Replace http:// links with https:// links. This reduces the likelihood of
man-in-the-middle attacks when developers open these links.

Deterministic algorithm:
For each file:
If not .svg:
For each line:
If doesn't contain `\bxmlns\b`:
For each link, `\bhttp://[^# \t\r\n]*(?:\w|/)`:
If both the HTTP and HTTPS versions
return 200 OK and serve the same content:
Replace HTTP with HTTPS.

[bhelgaas: also update samsung.com links, drop sourceforge link]
Link: https://lore.kernel.org/r/20200627103050.71712-1-grandmaster@al2klimov.de
Signed-off-by: Alexander A. Klimov <grandmaster@al2klimov.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: dwc: Program outbound ATU upper limit register

Function dw_pcie_prog_outbound_atu_unroll() does not program the upper
32-bit ATU limit register. Since ATU programming functions limit the
size of the translated region to 4GB by using a u32 size parameter,
these issues may combine into undefined behavior for resource sizes
with non-zero upper 32-bits.

For example, a 128GB address space starting at physical CPU address of
0x2000000000 with size of 0x2000000000 needs the following values
programmed into the lower and upper 32-bit limit registers:
0x3fffffff in the upper 32-bit limit register
0xffffffff in the lower 32-bit limit register

Currently, only the lower 32-bit limit register is programmed with a
value of 0xffffffff but the upper 32-bit limit register is not being
programmed. As a result, the upper 32-bit limit register remains at its
default value after reset of 0x0.

These issues may combine to produce undefined behavior since the ATU
limit address may be lower than the ATU base address. Programming the
upper ATU limit address register prevents such undefined behavior despite
the region size getting truncated due to the 32-bit size limit.

Link: https://lore.kernel.org/r/1585785493-23210-1-git-send-email-alan.mikhak@sifive.com
Signed-off-by: Alan Mikhak <alan.mikhak@sifive.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>

PCI: dwc: Fix dw_pcie_ep_raise_msix_irq() to get correct MSI-X table address

commit beb4641a787d ("PCI: dwc: Add MSI-X callbacks handler"),
in order to raise MSI-X interrupt, obtained MSIX table address from
Base Address Register (BAR). However BAR only holds PCI address
programmed by the host whereas the MSI-X table should be in the local
memory.

Store the MSI-X table address (virtual address) as part of ->set_bar()
callback and use that to get the message address and message data
here.

Fixes: beb4641a787d ("PCI: dwc: Add MSI-X callbacks handler")
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

PCI: dwc: Add API to notify core initialization completion

Add a new API dw_pcie_ep_init_notify() to let platform drivers
call it when the core is available for initialization.

Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>

PCI: dwc: Refactor core initialization code for EP mode

Split core initialization code for EP mode into two, one that doesn't
touch core registers and the other that touches core registers. The latter
would be called/skipped based on the EPC feature 'core_init_notifier'.

In platforms where this is skipped, it would be called indirectly
through hooks from the endpoint function driver.

Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>

PCI: dwc: intel: PCIe RC controller driver

Add support to PCIe RC controller on Intel Gateway SoCs.
PCIe controller is based of Synopsys DesignWare PCIe core.

Intel PCIe driver requires Upconfigure support, Fast Training
Sequence and link speed configurations. So adding the respective
helper functions in the PCIe DesignWare framework.
It also programs hardware autonomous speed during speed
configuration so defining it in pci_regs.h.

Also, mark Intel PCIe driver depends on MSI IRQ Domain
as Synopsys DesignWare framework depends on the
PCI_MSI_IRQ_DOMAIN.

Signed-off-by: Dilip Kota <eswara.kota@linux.intel.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Andrew Murray <andrew.murray@arm.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@intel.com>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>

PCI: Add PCI_STD_NUM_BARS for the number of standard BARs

Code that iterates over all standard PCI BARs typically uses
PCI_STD_RESOURCE_END. However, that requires the unusual test
"i <= PCI_STD_RESOURCE_END" rather than something the typical
"i < PCI_STD_NUM_BARS".

Add a definition for PCI_STD_NUM_BARS and change loops to use the more
idiomatic C style to help avoid fencepost errors.

Link: https://lore.kernel.org/r/20190927234026.23342-1-efremov@linux.com
Link: https://lore.kernel.org/r/20190927234308.23935-1-efremov@linux.com
Link: https://lore.kernel.org/r/20190916204158.6889-3-efremov@linux.com
Signed-off-by: Denis Efremov <efremov@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Sebastian Ott <sebott@linux.ibm.com> # arch/s390/
Acked-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com> # video/fbdev/
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com> # pci/controller/dwc/
Acked-by: Jack Wang <jinpu.wang@cloud.ionos.com> # scsi/pm8001/
Acked-by: Martin K. Petersen <martin.petersen@oracle.com> # scsi/pm8001/
Acked-by: Ulf Hansson <ulf.hansson@linaro.org> # memstick/

PCI: dwc: Add support to enable CDM register check

Add support to enable CDM (Configuration Dependent Module) register
check for any data corruption based on the DT property
'snps,enable-cdm-check'.

Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Thierry Reding <treding@nvidia.com>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>

PCI: dwc: Add extended configuration space capability search API

Add extended configuration space capability search API using struct
dw_pcie* pointer.

Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Acked-by: Thierry Reding <treding@nvidia.com>

PCI: dwc: Move config space capability search API

Move PCIe config space capability search API to common DesignWare file
as this can be used by both host and EP mode drivers.

Signed-off-by: Vidya Sagar <vidyaos@nvidia.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Thierry Reding <treding@nvidia.com>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>

PCI: dwc: Cleanup DBI,ATU read and write APIs

Cleanup DBI read and write APIs by removing leading "__" (underscore)
from their names as there is no reason to have leading underscores
in the first place in the function definition.

Remove dbi/dbi2 base address parameters as the same behaviour can be
obtained through read and write APIs. Since dw_pcie_{readl/writel}_dbi()
APIs can't be used for ATU read/write as ATU base address could be
different from DBI base address, implement ATU read/write APIs using ATU
base address without using dw_pcie_{readl/writel}_dbi() APIs.

Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Jingoo Han <jingoohan1@gmail.com>

PCI: dwc: Add API support to de-initialize host

Add an API to group all the tasks to be done to de-initialize host which
can then be called by any dwc core based driver implementations
while adding .remove() support in their respective drivers.

Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>

PCI: dwc: Save root bus for driver remove hooks

Currently DWC host does not support the remove callback, but nothing
prevents us from supporting it.

Save the root bus for clean up work in driver remove code paths to allow
DWC host drivers to implement their remove hook as, eg:

static int foo_pcie_remove(struct platform_device *pdev)
{
...
pci_stop_root_bus(pp->root_bus);
pci_remove_root_bus(pp->root_bus);
dw_pcie_free_msi(pp);
...
}

Signed-off-by: Jisheng Zhang <Jisheng.Zhang@synaptics.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>

PCI: dwc: Free MSI IRQ page in dw_pcie_free_msi()

To avoid a memory leak, free the page allocated for MSI IRQ in
dw_pcie_free_msi().

Signed-off-by: Jisheng Zhang <Jisheng.Zhang@synaptics.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>

PCI: dwc: imx6: Share PHY debug register definitions

Both pcie-designware.c and pci-imx6.c contain custom definitions for
PHY debug registers R0/R1 and on top of that there's already a
definition for R0 in pcie-designware.h. Move all of the definitions to
pcie-designware.h. No functional change intended.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Chris Healy <cphealy@gmail.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: linux-kernel@vger.kernel.org
Cc: linux-pci@vger.kernel.org

PCI: dwc: Add callbacks for accessing dbi2 address space

Certain platforms like TI's AM654 do not have aseparate address space for
dbi2 instead they are accessed using the same address space as dbi
with some configuration bit set. In order to support such platforms,
add callbacks for accessing dbi2 address space.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

PCI: dwc: Add const qualifier to struct dw_pcie_ep_ops

Add const qualifier to struct dw_pcie_ep_ops member of
struct dw_pcie_ep.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

PCI: dwc: Fix ATU identification for designware version >= 4.80

Synopsys designware version >= 4.80 uses a separate register space
for programming ATU. The current code identifies if there exists a
separate register space by accessing the register address of ATUs
in designware version < 4.80. Accessing this address results in
abort in the case of K2G.

Fix it here by adding "version" member to struct dw_pcie. This should be
set by platform specific drivers and designware core will use it to
identify if the platform has a separate ATU space. For platforms which
have not populated the version member, the old method of identification
will still be used.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

PCI: dwc: Remove Keystone specific dw_pcie_host_ops

Now that Keystone started using its own msi_irq_chip, remove
Keystone specific callback functions defined in dw_pcie_host_ops.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

PCI: dwc: Add support to use non default msi_irq_chip

Platforms using DesignWare IP use dw_pci_msi_bottom_irq_chip for
configuring the MSI controller logic within the DesignWare IP. However
certain platforms like Keystone (K2G) which uses DesignWare IP have
their own MSI controller logic. For handling such platforms,
the irqchip ops use msi_irq_ack(), msi_set_irq(), msi_clear_irq()
callback functions.

Add support to use different msi_irq_chip with default set to
dw_pci_msi_bottom_irq_chip. This is in preparation to get rid of
msi_irq_ack(), msi_set_irq(), msi_clear_irq() and other Keystone
specific dw_pcie_host_ops.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

PCI: dwc: Remove superfluous shifting in definitions

Surrounding definitions no longer use explicit shift, so "<< 0" here
serve no purpose. Remove them. No functional change intended.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Fabio Estevam <fabio.estevam@nxp.com>
Cc: Chris Healy <cphealy@gmail.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Leonard Crestez <leonard.crestez@nxp.com>
Cc: "A.s. Dong" <aisheng.dong@nxp.com>
Cc: Richard Zhu <hongxing.zhu@nxp.com>
Cc: linux-imx@nxp.com
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-pci@vger.kernel.org

PCI: dwc: Make use of GENMASK/FIELD_PREP

Convert various multi-bit fields to be defined using GENMASK/FIELD_PREP.
This way bit field boundaries are defined in a single place only, as
well as defined in a way that makes it easier to verify them against the
reference manual. No functional change intended.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Fabio Estevam <fabio.estevam@nxp.com>
Cc: Chris Healy <cphealy@gmail.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Leonard Crestez <leonard.crestez@nxp.com>
Cc: "A.s. Dong" <aisheng.dong@nxp.com>
Cc: Richard Zhu <hongxing.zhu@nxp.com>
Cc: linux-imx@nxp.com
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-pci@vger.kernel.org

PCI: dwc: Make use of BIT() in constant definitions

Avoid using explicit left shifts and convert various definitions to
use BIT() instead. No functional change intended.

Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
[lorenzo.pieralisi@arm.com: fixed PORT_LOGIC_SPEED_CHANGE redefinition]
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Fabio Estevam <fabio.estevam@nxp.com>
Cc: Chris Healy <cphealy@gmail.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Leonard Crestez <leonard.crestez@nxp.com>
Cc: "A.s. Dong" <aisheng.dong@nxp.com>
Cc: Richard Zhu <hongxing.zhu@nxp.com>
Cc: linux-imx@nxp.com
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-pci@vger.kernel.org

PCI: dwc: Add ->get_features() callback function to dw_pcie_ep_ops

Each platform using Designware PCIe core can support different set of
endpoint features. Add a new callback function ->get_features() in
dw_pcie_ep_ops so that each platform using Designware PCIe core can
advertise its supported features to the endpoint function driver.

Tested-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

PCI: dwc: Improve code readability and simplify mask/unmask operations

Improve code readability and simplifies mask/unmask operations by
inverting the applied logic (no functional change is intended).

Replace variable name from irq_status to irq_mask, since its goal is to
keep track of which interrupts are masked or not.

Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Joao Pinto <jpinto@synopsys.com>
Cc: Jingoo Han <jingoohan1@gmail.com>

PCI: dwc: Don't hard-code DBI/ATU offset

The DWC PCIe core contains various separate register spaces: DBI, DBI2,
ATU, DMA, etc. The relationship between the addresses of these register
spaces is entirely determined by the implementation of the IP block, not
by the IP block design itself. Hence, the DWC driver must not make
assumptions that one register space can be accessed at a fixed offset from
any other register space. To avoid such assumptions, introduce an
explicit/separate register pointer for the ATU register space. In
particular, the current assumption is not valid for NVIDIA's T194 SoC.

The ATU register space is only used on systems that require unrolled ATU
access. This property is detected at run-time for host controllers, and
when this is detected, this patch provides a default value for atu_base
that matches the previous assumption re: register layout. An alternative
would be to update all drivers for HW that requires unrolled access to
explicitly set atu_base. However, it's hard to tell which drivers would
require atu_base to be set. The unrolled property is not detected for
endpoint systems, and so any endpoint driver that requires unrolled access
must explicitly set the iatu_unroll_enabled flag (none do at present), and
so a check is added to require the driver to also set atu_base while at
it.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Acked-by: Vidya Sagar <vidyas@nvidia.com>

PCI: keystone: Cleanup ks_pcie_link_up()

ks_pcie_link_up() uses registers from the designware core to get the
status of the link. Move the register defines to pcie-designware.h
and cleanup ks_pcie_link_up().

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

PCI: dwc: Fix scheduling while atomic issues

When programming the inbound/outbound ATUs, we call usleep_range() after
each checking PCIE_ATU_ENABLE bit. Unfortunately, the ATU programming
can be executed in atomic context:

inbound ATU programming could be called through
pci_epc_write_header()
=>dw_pcie_ep_write_header()
=>dw_pcie_prog_inbound_atu()

outbound ATU programming could be called through
pci_bus_read_config_dword()
=>dw_pcie_rd_conf()
=>dw_pcie_prog_outbound_atu()

Fix this issue by calling mdelay() instead.

Fixes: f8aed6ec624f ("PCI: dwc: designware: Add EP mode support")
Fixes: d8bbeb39fbf3 ("PCI: designware: Wait for iATU enable")
Signed-off-by: Jisheng Zhang <Jisheng.Zhang@synaptics.com>
[lorenzo.pieralisi@arm.com: commit log update]
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>

PCI: dwc: Add legacy interrupt callback handler

Currently DesignWare IP does not handle legacy interrupts.
Add a legacy interrupt callback handler.

Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
[lorenzo.pieralisi@arm.com: updated commit log]
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>

PCI: dwc: Rework MSI callbacks handler

Remove duplicate defines located on pcie-designware.h file already
available on /include/uapi/linux/pci-regs.h file.

Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>

PCI: dwc: Add MSI-X callbacks handler

Add PCIe config space capability search function.

Add sysfs set/get interface to allow the change of EP MSI-X maximum number.

Add EP MSI-X callback for triggering interruptions.

Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>

PCI: Update xxx_pcie_ep_raise_irq() and pci_epc_raise_irq() signatures

Change {cdns, dra7xx, artpec6, dw, rockchip}_pcie_ep_raise_irq() and
pci_epc_raise_irq() signature, namely the interrupt_num variable type
from u8 to u16 to accommodate 2048 maximum MSI-X interrupts.

Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Alan Douglas <adouglas@cadence.com>
Acked-by: Shawn Lin <shawn.lin@rock-chips.com>
Acked-by: Jesper Nilsson <jesper.nilsson@axis.com>
Acked-by: Joao Pinto <jpinto@synopsys.com>
Acked-by: Kishon Vijay Abraham I <kishon@ti.com>

PCI: Collect all native drivers under drivers/pci/controller/

Native PCI drivers for root complex devices were originally all in
drivers/pci/host/. Some of these devices can also be operated in endpoint
mode. Drivers for endpoint mode didn't seem to fit in the "host"
directory, so we put both the root complex and endpoint drivers in
per-device directories, e.g., drivers/pci/dwc/, drivers/pci/cadence/, etc.

These per-device directories contain trivial Kconfig and Makefiles and
clutter drivers/pci/. Make a new drivers/pci/controllers/ directory and
collect all the device-specific drivers there.

No functional change intended.

Link: https://lkml.kernel.org/r/1520304202-232891-1-git-send-email-shawn.lin@rock-chips.com
Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com>
[bhelgaas: changelog]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>