History log of /linux-master/drivers/pci/ats.c
Revision Date Author Comments
# 8c09e896 12-Jul-2021 Zhangfei Gao <zhangfei.gao@linaro.org>

PCI: Allow PASID on fake PCIe devices without TLP prefixes

Some systems, e.g., HiSilicon KunPeng920 and KunPeng930, have devices that
appear as PCI but are actually on the AMBA bus. Some of these fake PCI
devices support a PASID-like feature and they do have a working PASID
capability even though they do not use the PCIe Transport Layer Protocol
and do not support TLP prefixes.

Add a pasid_no_tlp bit for this "PASID works without TLP prefixes" case and
update pci_enable_pasid() so it can enable PASID on these devices.

Set this bit for HiSilicon KunPeng920 and KunPeng930.

[bhelgaas: squashed, commit log]
Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/1626144876-11352-2-git-send-email-zhangfei.gao@linaro.org
Link: https://lore.kernel.org/r/1626144876-11352-3-git-send-email-zhangfei.gao@linaro.org
Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>


# 43395d9e 10-Mar-2021 Krzysztof Wilczyński <kw@linux.com>

PCI: Fix kernel-doc errors

Fix kernel-doc formatting errors, function names that don't match the doc,
and some missing parameter documentation. These are reported by:

make W=1 drivers/pci/

No functional change intended.

[bhelgaas: squashed into one patch since this only changes comments]
Link: https://lore.kernel.org/r/20210311001724.423356-1-kw@linux.com
Link: https://lore.kernel.org/r/20210311001724.423356-2-kw@linux.com
Link: https://lore.kernel.org/r/20210311001724.423356-3-kw@linux.com
Link: https://lore.kernel.org/r/20210311001724.423356-4-kw@linux.com
Link: https://lore.kernel.org/r/20210311001724.423356-5-kw@linux.com
Link: https://lore.kernel.org/r/20210311001724.423356-6-kw@linux.com
Link: https://lore.kernel.org/r/20210311001724.423356-7-kw@linux.com
Link: https://lore.kernel.org/r/20210311001724.423356-8-kw@linux.com
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>


# 9b41d19a 29-Jul-2020 Krzysztof Kozlowski <krzk@kernel.org>

PCI: Fix kerneldoc warnings

Fix kerneldoc warnings, e.g.,

$ make W=1 drivers/pci/
drivers/pci/ats.c:196: warning: Function parameter or member 'pdev' not described in 'pci_enable_pri'
drivers/pci/ats.c:196: warning: Function parameter or member 'reqs' not described in 'pci_enable_pri'
...

Link: https://lore.kernel.org/r/20200729201224.26799-2-krzk@kernel.org
Link: https://lore.kernel.org/r/20200729201224.26799-3-krzk@kernel.org
Link: https://lore.kernel.org/r/20200729201224.26799-4-krzk@kernel.org
Link: https://lore.kernel.org/r/20200729201224.26799-5-krzk@kernel.org
Link: https://lore.kernel.org/r/20200729201224.26799-6-krzk@kernel.org
Link: https://lore.kernel.org/r/20200729201224.26799-7-krzk@kernel.org
Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>


# 3f9a7a13 23-Jul-2020 Ashok Raj <ashok.raj@intel.com>

PCI/ATS: Add pci_pri_supported() to check device or associated PF

For SR-IOV, the PF PRI is shared between the PF and any associated VFs, and
the PRI Capability is allowed for PFs but not for VFs. Searching for the
PRI Capability on a VF always fails, even if its associated PF supports
PRI.

Add pci_pri_supported() to check whether device or its associated PF
supports PRI.

[bhelgaas: commit log, avoid "!!"]
Fixes: b16d0cb9e2fc ("iommu/vt-d: Always enable PASID/PRI PCI capabilities before ATS")
Link: https://lore.kernel.org/r/1595543849-19692-1-git-send-email-ashok.raj@intel.com
Signed-off-by: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
Acked-by: Joerg Roedel <jroedel@suse.de>
Cc: stable@vger.kernel.org # v4.4+


# 52137674 20-May-2020 Jean-Philippe Brucker <jean-philippe@linaro.org>

PCI/ATS: Only enable ATS for trusted devices

Add pci_ats_supported(), which checks whether a device has an ATS
capability, and whether it is trusted. A device is untrusted if it is
plugged into an external-facing port such as Thunderbolt and could be
spoofing an existing device to exploit weaknesses in the IOMMU
configuration. PCIe ATS is one such weaknesses since it allows
endpoints to cache IOMMU translations and emit transactions with
'Translated' Address Type (10b) that partially bypass the IOMMU
translation.

The SMMUv3 and VT-d IOMMU drivers already disallow ATS and transactions
with 'Translated' Address Type for untrusted devices. Add the check to
pci_enable_ats() to let other drivers (AMD IOMMU for now) benefit from
it.

By checking ats_cap, the pci_ats_supported() helper also returns whether
ATS was globally disabled with pci=noats, and could later include more
things, for example whether the whole PCIe hierarchy down to the
endpoint supports ATS.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Reviewed-by: Joerg Roedel <jroedel@suse.de>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20200520152201.3309416-2-jean-philippe@linaro.org
Signed-off-by: Joerg Roedel <jroedel@suse.de>


# 7682ce2b 24-Feb-2020 Jean-Philippe Brucker <jean-philippe@linaro.org>

PCI/ATS: Export symbols of PASID functions

The Arm SMMUv3 driver uses pci_{enable,disable}_pasid() and related
functions. Export them to allow the driver to be built as a module.

Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
Signed-off-by: Will Deacon <will@kernel.org>


# 2e34673b 29-Jan-2020 Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

PCI/ATS: Use PF PASID for VFs

Per PCIe r5.0, sec 9.3.7.14, if a PF implements the PASID Capability, the
PF PASID configuration is shared by its VFs, and VFs must not implement
their own PASID Capability. But commit 751035b8dc06 ("PCI/ATS: Cache PASID
Capability offset") changed pci_max_pasids() and pci_pasid_features() to
use the PASID Capability of the VF device instead of the associated PF
device. This leads to IOMMU bind failures when pci_max_pasids() and
pci_pasid_features() are called for VFs.

In pci_max_pasids() and pci_pasid_features(), always use the PF PASID
Capability.

Fixes: 751035b8dc06 ("PCI/ATS: Cache PASID Capability offset")
Link: https://lore.kernel.org/r/fe891f9755cb18349389609e7fed9940fc5b081a.1580325170.git.sathyanarayanan.kuppuswamy@linux.intel.com
Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
CC: stable@vger.kernel.org # v5.5+


# bb950bca 18-Dec-2019 Greg Kroah-Hartman <gregkh@google.com>

PCI/ATS: Restore EXPORT_SYMBOL_GPL() for pci_{enable,disable}_ats()

Commit d355bb209783 ("PCI/ATS: Remove unnecessary EXPORT_SYMBOL_GPL()")
unexported a bunch of symbols from the PCI core since the only external
users were non-modular IOMMU drivers. Although most of those symbols
can remain private for now, 'pci_{enable,disable_ats()' is required for
the ARM SMMUv3 driver to build as a module, otherwise we get a build
failure as follows:

| ERROR: "pci_enable_ats" [drivers/iommu/arm-smmu-v3.ko] undefined!
| ERROR: "pci_disable_ats" [drivers/iommu/arm-smmu-v3.ko] undefined!

Re-export these two functions so that the ARM SMMUv3 driver can be build
as a module.

Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
[will: rewrote commit message]
Signed-off-by: Will Deacon <will@kernel.org>
Tested-by: John Garry <john.garry@huawei.com> # smmu v3
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>


# d355bb20 09-Oct-2019 Bjorn Helgaas <bhelgaas@google.com>

PCI/ATS: Remove unnecessary EXPORT_SYMBOL_GPL()

The following functions are only used by the PCI core or by IOMMU drivers
that cannot be modular, so there's no need to export them at all:

pci_enable_ats()
pci_disable_ats()
pci_restore_ats_state()
pci_ats_queue_depth()
pci_ats_page_aligned()

pci_enable_pri()
pci_restore_pri_state()
pci_reset_pri()
pci_prg_resp_pasid_required()

pci_enable_pasid()
pci_disable_pasid()
pci_restore_pasid_state()
pci_pasid_features()
pci_max_pasids()

Remove the unnecessary EXPORT_SYMBOL_GPL()s.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>


# e5adf79a 09-Oct-2019 Bjorn Helgaas <bhelgaas@google.com>

PCI/ATS: Cache PRI PRG Response PASID Required bit

The PRG Response PASID Required bit in the PRI Capability is read-only.
Read it once when we enumerate the device and cache the value so we don't
need to read it again.

Based-on-patch-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>


# 751035b8 05-Sep-2019 Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

PCI/ATS: Cache PASID Capability offset

Previously each PASID interface searched for the PASID Capability. Cache
the capability offset the first time we use it instead of searching each
time.

[bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from
pci_init_capabilities()]
Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com
Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org
Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>


# c065190b 05-Sep-2019 Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

PCI/ATS: Cache PRI Capability offset

Previously each PRI interface searched for the PRI Capability. Cache the
capability offset the first time we use it instead of searching each time.

[bhelgaas: commit log, reorder patch to later, call pci_pri_init() from
pci_init_capabilities()]
Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156
7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com
Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org
Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>


# 3ad62192 05-Sep-2019 Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

PCI/ATS: Disable PF/VF ATS service independently

Previously we didn't disable the PF ATS until all associated VFs had
disabled it. But per PCIe spec r5.0, sec 9.3.7.8, the ATS Capability in
VFs and associated PFs may be enabled independently. Leaving ATS enabled
in the PF unnecessarily may have power and performance impacts.

Remove this dependency logic in the ATS enable/disable code.

[bhelgaas: commit log]
Suggested-by: Ashok Raj <ashok.raj@intel.com>
Link: https://lore.kernel.org/r/8163ab8fa66afd2cba514ae95d29ab12104781aa.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com
Link: https://lore.kernel.org/r/20190905193146.90250-4-helgaas@kernel.org
Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Keith Busch <keith.busch@intel.com>


# 2b0ae7cc 05-Sep-2019 Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

PCI/ATS: Handle sharing of PF PASID Capability with all VFs

Per PCIe r5.0, sec 9.3.7.14, if a PF implements the PASID Capability, the
PF PASID configuration is shared by its VFs. VFs must not implement their
own PASID Capability. Since VFs don't have a PASID Capability,
pci_enable_pasid() always failed, which caused IOMMU setup to fail.

Update the PASID interfaces so for VFs they reflect the state of the PF
PASID.

[bhelgaas: rebase without pasid_cap caching, commit log]
Suggested-by: Ashok Raj <ashok.raj@intel.com>
Link: https://lore.kernel.org/r/8ba1ac192e4ac737508b6ac15002158e176bab91.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com
Link: https://lore.kernel.org/r/20190905193146.90250-3-helgaas@kernel.org
Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Keith Busch <keith.busch@intel.com>


# 9bf49e36 05-Sep-2019 Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

PCI/ATS: Handle sharing of PF PRI Capability with all VFs

Per PCIe r5.0, sec 9.3.7.11, VFs must not implement the PRI Capability. If
the PF implements PRI, it is shared by the VFs. Since VFs don't have a PRI
Capability, pci_enable_pri() always failed, which caused IOMMU setup to
fail.

Update the PRI interfaces so for VFs they reflect the state of the PF PRI.

[bhelgaas: rebase without pri_cap caching, commit log]
Suggested-by: Ashok Raj <ashok.raj@intel.com>
Link: https://lore.kernel.org/r/b971e31f8695980da8e4a7f93e3b6a3edba3edaa.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com
Link: https://lore.kernel.org/r/20190905193146.90250-2-helgaas@kernel.org
Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Keith Busch <keith.busch@intel.com>


# 8cbb8a93 09-Oct-2019 Bjorn Helgaas <bhelgaas@google.com>

PCI/ATS: Move pci_prg_resp_pasid_required() to CONFIG_PCI_PRI

pci_prg_resp_pasid_required() returns the value of the "PRG Response PASID
Required" bit from the PRI capability, but the interface was previously
defined under #ifdef CONFIG_PCI_PASID.

Move it from CONFIG_PCI_PASID to CONFIG_PCI_PRI so it's with the other
PRI-related things.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>


# f6b6aefe 30-May-2019 Bjorn Helgaas <bhelgaas@google.com>

PCI: Fix typos and whitespace errors

Fix typos in drivers/pci. Comment and whitespace changes only.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Randy Dunlap <rdunlap@infradead.org>


# 8c938ddc 19-Feb-2019 Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

PCI/ATS: Add pci_ats_page_aligned() interface

Return the Page Aligned Request bit in the ATS Capability Register.

As per PCIe spec r4.0, sec 10.5.1.2, if the Page Aligned Request bit is
set, it indicates the Untranslated Addresses generated by the device are
always aligned to a 4096 byte boundary.

An IOMMU that can only translate page-aligned addresses can only be used
with devices that always produce aligned Untranslated Addresses. This
interface will be used by drivers for such IOMMUs to determine whether
devices can use the ATS service.

Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc: Keith Busch <keith.busch@intel.com>
Suggested-by: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>


# e5567f5f 19-Feb-2019 Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

PCI/ATS: Add pci_prg_resp_pasid_required() interface.

Return the PRG Response PASID Required bit in the Page Request
Status Register.

As per PCIe spec r4.0, sec 10.5.2.3, if this bit is Set, the device
expects a PASID TLP Prefix on PRG Response Messages when the
corresponding Page Requests had a PASID TLP Prefix. If Clear, the device
does not expect PASID TLP Prefixes on any PRG Response Message, and the
device behavior is undefined if the device receives a PRG Response Message
with a PASID TLP Prefix. Also the device behavior is undefined if this
bit is Set and the device receives a PRG Response Message with no PASID TLP
Prefix when the corresponding Page Requests had a PASID TLP Prefix.

This function will be used by drivers like IOMMU, if it is required to
check the status of the PRG Response PASID Required bit before enabling
the PASID support of the device.

Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc: Keith Busch <keith.busch@intel.com>
Suggested-by: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>


# 7ce3f912 30-Jun-2018 Sinan Kaya <okaya@codeaurora.org>

PCI: Enable PASID only if entire path supports End-End TLP prefixes

A PCIe endpoint carries the process address space identifier (PASID) in
the TLP prefix as part of the memory read/write transaction. The address
information in the TLP is relevant only for a given PASID context.

An IOMMU takes PASID value and the address information from the
TLP to look up the physical address in the system.

PASID is an End-End TLP Prefix (PCIe r4.0, sec 6.20). Sec 2.2.10.2 says

It is an error to receive a TLP with an End-End TLP Prefix by a
Receiver that does not support End-End TLP Prefixes. A TLP in
violation of this rule is handled as a Malformed TLP. This is a
reported error associated with the Receiving Port (see Section 6.2).

Prevent error condition by proactively requiring End-End TLP prefix to be
supported on the entire data path between the endpoint and the root port
before enabling PASID.

Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>


# cef74409 10-May-2018 Gil Kupfer <gilkup@gmail.com>

PCI: Add "pci=noats" boot parameter

Adds a "pci=noats" boot parameter. When supplied, all ATS related
functions fail immediately and the IOMMU is configured to not use
device-IOTLB.

Any function that checks for ATS capabilities directly against the devices
should also check this flag. Currently, such functions exist only in IOMMU
drivers, and they are covered by this patch.

The motivation behind this patch is the existence of malicious devices.
Lots of research has been done about how to use the IOMMU as protection
from such devices. When ATS is supported, any I/O device can access any
physical address by faking device-IOTLB entries. Adding the ability to
ignore these entries lets sysadmins enhance system security.

Signed-off-by: Gil Kupfer <gilkup@cs.technion.ac.il>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Joerg Roedel <jroedel@suse.de>


# df62ab5e 09-Mar-2018 Bjorn Helgaas <bhelgaas@google.com>

PCI: Tidy comments

Remove pointless comments that tell us the file name, remove blank line
comments, follow multi-line comment conventions. No functional change
intended.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>


# 7328c8f4 26-Jan-2018 Bjorn Helgaas <bhelgaas@google.com>

PCI: Add SPDX GPL-2.0 when no license was specified

b24413180f56 ("License cleanup: add SPDX GPL-2.0 license identifier to
files with no license") added SPDX GPL-2.0 to several PCI files that
previously contained no license information.

Add SPDX GPL-2.0 to all other PCI files that did not contain any license
information and hence were under the default GPL version 2 license of the
kernel.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>


# 4ebeb1ec 30-May-2017 CQ Tang <cq.tang@intel.com>

PCI: Restore PRI and PASID state after Function-Level Reset

After a Function-Level Reset, PCI states need to be restored. Save PASID
features and PRI reqs cached.

[bhelgaas: search for capability only if PRI/PASID were enabled]
Signed-off-by: CQ Tang <cq.tang@intel.com>
Signed-off-by: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com>
Cc: David Woodhouse <dwmw2@infradead.org>


# a4f4fa68 30-May-2017 Jean-Philippe Brucker <jean-philippe@linaro.org>

PCI: Cache PRI and PASID bits in pci_dev

Device drivers need to check if an IOMMU enabled ATS, PRI and PASID in
order to know when they can use the SVM API. Cache PRI and PASID bits in
the pci_dev structure, similarly to what is currently done for ATS.

Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>


# f7ef1340 20-Jul-2015 Bjorn Helgaas <bhelgaas@google.com>

PCI: Remove pci_ats_enabled()

Remove pci_ats_enabled(). There are no callers outside the ATS code
itself. We don't need to check ats_cap, because if we don't find an ATS
capability, we'll never set ats_enabled.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>


# a71f938f 20-Jul-2015 Bjorn Helgaas <bhelgaas@google.com>

PCI: Stop caching ATS Invalidate Queue Depth

Stop caching the Invalidate Queue Depth in struct pci_dev.
pci_ats_queue_depth() is typically called only once per device, and it
returns a fixed value per-device, so callers who need the value frequently
can cache it themselves.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>


# a021f301 17-Jul-2015 Bjorn Helgaas <bhelgaas@google.com>

PCI: Clean up ATS error handling

There's no need to BUG() if we enable ATS when it's already enabled. We
don't need to BUG() when disabling ATS on a device that doesn't support ATS
or if it's already disabled. If ATS is enabled, certainly we found an ATS
capability in the past, so it should still be there now.

Clean up these error paths.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>


# c39127db 17-Jul-2015 Bjorn Helgaas <bhelgaas@google.com>

PCI: Use pci_physfn() rather than looking up physfn by hand

Use the pci_physfn() helper rather than looking up physfn by hand.
No functional change.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>


# afdd596c 17-Jul-2015 Bjorn Helgaas <bhelgaas@google.com>

PCI: Inline the ATS setup code into pci_ats_init()

The ATS setup code in ats_alloc_one() is only used by pci_ats_init(), so
inline it there. No functional change.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>


# 3c765399 17-Jul-2015 Bjorn Helgaas <bhelgaas@google.com>

PCI: Rationalize pci_ats_queue_depth() error checking

We previously returned -ENODEV for devices that don't support ATS (except
that we always returned 0 for VFs, whether or not they support ATS).

For consistency, always return -EINVAL (not -ENODEV) if the device doesn't
support ATS. Return zero for VFs that support ATS.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>


# d544d75a 17-Jul-2015 Bjorn Helgaas <bhelgaas@google.com>

PCI: Embed ATS info directly into struct pci_dev

The pci_ats struct is small and will get smaller, so I don't think it's
worth allocating it separately from the pci_dev struct.

Embed the ATS fields directly into struct pci_dev.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>


# edc90fee 17-Jul-2015 Bjorn Helgaas <bhelgaas@google.com>

PCI: Allocate ATS struct during enumeration

Previously, we allocated pci_ats structures when an IOMMU driver called
pci_enable_ats(). An SR-IOV VF shares the STU setting with its PF, so when
enabling ATS on the VF, we allocated a pci_ats struct for the PF if it
didn't already have one. We held the sriov->lock to serialize threads
concurrently enabling ATS on several VFS so only one would allocate the PF
pci_ats.

Gregor reported a deadlock here:

pci_enable_sriov
sriov_enable
virtfn_add
mutex_lock(dev->sriov->lock) # acquire sriov->lock
pci_device_add
device_add
BUS_NOTIFY_ADD_DEVICE notifier chain
iommu_bus_notifier
amd_iommu_add_device # iommu_ops.add_device
init_iommu_group
iommu_group_get_for_dev
iommu_group_add_device
__iommu_attach_device
amd_iommu_attach_device # iommu_ops.attach_device
attach_device
pci_enable_ats
mutex_lock(dev->sriov->lock) # deadlock

There's no reason to delay allocating the pci_ats struct, and if we
allocate it for each device at enumeration-time, there's no need for
locking in pci_enable_ats().

Allocate pci_ats struct during enumeration, when we initialize other
capabilities.

Note that this implementation requires ATS to be enabled on the PF first,
before on any of the VFs because the PF controls the STU for all the VFs.

Link: http://permalink.gmane.org/gmane.linux.kernel.iommu/9433
Reported-by: Gregor Dick <gdick@solarflare.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Joerg Roedel <jroedel@suse.de>


# b340cacc 10-Jan-2014 Stephen Hemminger <stephen@networkplumber.org>

PCI: Removed unused parts of Page Request Interface support

My philosophy is unused code is dead code. And dead code is subject to bit
rot and is a likely source of bugs. Use it or lose it.

This reverts parts of c320b976d783 ("PCI: Add implementation for PRI
capability"), removing these interfaces:

pci_pri_enabled()
pci_pri_stopped()
pci_pri_status()

[bhelgaas: split to separate patch]
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
CC: Joerg Roedel <joro@8bytes.org>


# f7625980 14-Nov-2013 Bjorn Helgaas <bhelgaas@google.com>

PCI: Fix whitespace, capitalization, and spelling errors

Fix whitespace, capitalization, and spelling errors. No functional change.
I know "busses" is not an error, but "buses" was more common, so I used it
consistently.

Signed-off-by: Marta Rybczynska <rybczynska@gmail.com> (pci_reset_bridge_secondary_bus())
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>


# 1900ca13 17-Dec-2011 Hao, Xudong <xudong.hao@intel.com>

PCI: Enable ATS at the device state restore

During S3 or S4 resume or PCI reset, ATS regs aren't restored correctly.
This patch enables ATS at the device state restore if PCI device has ATS
capability.

Signed-off-by: Xudong Hao <xudong.hao@intel.com>
Signed-off-by: Xiantao Zhang <xiantao.zhang@intel.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>


# cfa4d8cc 02-Nov-2011 Alex Williamson <alex.williamson@redhat.com>

PCI: Fix PRI and PASID consistency

These are extended capabilities, rename and move to proper
group for consistency.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>


# 8c451945 29-Nov-2011 James Bottomley <jbottomley@parallels.com>

PCI: fix ats compile failure

I get this compile failure on parisc:

drivers/pci/ats.c: In function 'ats_alloc_one':
drivers/pci/ats.c:29: error: implicit declaration of function 'kzalloc'
drivers/pci/ats.c:29: warning: assignment makes pointer from integer without a cast
drivers/pci/ats.c: In function 'ats_free_one':
drivers/pci/ats.c:45: error: implicit declaration of function 'kfree'

Because ats.c is missing linux/slab.h as an include. This patch fixes it

Signed-off-by: James Bottomley <JBottomley@Parallels.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>


# 91f57d5e 11-Nov-2011 Alex Williamson <alex.williamson@redhat.com>

PCI: More PRI/PASID cleanup

More consistency cleanups. Drop the _OFF, separate and indent
CTRL/CAP/STATUS bit definitions. This helped find the previous
mis-use of bit 0 in the PASID capability register.

Reviewed-by: Joerg Roedel <joerg.roedel@amd.com>
Tested-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>


# 60fe8238 11-Nov-2011 Alex Williamson <alex.williamson@redhat.com>

PCI: Enable is not exposed as a PASID capability

The PASID ECN indicates bit 0 is reserved in the capability register.
Switch pci_enable_pasid() to error if PASID is already enabled and
don't expose enable as a feature in pci_pasid_features().

Reviewed-by: Joerg Roedel <joerg.roedel@amd.com>
Tested-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>


# 69166fbf 02-Nov-2011 Alex Williamson <alex.williamson@redhat.com>

PCI: Fix PRI and PASID consistency

These are extended capabilities, rename and move to proper
group for consistency.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>


# 363c75db 27-May-2011 Paul Gortmaker <paul.gortmaker@windriver.com>

pci: Fix files needing export.h for EXPORT_SYMBOL/THIS_MODULE

They were implicitly getting it from device.h --> module.h but
we want to clean that up. So add the minimal header for these
macros.

Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>


# 086ac11f 27-Sep-2011 Joerg Roedel <joerg.roedel@amd.com>

PCI: Add support for PASID capability

Devices supporting Process Address Space Identifiers
(PASIDs) can use an IOMMU to access multiple IO address
spaces at the same time. A PCIe device indicates support for
this feature by implementing the PASID capability. This
patch adds support for the capability to the Linux kernel.

Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>


# c320b976 27-Sep-2011 Joerg Roedel <joerg.roedel@amd.com>

PCI: Add implementation for PRI capability

Implement the necessary functions to handle PRI capabilities
on PCIe devices. With PRI devices behind an IOMMU can signal
page fault conditions to software and recover from such
faults.

Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>


# d4c0636c 27-Sep-2011 Joerg Roedel <joerg.roedel@amd.com>

PCI: Export ATS functions to modules

This patch makes the ATS functions usable for modules.
They will be used by a module implementing some advanced
AMD IOMMU features.

Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>


# db3c33c6 27-Sep-2011 Joerg Roedel <joerg.roedel@amd.com>

PCI: Move ATS implementation into own file

ATS does not depend on IOV support, so move the code into
its own file. This file will also include support for the
PRI and PASID capabilities later.
Also give ATS its own Kconfig variable to allow selecting it
without IOV support.

Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>