PCI: Use resource names in PCI log messages

Use the pci_resource_name() to get the name of the resource and use it
while printing log messages.

[bhelgaas: rename to match struct resource * names, also use names in other
BAR messages]
Link: https://lore.kernel.org/r/20211106112606.192563-3-puranjay12@gmail.com
Signed-off-by: Puranjay Mohan <puranjay12@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/IOV: Use pci_dev_id() to simplify the code

When we have a struct pci_dev *, use pci_dev_id() instead of manually
composing the ID with PCI_DEVID() from dev->bus->number and dev->devfn.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20230807134858.116051-4-wangxiongfeng2@huawei.com
Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/IOV: Enlarge virtfn sysfs name buffer

The sysfs link name "virtfn%u" constructed by pci_iov_sysfs_link() requires
17 bytes to contain the longest possible string. Increase VIRTFN_ID_LEN to
accommodate that.

Found by Linux Verification Center (linuxtesting.org) with SVACE.

[bhelgaas: commit log, comment at #define]
Fixes: dd7cc44d0bce ("PCI: add SR-IOV API for Physical Function driver")
Link: https://lore.kernel.org/r/20221218033347.23743-1-gremlin@altlinux.org
Signed-off-by: Alexey V. Vissarionov <gremlin@altlinux.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/IOV: Fix wrong kernel-doc identifier

Replace "-" to be ":" in comment section to be aligned with
kernel-doc format.

drivers/pci/iov.c:67: warning: Function parameter or member 'dev' not described in 'pci_iov_get_pf_drvdata'
drivers/pci/iov.c:67: warning: Function parameter or member 'pf_driver' not described in 'pci_iov_get_pf_drvdata'

Fixes: a7e9f240c0da ("PCI/IOV: Add pci_iov_get_pf_drvdata() to allow VF reaching the drvdata of a PF")
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/8cecf7df45948a256dc56148cf9e87b2f2bb4198.1646652504.git.leonro@nvidia.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

PCI/IOV: Add pci_iov_get_pf_drvdata() to allow VF reaching the drvdata of a PF

There are some cases where a SR-IOV VF driver will need to reach into and
interact with the PF driver. This requires accessing the drvdata of the PF.

Provide a function pci_iov_get_pf_drvdata() to return this PF drvdata in a
safe way. Normally accessing a drvdata of a foreign struct device would be
done using the device_lock() to protect against device driver
probe()/remove() races.

However, due to the design of pci_enable_sriov() this will result in a
ABBA deadlock on the device_lock as the PF's device_lock is held during PF
sriov_configure() while calling pci_enable_sriov() which in turn holds the
VF's device_lock while calling VF probe(), and similarly for remove.

This means the VF driver can never obtain the PF's device_lock.

Instead use the implicit locking created by pci_enable/disable_sriov(). A
VF driver can access its PF drvdata only while its own driver is attached,
and the PF driver can control access to its own drvdata based on when it
calls pci_enable/disable_sriov().

To use this API the PF driver will setup the PF drvdata in the probe()
function. pci_enable_sriov() is only called from sriov_configure() which
cannot happen until probe() completes, ensuring no VF races with drvdata
setup.

For removal, the PF driver must call pci_disable_sriov() in its remove
function before destroying any of the drvdata. This ensures that all VF
drivers are unbound before returning, fencing concurrent access to the
drvdata.

The introduction of a new function to do this access makes clear the
special locking scheme and the documents the requirements on the PF/VF
drivers using this.

Link: https://lore.kernel.org/all/20220224142024.147653-5-yishaih@nvidia.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>

PCI/IOV: Add pci_iov_vf_id() to get VF index

The PCI core uses the VF index internally, often called the vf_id,
during the setup of the VF, eg pci_iov_add_virtfn().

This index is needed for device drivers that implement live migration
for their internal operations that configure/control their VFs.

Specifically, mlx5_vfio_pci driver that is introduced in coming patches
from this series needs it and not the bus/device/function which is
exposed today.

Add pci_iov_vf_id() which computes the vf_id by reversing the math that
was used to create the bus/device/function.

Link: https://lore.kernel.org/all/20220224142024.147653-2-yishaih@nvidia.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>

Revert "PCI: Use to_pci_driver() instead of pci_dev->driver"

This reverts commit 2a4d9408c9e8b6f6fc150c66f3fef755c9e20d4a.

Robert reported a NULL pointer dereference caused by the PCI core
(local_pci_probe()) calling the i2c_designware_pci driver's
.runtime_resume() method before the .probe() method. i2c_dw_pci_resume()
depends on initialization done by i2c_dw_pci_probe().

Prior to 2a4d9408c9e8 ("PCI: Use to_pci_driver() instead of
pci_dev->driver"), pci_pm_runtime_resume() avoided calling the
.runtime_resume() method because pci_dev->driver had not been set yet.

2a4d9408c9e8 and b5f9c644eb1b ("PCI: Remove struct pci_dev->driver"),
removed pci_dev->driver, replacing it by device->driver, which *has* been
set by this time, so pci_pm_runtime_resume() called the .runtime_resume()
method when it previously had not.

Fixes: 2a4d9408c9e8 ("PCI: Use to_pci_driver() instead of pci_dev->driver")
Link: https://lore.kernel.org/linux-i2c/CAP145pgdrdiMAT7=-iB1DMgA7t_bMqTcJL4N0=6u8kNY3EU0dw@mail.gmail.com/
Reported-by: Robert Święcki <robert@swiecki.net>
Tested-by: Robert Święcki <robert@swiecki.net>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Use to_pci_driver() instead of pci_dev->driver

Struct pci_driver contains a struct device_driver, so for PCI devices, it's
easy to convert a device_driver * to a pci_driver * with to_pci_driver().
The device_driver * is in struct device, so we don't need to also keep
track of the pci_driver * in struct pci_dev.

Replace pci_dev->driver with to_pci_driver(). This is a step toward
removing pci_dev->driver.

[bhelgaas: split to separate patch]
Link: https://lore.kernel.org/r/20211004125935.2300113-11-u.kleine-koenig@pengutronix.de
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/sysfs: Return -EINVAL consistently from "store" functions

Most of the "store" functions that handle userspace input via sysfs return
-EINVAL should the value fail validation and/or type conversion. This
error code is a clear message to userspace that the value is not a valid
input.

However, some of the "show" functions return input parsing error codes
as-is, which may be either -EINVAL or -ERANGE. The former would often be
from kstrtobool(), and the latter typically from other kstr*() functions
such as kstrtou8(), kstrtou32(), kstrtoint(), etc.

-EINVAL is commonly returned as the error code to indicate that the value
provided is invalid, but -ERANGE is not very useful in userspace.

Therefore, normalize the return error code to be -EINVAL for when the
validation and/or type conversion fails.

Link: https://lore.kernel.org/r/20210915230127.2495723-2-kw@linux.com
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/IOV: Clarify error message for unbound devices

Be more verbose to disambiguate the error case when trying to configure
SR-IOV with no driver bound vs. a driver that does not implement the
.sriov_configure() callback.

Link: https://lore.kernel.org/r/20210327175140.682708-1-mdf@kernel.org
Reported-by: Brian Foley <bpfoley@google.com>
Signed-off-by: Moritz Fischer <mdf@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Krzysztof Wilczyński <kw@linux.com>

PCI/sysfs: Use sysfs_emit() and sysfs_emit_at() in "show" functions

The sysfs_emit() and sysfs_emit_at() functions were introduced to make
it less ambiguous which function is preferred when writing to the output
buffer in a device attribute's "show" callback [1].

Convert the PCI sysfs object "show" functions from sprintf(), snprintf()
and scnprintf() to sysfs_emit() and sysfs_emit_at() accordingly, as the
latter is aware of the PAGE_SIZE buffer and correctly returns the number
of bytes written into the buffer.

No functional change intended.

[1] Documentation/filesystems/sysfs.rst

Related commit: ad025f8e46f3 ("PCI/sysfs: Use sysfs_emit() and
sysfs_emit_at() in "show" functions").

Link: https://lore.kernel.org/r/20210603000112.703037-2-kw@linux.com
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>

PCI/IOV: Add sysfs MSI-X vector assignment interface

A typical cloud provider SR-IOV use case is to create many VFs for use by
guest VMs. The VFs may not be assigned to a VM until a customer requests a
VM of a certain size, e.g., number of CPUs. A VF may need MSI-X vectors
proportional to the number of CPUs in the VM, but there is no standard way
to change the number of MSI-X vectors supported by a VF.

Some Mellanox ConnectX devices support dynamic assignment of MSI-X vectors
to SR-IOV VFs. This can be done by the PF driver after VFs are enabled,
and it can be done without affecting VFs that are already in use. The
hardware supports a limited pool of MSI-X vectors that can be assigned to
the PF or to individual VFs. This is device-specific behavior that
requires support in the PF driver.

Add a read-only "sriov_vf_total_msix" sysfs file for the PF and a writable
"sriov_vf_msix_count" file for each VF. Management software may use these
to learn how many MSI-X vectors are available and to dynamically assign
them to VFs before the VFs are passed through to a VM.

If the PF driver implements the ->sriov_get_vf_total_msix() callback,
"sriov_vf_total_msix" contains the total number of MSI-X vectors available
for distribution among VFs.

If no driver is bound to the VF, writing "N" to "sriov_vf_msix_count" uses
the PF driver ->sriov_set_msix_vec_count() callback to assign "N" MSI-X
vectors to the VF. When a VF driver subsequently reads the MSI-X Message
Control register, it will see the new Table Size "N".

Link: https://lore.kernel.org/linux-pci/20210314124256.70253-2-leon@kernel.org
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>

PCI/IOV: Mark VFs as not implementing PCI_COMMAND_MEMORY

For VFs, the Memory Space Enable bit in the Command Register is
hard-wired to 0.

Add a new bit to signify devices where the Command Register Memory
Space Enable bit does not control the device's response to MMIO
accesses.

Fixes: abafbc551fdd ("vfio-pci: Invalidate mmaps and block MMIO access on disabled memory")
Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

s390/pci: create links between PFs and VFs

On s390 PCI Virtual Functions (VFs) are scanned by firmware and are made
available to Linux via the hot-plug interface. As such the common code
path of doing the scan directly using the parent Physical Function (PF)
is not used and fenced off with the no_vf_scan attribute.

Even if the partition created the VFs itself e.g. using the sriov_numvfs
attribute of a PF, the PF/VF links thus need to be established after the
fact. To do this when a VF is plugged we scan through all functions on
the same zbus and test whether they are the parent PF in which case we
establish the necessary links.

With these links established there is now no more need to fence off
pci_iov_remove_virtfn() for pdev->no_vf_scan as the common code now
works fine.

Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Link: https://lore.kernel.org/r/20200506154139.90609-3-schnelle@linux.ibm.com
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

PCI/IOV: Introduce pci_iov_sysfs_link() function

Currently pci_iov_add_virtfn() scans the SR-IOV BARs, adds the VF to the
bus and also creates the sysfs links between the newly added VF and its
parent PF.

With pdev->no_vf_scan fencing off the entire pci_iov_add_virtfn() call
s390 as the sole pdev->no_vf_scan user thus ends up missing these sysfs
links which are required for example by QEMU/libvirt.

Instead of duplicating the code refactor pci_iov_add_virtfn() to make
sysfs link creation callable separately.

Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Link: https://lore.kernel.org/r/20200506154139.90609-1-schnelle@linux.ibm.com
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

PCI/IOV: Fix memory leak in pci_iov_add_virtfn()

In the implementation of pci_iov_add_virtfn() the allocated virtfn is
leaked if pci_setup_device() fails. The error handling is not calling
pci_stop_and_remove_bus_device(). Change the goto label to failed2.

Fixes: 156c55325d30 ("PCI: Check for pci_setup_device() failure in pci_iov_add_virtfn()")
Link: https://lore.kernel.org/r/20191125195255.23740-1-navid.emamdoost@gmail.com
Signed-off-by: Navid Emamdoost <navid.emamdoost@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Remove unnecessary includes

Remove unnecessary includes.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/IOV: Serialize sysfs sriov_numvfs reads vs writes

When sriov_numvfs is being updated, we call the driver->sriov_configure()
function, which may enable VFs and call probe functions, which may make new
devices visible. This all happens before before sriov_numvfs_store()
updates sriov->num_VFs, so previously, concurrent sysfs reads of
sriov_numvfs returned stale values.

Serialize the sysfs read vs the write so the read returns the correct
num_VFs value.

[bhelgaas: hold device_lock instead of checking mutex_is_locked()]
Link: https://bugzilla.kernel.org/show_bug.cgi?id=202991
Link: https://lore.kernel.org/r/20190911072736.32091-1-pierre.cregut@orange.com
Signed-off-by: Pierre Crégut <pierre.cregut@orange.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/IOV: Remove group write permission from sriov_numvfs, sriov_drivers_autoprobe

Previously the sriov_numvfs and sriov_drivers_autoprobe sysfs files had
0664 permissions, which allowed group write. libvirt runs as root when
dealing with PCI, and it chowns files needed by qemu, so group write
permission should not be needed.

Change these permissions from 0664 to 0644, which is what DEVICE_ATTR_RW()
does by default.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20190905063226.43269-1-skunberg.kelsey@gmail.com
Link: https://lore.kernel.org/r/850cf536-0b72-d78c-efaf-855dcb391087@redhat.com
Signed-off-by: Kelsey Skunberg <skunberg.kelsey@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Donald Dutile <ddutile@redhat.com>

PCI/IOV: Move sysfs SR-IOV functions to iov.c

The sysfs SR-IOV functions are only needed when the kernel is built with
SR-IOV support. Rather than put them in pci-sysfs.c under #ifdef
CONFIG_PCI_IOV, move them to iov.c, which is only compiled when
CONFIG_PCI_IOV=y.

Link: https://lore.kernel.org/r/20190813204513.4790-4-skunberg.kelsey@gmail.com
Signed-off-by: Kelsey Skunberg <skunberg.kelsey@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Donald Dutile <ddutile@redhat.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

PCI: Use PCI_SRIOV_NUM_BARS in loops instead of PCI_IOV_RESOURCE_END

Writing loop conditions as "i < NUM" is a common C idiom; using "i <= END"
is unusual and thus prone to errors. Change loops to use the former.

Link: https://lore.kernel.org/r/20190806140715.19847-1-efremov@linux.com
Signed-off-by: Denis Efremov <efremov@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

Revert "PCI/IOV: Use VF0 cached config space size for other VFs"

Revert 975bb8b4dc93 ("PCI/IOV: Use VF0 cached config space size for other
VFs"), which attempted to cache the config space size from the first VF to
re-use for subsequent VFs.

The cached value was determined prior to discovering the PCIe capability on
the VF, which resulted in the first VF reporting the correct config space
size (4K), as it has a special case through pci_cfg_space_size(), while all
the other VFs only reported 256 bytes. As this was only a performance
optimization, we're better off without it.

Fixes: 975bb8b4dc93 ("PCI/IOV: Use VF0 cached config space size for other VFs")
Link: https://lore.kernel.org/r/156046663197.29869.3633634445109057665.stgit@gimli.home
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: KarimAllah Ahmed <karahmed@amazon.de>
Cc: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Cc: Hao Zheng <yinhe@linux.alibaba.com>

PCI/IOV: Add flag so platforms can skip VF scanning

Provide a flag to skip scanning for new VFs after SR-IOV enablement. This
can be set by implementations for which the VFs are already reported by
other means.

Signed-off-by: Sebastian Ott <sebott@linux.ibm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

PCI/IOV: Factor out sriov_add_vfs()

Provide sriov_add_vfs() as a wrapper to scan for VFs that cleans up after
itself. This is just a code simplification. No functional change.

Signed-off-by: Sebastian Ott <sebott@linux.ibm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

PCI/IOV: Remove unnecessary include of <linux/pci-ats.h>

iov.c uses nothing declared in <linux/pci-ats.h>, so remove the include of
it. No functional change intended.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/IOV: Use VF0 cached config space size for other VFs

Cache the config space size from VF0 and use it for all other VFs instead
of reading it from the config space of each VF. We assume that it will be
the same across all associated VFs.

This is an optimization when enabling SR-IOV on a device with many VFs.

Signed-off-by: KarimAllah Ahmed <karahmed@amazon.de>
[bhelgaas: use CONFIG_PCI_IOV (not CONFIG_PCI_ATS)]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/IOV: Tidy pci_sriov_set_totalvfs()

Fix minor style issues in pci_sriov_set_totalvfs(). No functional change
intended.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/IOV: Reset total_VFs limit after detaching PF driver

The TotalVFs register in the SR-IOV capability is the hardware limit on the
number of VFs. A PF driver can limit the number of VFs further with
pci_sriov_set_totalvfs(). When the PF driver is removed, reset any VF
limit that was imposed by the driver because that limit may not apply to
other drivers.

Before 8d85a7a4f2c9 ("PCI/IOV: Allow PF drivers to limit total_VFs to 0"),
pci_sriov_set_totalvfs(pdev, 0) meant "we can enable TotalVFs virtual
functions", and the nfp driver used that to remove the VF limit when the
driver unloads.

8d85a7a4f2c9 broke that because instead of removing the VF limit,
pci_sriov_set_totalvfs(pdev, 0) actually sets the limit to zero, and that
limit persists even if another driver is loaded.

We could fix that by making the nfp driver reset the limit when it unloads,
but it seems more robust to do it in the PCI core instead of relying on the
driver.

The regression scenario is:

nfp_pci_probe (driver 1)
...
nfp_pci_remove
pci_sriov_set_totalvfs(pf->pdev, 0) # limits VFs to 0

...
nfp_pci_probe (driver 2)
nfp_rtsym_read_le("nfd_vf_cfg_max_vfs")
# no VF limit from firmware

Now driver 2 is broken because the VF limit is still 0 from driver 1.

Fixes: 8d85a7a4f2c9 ("PCI/IOV: Allow PF drivers to limit total_VFs to 0")
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
[bhelgaas: changelog, rename functions]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/IOV: Allow PF drivers to limit total_VFs to 0

Some SR-IOV PF drivers implement .sriov_configure(), which allows
user-space to enable VFs by writing the desired number of VFs to the sysfs
"sriov_numvfs" file (see sriov_numvfs_store()).

The PCI core limits the number of VFs to the TotalVFs advertised by the
device in its SR-IOV capability. The PF driver can limit the number of VFs
to even fewer (it may have pre-allocated data structures or knowledge of
device limitations) by calling pci_sriov_set_totalvfs(), but previously it
could not limit the VFs to 0.

Change pci_sriov_get_totalvfs() so it always respects the VF limit imposed
by the PF driver, even if the limit is 0.

This sequence:

pci_sriov_set_totalvfs(dev, 0);
x = pci_sriov_get_totalvfs(dev);

previously set "x" to TotalVFs from the SR-IOV capability. Now it will set
"x" to 0.

Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
[bhelgaas: split to separate patch]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/IOV: Add pci_sriov_configure_simple()

SR-IOV (Single Root I/O Virtualization) is an optional PCIe capability (see
PCIe r4.0, sec 9). A PCIe Function with the SR-IOV capability is referred
to as a PF (Physical Function). If SR-IOV is enabled on the PF, several
VFs (Virtual Functions) may be created. The VFs can be individually
assigned to virtual machines, which allows them to share a single hardware
device while being isolated from each other.

Some SR-IOV devices have resources such as queues and interrupts that must
be set up in the PF before enabling the VFs, so they require a PF driver to
do that.

Other SR-IOV devices don't require any PF setup before enabling VFs. Add a
pci_sriov_configure_simple() interface so PF drivers for such devices can
use it without repeating the VF-enabling code.

Tested-by: Mark Rustad <mark.d.rustad@intel.com>
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
[bhelgaas: changelog, comment]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Greg Rose <gvrose8192@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>:wq

PCI/IOV: Use VF0 cached config registers for other VFs

Cache some config data from VF0 and use it for all other VFs instead of
reading it from the config space of each VF. We assume these items are the
same across all associated VFs:

Revision ID
Class Code
Subsystem Vendor ID
Subsystem ID

This is an optimization when enabling SR-IOV on a device with many VFs.

Signed-off-by: KarimAllah Ahmed <karahmed@amazon.de>
[bhelgaas: changelog, simplify comments, remove unused "device", test
CONFIG_PCI_IOV instead of CONFIG_PCI_ATS, rename functions]
Signed-off-by: Bjorn Helgaas <helgaas@kernel.org>

PCI: Tidy comments

Remove pointless comments that tell us the file name, remove blank line
comments, follow multi-line comment conventions. No functional change
intended.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Add SPDX GPL-2.0 when no license was specified

b24413180f56 ("License cleanup: add SPDX GPL-2.0 license identifier to
files with no license") added SPDX GPL-2.0 to several PCI files that
previously contained no license information.

Add SPDX GPL-2.0 to all other PCI files that did not contain any license
information and hence were under the default GPL version 2 license of the
kernel.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

PCI: Add wrappers for dev_printk()

Add PCI-specific dev_printk() wrappers and use them to simplify the code
slightly. No functional change intended.

Signed-off-by: Frederick Lawler <fred@fredlawl.com>
[bhelgaas: squash into one patch]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/IOV: Add pci_vf_drivers_autoprobe() interface

Add a pci_vf_drivers_autoprobe() interface. Setting autoprobe to false
on the PF prevents drivers from binding to VFs when they are enabled.

Signed-off-by: Bryant G. Ly <bryantly@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalvare@linux.vnet.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Russell Currey <ruscur@russell.cc>
Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>

PCI: Restore ARI Capable Hierarchy before setting numVFs

In the restore path, we previously read PCI_SRIOV_VF_OFFSET and
PCI_SRIOV_VF_STRIDE before restoring PCI_SRIOV_CTRL_ARI:

pci_restore_state
pci_restore_iov_state
sriov_restore_state
pci_iov_set_numvfs
pci_read_config_word(... PCI_SRIOV_VF_OFFSET, &iov->offset)
pci_read_config_word(... PCI_SRIOV_VF_STRIDE, &iov->stride)
pci_write_config_word(... PCI_SRIOV_CTRL, iov->ctrl)

But per SR-IOV r1.1, sec 3.3.3.5, the device can use PCI_SRIOV_CTRL_ARI to
determine PCI_SRIOV_VF_OFFSET and PCI_SRIOV_VF_STRIDE. Therefore, this
path, which is used for suspend/resume and AER recovery, can corrupt
iov->offset and iov->stride.

Since the iov state is associated with the device, not the driver, if we
reload the driver, it will use the the corrupted data, which may cause
crashes like this:

kernel BUG at drivers/pci/iov.c:157!
RIP: 0010:pci_iov_add_virtfn+0x2eb/0x350
Call Trace:
pci_enable_sriov+0x353/0x440
ixgbe_pci_sriov_configure+0xd5/0x1f0 [ixgbe]
sriov_numvfs_store+0xf7/0x170
dev_attr_store+0x18/0x30
sysfs_kf_write+0x37/0x40
kernfs_fop_write+0x120/0x1b0
vfs_write+0xb5/0x1a0
SyS_write+0x55/0xc0

Restore PCI_SRIOV_CTRL_ARI before calling pci_iov_set_numvfs(), then
restore the rest of PCI_SRIOV_CTRL (which may set PCI_SRIOV_CTRL_VFE)
afterwards.

Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
[bhelgaas: changelog, add comment, also clear ARI if necessary]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Alexander Duyck <alexander.h.duyck@intel.com>
CC: Emil Tantilov <emil.s.tantilov@intel.com>

PCI: Create SR-IOV virtfn/physfn links before attaching driver

When creating virtual functions, create the "virtfn%u" and "physfn" links
in sysfs *before* attaching the driver instead of after. When we attach
the driver to the new virtual network interface first, there is a race when
the driver attaches to the new sends out an "add" udev event, and the
network interface naming software (biosdevname or systemd, for example)
tries to look at these links.

Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Cache the VF device ID in the SR-IOV structure

Cache the VF device ID in the SR-IOV structure and use it instead of
reading it over and over from the PF config space capability.

Signed-off-by: Filippo Sironi <sironi@amazon.de>
[bhelgaas: rename to "vf_device" to match pci_dev->device]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Remove reset argument from pci_iov_{add,remove}_virtfn()

The "reset" argument passed to pci_iov_add_virtfn() and
pci_iov_remove_virtfn() is always zero since 46cb7b1bd86f ("PCI: Remove
unused SR-IOV VF Migration support")

Remove the argument together with the associated code.

Signed-off-by: Jan H. Schönherr <jschoenh@amazon.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Russell Currey <ruscur@russell.cc>

PCI: Disable VF decoding before pcibios_sriov_disable() updates resources

A struct resource represents the address space consumed by a device. We
should not modify that resource while the device is actively using the
address space. For VFs, pci_iov_update_resource() enforces this by
printing a warning and doing nothing if the VFE (VF Enable) and MSE (VF
Memory Space Enable) bits are set.

Previously, both sriov_enable() and sriov_disable() called the
pcibios_sriov_disable() arch hook, which may update the struct resource,
while VFE and MSE were enabled. This effectively dropped the resource
update pcibios_sriov_disable() intended to do.

Disable VF memory decoding before calling pcibios_sriov_disable().

Reported-by: Carol L Soto <clsoto@us.ibm.com>
Tested-by: Carol L Soto <clsoto@us.ibm.com>
Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
[bhelgaas: changelog]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: shan.gavin@gmail.com
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>

PCI: Protect pci_driver->sriov_configure() usage with device_lock()

Every method in struct device_driver or structures derived from it like
struct pci_driver MUST provide exclusion vs the driver's ->remove() method,
usually by using device_lock().

Protect use of pci_driver->sriov_configure() by holding the device lock
while calling it.

The PCI core sets the pci_dev->driver pointer in local_pci_probe() before
calling ->probe() and only clears it after ->remove(). This means driver's
->sriov_configure() callback will happily race with probe() and remove(),
most likely leading to BUGs, since drivers don't expect this.

Remove the iov lock completely, since we remove the last user.

[bhelgaas: changelog, thanks to Christoph for locking rule]
Link: http://lkml.kernel.org/r/20170522225023.14010-1-jakub.kicinski@netronome.com
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

PCI: Add sysfs sriov_drivers_autoprobe to control VF driver binding

Sometimes it is not desirable to bind SR-IOV VFs to drivers. This can save
host side resource usage by VF instances that will be assigned to VMs.

Add a new PCI sysfs interface "sriov_drivers_autoprobe" to control that
from the PF. To modify it, echo 0/n/N (disable probe) or 1/y/Y (enable
probe) to:

/sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_drivers_autoprobe

Note that this must be done before enabling VFs. The change will not take
effect if VFs are already enabled. Simply, one can disable VFs by setting
sriov_numvfs to 0, choose whether to probe or not, and then re-enable the
VFs by restoring sriov_numvfs.

[bhelgaas: changelog, ABI doc]
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Signed-off-by: Eli Cohen <eli@mellanox.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
Reviewed-by: Alex Williamson <alex.williamson@redhat.com>

PCI: Lock each enable/disable num_vfs operation in sysfs

Enabling/disabling SRIOV via sysfs by echo-ing multiple values
simultaneously:

# echo 63 > /sys/class/net/ethX/device/sriov_numvfs&
# echo 63 > /sys/class/net/ethX/device/sriov_numvfs

# sleep 5

# echo 0 > /sys/class/net/ethX/device/sriov_numvfs&
# echo 0 > /sys/class/net/ethX/device/sriov_numvfs

results in the following bug:

kernel BUG at drivers/pci/iov.c:495!
invalid opcode: 0000 [#1] SMP
CPU: 1 PID: 8050 Comm: bash Tainted: G W 4.9.0-rc7-net-next #2092
RIP: 0010:[<ffffffff813b1647>]
[<ffffffff813b1647>] pci_iov_release+0x57/0x60

Call Trace:
[<ffffffff81391726>] pci_release_dev+0x26/0x70
[<ffffffff8155be6e>] device_release+0x3e/0xb0
[<ffffffff81365ee7>] kobject_cleanup+0x67/0x180
[<ffffffff81365d9d>] kobject_put+0x2d/0x60
[<ffffffff8155bc27>] put_device+0x17/0x20
[<ffffffff8139c08a>] pci_dev_put+0x1a/0x20
[<ffffffff8139cb6b>] pci_get_dev_by_id+0x5b/0x90
[<ffffffff8139cca5>] pci_get_subsys+0x35/0x40
[<ffffffff8139ccc8>] pci_get_device+0x18/0x20
[<ffffffff8139ccfb>] pci_get_domain_bus_and_slot+0x2b/0x60
[<ffffffff813b09e7>] pci_iov_remove_virtfn+0x57/0x180
[<ffffffff813b0b95>] pci_disable_sriov+0x65/0x140
[<ffffffffa00a1af7>] ixgbe_disable_sriov+0xc7/0x1d0 [ixgbe]
[<ffffffffa00a1e9d>] ixgbe_pci_sriov_configure+0x3d/0x170 [ixgbe]
[<ffffffff8139d28c>] sriov_numvfs_store+0xdc/0x130
...
RIP [<ffffffff813b1647>] pci_iov_release+0x57/0x60

Use the existing mutex lock to protect each enable/disable operation.

Signed-off-by: Emil Tantilov <emil.s.tantilov@intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
CC: Alexander Duyck <alexander.h.duyck@intel.com>

PCI: Remove pci_resource_bar() and pci_iov_resource_bar()

pci_std_update_resource() only deals with standard BARs, so we don't have
to worry about the complications of VF BARs in an SR-IOV capability.

Compute the BAR address inline and remove pci_resource_bar(). That makes
pci_iov_resource_bar() unused, so remove that as well.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Gavin Shan <gwshan@linux.vnet.ibm.com>

PCI: Don't update VF BARs while VF memory space is enabled

If we update a VF BAR while it's enabled, there are two potential problems:

1) Any driver that's using the VF has a cached BAR value that is stale
after the update, and

2) We can't update 64-bit BARs atomically, so the intermediate state
(new lower dword with old upper dword) may conflict with another
device, and an access by a driver unrelated to the VF may cause a bus
error.

Warn about attempts to update VF BARs while they are enabled. This is a
programming error, so use dev_WARN() to get a backtrace.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Gavin Shan <gwshan@linux.vnet.ibm.com>

PCI: Separate VF BAR updates from standard BAR updates

Previously pci_update_resource() used the same code path for updating
standard BARs and VF BARs in SR-IOV capabilities.

Split the VF BAR update into a new pci_iov_update_resource() internal
interface, which makes it simpler to compute the BAR address (we can get
rid of pci_resource_bar() and pci_iov_resource_bar()).

This patch:

- Renames pci_update_resource() to pci_std_update_resource(),
- Adds pci_iov_update_resource(),
- Makes pci_update_resource() a wrapper that calls the appropriate one,

No functional change intended.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Gavin Shan <gwshan@linux.vnet.ibm.com>

PCI: Do any VF BAR updates before enabling the BARs

Previously we enabled VFs and enable their memory space before calling
pcibios_sriov_enable(). But pcibios_sriov_enable() may update the VF BARs:
for example, on PPC PowerNV we may change them to manage the association of
VFs to PEs.

Because 64-bit BARs cannot be updated atomically, it's unsafe to update
them while they're enabled. The half-updated state may conflict with other
devices in the system.

Call pcibios_sriov_enable() before enabling the VFs so any BAR updates
happen while the VF BARs are disabled.

[bhelgaas: changelog]
Tested-by: Carol Soto <clsoto@us.ibm.com>
Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Check for pci_setup_device() failure in pci_iov_add_virtfn()

If pci_setup_device() returns failure, we must return failure from
pci_iov_add_virtfn(). If we ignore the failure and continue with an
uninitialized pci_dev for virtfn, we crash later when we try to use those
uninitialized parts.

Signed-off-by: Po Liu <po.liu@nxp.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/IOV: Rename and export virtfn_{add, remove}

During EEH recovery, hotplug is applied to the devices which don't
have drivers or their drivers don't support EEH. However, the hotplug,
which was implemented based on PCI bus, can't be applied to VF directly.
Instead, we unplug and plug individual PCI devices (VFs).

This renames virtn_{add,remove}() and exports them so they can be used
in PCI hotplug during EEH recovery.

Signed-off-by: Wei Yang <weiyang@linux.vnet.ibm.com>
Reviewed-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>

PCI: Support SR-IOV on any function type

Previously, we only supported SR-IOV on PCI Express Endpoints and Root
Complex Integrated Endpoints. This restriction has been present since
d1b054da8f59 ("PCI: initialize and release SR-IOV capability") added SR-IOV
support, but the spec does not require it. In fact, the SR-IOV spec r1.1,
sec 3.3, says the SR-IOV extended capability may be present for any Type 0
function.

Remove the function type test, so we can support SR-IOV on any function.

Some AMD GPUs have display outputs, use the VGA class code, are Legacy
Endpoints, and support SR-IOV. This change allows Linux to enable SR-IOV
on these devices.

[bhelgaas: changelog]
Link: https://bugzilla.kernel.org/show_bug.cgi?id=112221
Signed-off-by: Kelly Zytaruk <kelly.zytaruk@amd.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Fix sriov_enable() error path for pcibios_enable_sriov() failures

Disable VFs if pcibios_enable_sriov() fails, just like we do for other
errors in sriov_enable(). Call pcibios_sriov_disable() if virtfn_add()
fails.

[bhelgaas: changelog, split to separate patch for reviewability]
Fixes: 995df527f399 ("PCI: Add pcibios_sriov_enable() and pcibios_sriov_disable()")
Signed-off-by: Alexander Duyck <aduyck@mirantis.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Wei Yang <weiyang@linux.vnet.ibm.com>

PCI: Wait 1 second between disabling VFs and clearing NumVFs

Per sec 3.3.3.1 of the SR-IOV spec, r1.1, we must allow 1.0s after clearing
VF Enable before reading any field in the SR-IOV Extended Capability.

Wait 1 second before calling pci_iov_set_numvfs(), which reads
PCI_SRIOV_VF_OFFSET and PCI_SRIOV_VF_STRIDE after it sets PCI_SRIOV_NUM_VF.

[bhelgaas: split to separate patch for reviewability, add spec reference]
Signed-off-by: Alexander Duyck <aduyck@mirantis.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Reorder pcibios_sriov_disable()

Move pcibios_sriov_disable() up so it's defined before a future use.

[bhelgaas: split to separate patch for reviewability]
Signed-off-by: Alexander Duyck <aduyck@mirantis.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Wei Yang <weiyang@linux.vnet.ibm.com>

PCI: Remove VFs in reverse order if virtfn_add() fails

If virtfn_add() fails, we call virtfn_remove() for any previously added
devices. Remove the devices in reverse order (first-added is
last-removed), which is more natural and doesn't require an additional
variable.

[bhelgaas: changelog, split to separate patch for reviewability]
Signed-off-by: Alexander Duyck <aduyck@mirantis.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Wei Yang <weiyang@linux.vnet.ibm.com>

PCI: Handle Enhanced Allocation capability for SR-IOV devices

SR-IOV BARs can be specified via EA entries. Extend the EA parser to
extract the SRIOV BAR resources, and modify sriov_init() to use resources
previously obtained via EA.

Signed-off-by: David Daney <david.daney@cavium.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Sean O. Stalley <sean.stalley@intel.com>

PCI: Remove redundant validation of SR-IOV offset/stride registers

Previously, we read, validated, and cached PCI_SRIOV_VF_OFFSET and
PCI_SRIOV_VF_STRIDE in sriov_enable(). But sriov_init() now does
that via compute_max_vf_buses(), so we don't need to do it again.

Remove the PCI_SRIOV_VF_OFFSET and PCI_SRIOV_VF_STRIDE config reads from
sriov_enable(). The pci_sriov structure already contains the offset and
stride corresponding to the current NumVFs.

[bhelgaas: split to separate patch for reviewability]
Signed-off-by: Alexander Duyck <aduyck@mirantis.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Wei Yang <weiyang@linux.vnet.ibm.com>

PCI: Set SR-IOV NumVFs to zero after enumeration

The enumeration path should leave NumVFs set to zero. But after
4449f079722c ("PCI: Calculate maximum number of buses required for VFs"),
we call virtfn_max_buses() in the enumeration path, which changes NumVFs.
This NumVFs change is visible via lspci and sysfs until a driver enables
SR-IOV.

Iterate from TotalVFs down to zero so NumVFs is zero when we're finished
computing the maximum number of buses. Validate offset and stride in
the loop, so we can test it at every possible NumVFs setting. Rename
virtfn_max_buses() to compute_max_vf_buses() to hint that it does have a
side effect of updating iov->max_VF_buses.

[bhelgaas: changelog, rename, allow numVF==1 && stride==0, rework loop,
reverse sense of error path]
Fixes: 4449f079722c ("PCI: Calculate maximum number of buses required for VFs")
Based-on-patch-by: Ethan Zhao <ethan.zhao@oracle.com>
Signed-off-by: Alexander Duyck <aduyck@mirantis.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Enable SR-IOV ARI Capable Hierarchy before reading TotalVFs

For some SR-IOV devices, the number of available virtual functions, i.e.,
TotalVFs, increases after setting the ARI Capable Hierarchy bit in the
SR-IOV Control register. This violates the SR-IOV spec, r1.1, sec 3.3.6,
which says TotalVFs is HwInit, but we don't need TotalVFs before setting
the ARI Capable bit anyway.

Set the ARI Capable Hierarchy bit (if ARI is enabled in the upstream
bridge) before reading TotalVFs.

[bhelgaas: changelog]
Signed-off-by: Ben Shelton <benjamin.h.shelton@intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Add pcibios_iov_resource_alignment() interface

Per the SR-IOV spec r1.1, sec 3.3.14, the required alignment of a PF's IOV
BAR is the size of an individual VF BAR, and the size consumed is the
individual VF BAR size times NumVFs.

The PowerNV platform has additional alignment requirements to help support
its Partitionable Endpoint device isolation feature (see
Documentation/powerpc/pci_iov_resource_on_powernv.txt).

Add a pcibios_iov_resource_alignment() interface to allow platforms to
request additional alignment.

[bhelgaas: changelog, adapt to reworked pci_sriov_resource_alignment(),
drop "align" parameter]
Signed-off-by: Wei Yang <weiyang@linux.vnet.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

PCI: Add pcibios_sriov_enable() and pcibios_sriov_disable()

VFs are dynamically created when a driver enables them. On some platforms,
like PowerNV, special resources are necessary to enable VFs.

Add platform hooks for enabling and disabling VFs.

Signed-off-by: Wei Yang <weiyang@linux.vnet.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

PCI: Export pci_iov_virtfn_bus() and pci_iov_virtfn_devfn()

On PowerNV, some resource reservation is needed for SR-IOV VFs that don't
exist at the bootup stage. To do the match between resources and VFs, the
code need to get the VF's BDF in advance.

Rename virtfn_bus() and virtfn_devfn() to pci_iov_virtfn_bus() and
pci_iov_virtfn_devfn() and export them.

[bhelgaas: changelog, make "busnr" int]
Signed-off-by: Wei Yang <weiyang@linux.vnet.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

PCI: Calculate maximum number of buses required for VFs

An SR-IOV device can change its First VF Offset and VF Stride based on the
values of ARI Capable Hierarchy and NumVFs. The number of buses required
for all VFs is determined by NumVFs, First VF Offset, and VF Stride (see
SR-IOV spec r1.1, sec 2.1.2).

Previously pci_iov_bus_range() computed how many buses would be required by
TotalVFs, but this was based on a single NumVFs value and may not have been
the maximum for all NumVFs configurations.

Iterate over all valid NumVFs and calculate the maximum number of bus
numbers that could ever be required for VFs of this device.

[bhelgaas: changelog, compute busnr of NumVFs, not TotalVFs, remove
kerenl-doc comment marker]
Signed-off-by: Wei Yang <weiyang@linux.vnet.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

PCI: Refresh First VF Offset and VF Stride when updating NumVFs

The First VF Offset and VF Stride fields depend on the NumVFs setting, so
refresh the cached fields in struct pci_sriov when updating NumVFs. See
the SR-IOV spec r1.1, sec 3.3.9 and 3.3.10.

[bhelgaas: changelog, remove kernel-doc comment marker]
Signed-off-by: Wei Yang <weiyang@linux.vnet.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

PCI: Index IOV resources in the conventional style

Most of PCI uses "res = &dev->resource[i]", not "res = dev->resource + i".
Use that style in iov.c also.

No functional change.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Wei Yang <weiyang@linux.vnet.ibm.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

PCI: Keep individual VF BAR size in struct pci_sriov

Currently we don't store the individual VF BAR size. We calculate it when
needed by dividing the PF's IOV resource size (which contains space for
*all* the VFs) by total_VFs or by reading the BAR in the SR-IOV capability
again.

Keep the individual VF BAR size in struct pci_sriov.barsz[], add
pci_iov_resource_size() to retrieve it, and use that instead of doing the
division or reading the SR-IOV capability BAR.

[bhelgaas: rename to "barsz[]", simplify barsz[] index computation, remove
SR-IOV capability BAR sizing]
Signed-off-by: Wei Yang <weiyang@linux.vnet.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

PCI: Print PF SR-IOV resource that contains all VF(n) BAR space

When we size VF BAR0, VF BAR1, etc., from the SR-IOV Capability of a PF, we
learn the alignment requirement and amount of space consumed by a single
VF. But when VFs are enabled, *each* of the NumVFs consumes that amount of
space, so the total size of the PF resource is "VF BAR size * NumVFs".

Add a printk of the total space consumed by the VFs corresponding to what
we already do for normal non-IOV BARs.

No functional change; new message only.

[bhelgaas: split out into its own patch]
Signed-off-by: Wei Yang <weiyang@linux.vnet.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

PCI: Print more info in sriov_enable() error message

If we don't have space for all the bus numbers required to enable VFs,
print the largest bus number required and the range available.

No functional change; improved error message only.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Wei Yang <weiyang@linux.vnet.ibm.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

PCI: Remove fixed parameter in pci_iov_resource_bar()

pci_iov_resource_bar() always sets its 'pci_bar_type' parameter to
'pci_bar_unknown'. Drop the parameter and just use 'pci_bar_unknown'
directly in the callers.

No functional change intended.

Signed-off-by: Myron Stowe <myron.stowe@redhat.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
CC: Chris Wright <chrisw@sous-sol.org>
CC: Yu Zhao <yuzhao@google.com>

PCI: Use device flag helper functions

Use PCI device flag helper functions when checking whether a device is
assigned. No functional change.

Signed-off-by: Ethan Zhao <ethan.zhao@oracle.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Make pci_bus_add_device() void

pci_bus_add_device() always returns 0, so there's no point in returning
anything at all. Make it a void function and remove the tests of the
return value from the callers.

[bhelgaas: changelog, remove unused "err" from i82875p_setup_overfl_dev()]
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Remove unused SR-IOV VF Migration support

This reverts commit 74bb1bcc7dbb ("PCI: handle SR-IOV Virtual Function
Migration"), removing this exported interface:

pci_sriov_migration()

Since pci_sriov_migration() is unused, it is impossible to schedule
sriov_migration_task() or use any of the other migration infrastructure.

This is based on Stephen Hemminger's patch (see link below), but goes a bit
further.

Link: http://lkml.kernel.org/r/20131227132710.7190647c@nehalam.linuxnetplumber.net
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
CC: Stephen Hemminger <stephen@networkplumber.org>

PCI: Never treat a VF as a multifunction device

Per the SR-IOV spec rev 1.1:

3.4.1.9 Header Type (Offset 0Eh)

"... For VFs, this register must be RO Zero."

Unfortunately some devices get this wrong, ex. Emulex OneConnect 10Gb NIC.
When they do it makes us handle ACS testing and therefore IOMMU groups as
if they were actual multifunction devices and require ACS capabilities to
make sure there's no peer-to-peer between functions. VFs are never
traditional multifunction devices, so simply clear this bit before we get
any further into setup.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=68431
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Clear NumVFs when disabling SR-IOV in sriov_init()

When SR-IOV is disabled (VF Enable is cleared), NumVFs is not very useful,
so this patch clears it out to prevent confusing lspci output like that
below. We already clear NumVFs in sriov_disable(), and this does the same
when we disable SR-IOV as part of parsing the SR-IOV capability.

$ lspci -vvv -s 13:00.0
13:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy+
Initial VFs: 64, Total VFs: 64, Number of VFs: 64, ...

[bhelgaas: changelog]
Signed-off-by: ethan.zhao <ethan.kernel@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Fix whitespace, capitalization, and spelling errors

Fix whitespace, capitalization, and spelling errors. No functional change.
I know "busses" is not an error, but "buses" was more common, so I used it
consistently.

Signed-off-by: Marta Rybczynska <rybczynska@gmail.com> (pci_reset_bridge_secondary_bus())
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PCI: Return -ENOSYS for SR-IOV operations on non-SR-IOV devices

Change the return value to -ENOSYS if a device is not an SR-IOV PF.
Previously we returned either -ENODEV or -EINVAL.

Also have pci_sriov_get_totalvfs() return 0 in the error case to make the
behaviour consistent whether CONFIG_PCI_IOV is enabled or not.

Signed-off-by: Stefan Assmann <sassmann@kpanic.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Update NumVFs register when disabling SR-IOV

Currently, we only update NumVFs register during sriov_enable().
This register should also be updated during sriov_disable() and when
sriov_enable() fails. Otherwise, we will get the stale "Number of VFs"
info from lspci.

[bhelgaas: changelog]
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Fix comment typo in iov.c

"Devic3" should be "device."

Signed-off-by: Jonghwan Choi <jhbird.choi@samsung.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Simplify IOV implementation and fix reference count races

Trivial changes to IOV:

1) use new PCI interfaces to simplify IOV implementation
2) fix some reference count related race windows

[bhelgaas: fix virtfn_add() add bus/alloc dev error paths]
Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Donald Dutile <ddutile@redhat.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Ram Pai <linuxram@us.ibm.com>

PCI: Drop redundant setting of bus->is_added in virtfn_add_bus()

The flag pci_bus->is_added is used to guard invocation of
pcibios_fixup_bus(pci_bus). When virtfn_add_bus() is called, the
pci_bus->is_added flag has already been set, so remove the redundant

bus->is_added = 1;

Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Donald Dutile <ddutile@redhat.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Ram Pai <linuxram@us.ibm.com>

PCI: Convert alloc_pci_dev(void) to pci_alloc_dev(bus)

Use the new pci_alloc_dev(bus) to replace the existing using of
alloc_pci_dev(void).

[bhelgaas: drop pci_bus ref later in pci_release_dev()]
Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: David Airlie <airlied@linux.ie>
Cc: Neela Syam Kolli <megaraidlinux@lsi.com>
Cc: "James E.J. Bottomley" <JBottomley@parallels.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>

PCI: Finish SR-IOV VF setup before adding the device

Commit 4f535093cf "PCI: Put pci_dev in device tree as early as possible"
moves device registering from pci_bus_add_devices() to pci_device_add().
That causes problems for virtual functions because device_add(&virtfn->dev)
is called before setting the virtfn->is_virtfn flag, which then causes Xen
to report PCI virtual functions as PCI physical functions.

Fix it by setting virtfn->is_virtfn before calling pci_device_add().

[Jiang Liu]: Move the setting of virtfn->is_virtfn ahead further for better
readability and modify changelog.

Signed-off-by: Xudong Hao <xudong.hao@intel.com>
Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: stable@vger.kernel.org # v3.9+

pci: Add SRIOV helper function to determine if VFs are assigned to guest

This function is meant to add a helper function that will determine if a PF
has any VFs that are currently assigned to a guest. We currently have been
implementing this function per driver, and going forward I would like to avoid
that by making this function generic and using this helper.

v2: Removed extern from declaration of pci_vfs_assigned in pci.h and return
0 if SR-IOV is disabled with is inline with other PCI SRIOV functions.

Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>

PCI: Remove unused "rc" in virtfn_add_bus()

Delete unused variable "rc" to fix build warning in drivers/pci/iov.c

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Put pci_dev in device tree as early as possible

We want to put pci_dev structs in the device tree as soon as possible so
for_each_pci_dev() iteration will not miss them, but driver attachment
needs to be delayed until after pci_assign_unassigned_resources() to make
sure all devices have resources assigned first.

This patch moves device registering from pci_bus_add_devices() to
pci_device_add(), which happens earlier, leaving driver attachment in
pci_bus_add_devices().

It also removes unattached child bus handling in pci_bus_add_devices().
That's not needed because child bus via pci_add_new_bus() is already
in parent bus children list.

[bhelgaas: changelog]
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

pci: fix iov.c kernel-doc warnings

Fix kernel-doc warning in iov.c:

Warning(drivers/pci/iov.c:752): No description found for parameter 'numvfs'

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Sorry-by: Don Dutile <ddutile@redhat.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

PCI: Remove useless "!dev" tests

No need to check "!dev" when the caller should always supply a valid
pointer. If the caller *doesn't* supply a valid pointer, it probably
won't check for a failure return either. This way we'll oops and get a
backtrace.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Use spec names for SR-IOV capability fields

Use the same names (almost) as the spec for TotalVFs, InitialVFs, NumVFs.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Provide method to reduce the number of total VFs supported

Some implementations of SRIOV provide a capability structure
value of TotalVFs that is greater than what the software can support.
Provide a method to reduce the capability structure reported value
to the value the driver can support.
This ensures sysfs reports the current capability of the system,
hardware and software.
Example for its use: igb & ixgbe -- report 8 & 64 as TotalVFs,
but drivers only support 7 & 63 maximum.

Signed-off-by: Donald Dutile <ddutile@redhat.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Revert "PCI: Use hotplug-safe pci_get_domain_bus_and_slot()"

This reverts commit 433efd2247b0cbf5e7e86275e1f21281d3b99047.

When we remove an SR-IOV device, we have this call chain:

driver .remove() method
pci_disable_sriov()
sriov_disable()
virtfn_remove()
pci_get_domain_bus_and_slot()

sriov_disable() is only called for PFs, not for VFs. When it's called
for a PF, it loops through all the VFs and calls virtfn_remove() for
each. But we stop and remove VFs before PFs, so by the time we get
to virtfn_remove(), the VFs have already been stopped and deleted
from the device list. Now pci_get_domain_bus_and_slot(), which uses
bus_find_device() and relies on that device list, doesn't find the
VFs, so the VF references aren't released correctly.

Reported-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Use hotplug-safe pci_get_domain_bus_and_slot()

Following code has a race window between pci_find_bus() and pci_get_slot()
if PCI hotplug operation happens between them which removes the pci_bus.
So use PCI hotplug safe interface pci_get_domain_bus_and_slot() instead,
which also reduces code complexity.

struct pci_bus *pci_bus = pci_find_bus(domain, busno);
struct pci_dev *pci_dev = pci_get_slot(pci_bus, devfn);

Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Introduce pci_pcie_type(dev) to replace pci_dev->pcie_type

Introduce an inline function pci_pcie_type(dev) to extract PCIe
device type from pci_dev->pcie_flags_reg field, and prepare for
removing pci_dev->pcie_type.

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: register busn_res for iov bus

Insert that to tree.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: replace struct pci_bus secondary/subordinate with busn_res

Replace the struct pci_bus secondary/subordinate members with the
struct resource busn_res. Later we'll build a resource tree of these
bus numbers.

[bhelgaas: changelog]
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Rename pci_remove_bus_device to pci_stop_and_remove_bus_device

The old pci_remove_bus_device actually did stop and remove.

Make the name reflect that to reduce confusion.

This patch is done by sed scripts and changes back some incorrect
__pci_remove_bus_device changes.

Suggested-by: Jesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: Fix /sys warning when sriov enabled and card is hot removed

sysfs is a bit stricter now and emits warnings in more cases.

For SRIOV hotplug, we are calling pci_stop_dev() for each VF first
(after we update pci_stop_bus_devices) which remove each VF subdir. So
double check the VF dir in /sys before trying to remove the physfn link.

Signed-of-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: set pci sriov page size before reading SRIOV BAR

For an SRIOV device, PCI_SRIOV_SYS_PGSIZE should be set before
the PCI_SRIOV_BAR are queried. The sys pagesize defaults to 4k,
so this change is required on powerpc box with 64k base page size.

This is a regression caused due to moving SRIOV init to sriov_enable().

| commit afd24ece5c76af87f6fc477f2747b83a764f161c
| Author: Ram Pai <linuxram@us.ibm.com>

| PCI: delay configuration of SRIOV capability
| The SRIOV capability, namely page size and total_vfs of a device are
| configured during enumeration phase of the device. This can potentially
| interfere with the PCI operations of the platform, if the IOV capability
| of the device is not enabled.

Signed-off-by: Vaidyanathan Srinivasan <svaidy@linux.vnet.ibm.com>
Acked-by: Ram Pai <linuxram@us.ibm.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: delay configuration of SRIOV capability

The SRIOV capability, namely page size and total_vfs of a device are
configured during enumeration phase of the device. This can potentially
interfere with the PCI operations of the platform, if the IOV capability
of the device is not enabled.

The following patch postpones the configuration of the IOV capability of
the device to a later point, when the IOV capability is explicitly
enabled by the device driver.

The patch is tested on x86 and power platform.

Tested-by: Donald Dutile <ddutile@redhat.com>
Signed-off-by: Ram Pai <linuxram@us.ibm.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: Rework config space blocking services

pci_block_user_cfg_access was designed for the use case that a single
context, the IPR driver, temporarily delays user space accesses to the
config space via sysfs. This assumption became invalid by the time
pci_dev_reset was added as locking instance. Today, if you run two loops
in parallel that reset the same device via sysfs, you end up with a
kernel BUG as pci_block_user_cfg_access detect the broken assumption.

This reworks the pci_block_user_cfg_access to a sleeping service
pci_cfg_access_lock and an atomic-compatible variant called
pci_cfg_access_trylock. The former not only blocks user space access as
before but also waits if access was already locked. The latter service
just returns false in this case, allowing the caller to resolve the
conflict instead of raising a BUG.

Adaptions of the ipr driver were originally written by Brian King.

Acked-by: Brian King <brking@linux.vnet.ibm.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: defer enablement of SRIOV BARS

All the PCI BARs of a device are enabled when the device is enabled
using pci_enable_device(). This unnecessarily enables SRIOV BARs of the
device.

On some platforms, which do not support SRIOV as yet, the
pci_enable_device() fails to enable the device if its SRIOV BARs are not
allocated resources correctly.

The following patch fixes the above problem. The SRIOV BARs are now
enabled when IOV capability of the device is enabled in sriov_enable().

NOTE: Note, there is subtle change in the pci_enable_device() API. Any
driver that depends on SRIOV BARS to be enabled in pci_enable_device()
can fail.

The patch has been touch tested on power and x86 platform.

Tested-by: Michael Wang <wangyun@linux.vnet.ibm.com>
Signed-off-by: Ram Pai <linuxram@us.ibm.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

pci: Fix files needing export.h for EXPORT_SYMBOL/THIS_MODULE

They were implicitly getting it from device.h --> module.h but
we want to clean that up. So add the minimal header for these
macros.

Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>

PCI: Move ATS implementation into own file

ATS does not depend on IOV support, so move the code into
its own file. This file will also include support for the
PRI and PASID capabilities later.
Also give ATS its own Kconfig variable to allow selecting it
without IOV support.

Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: Move ATS declarations in seperate header file

This patch moves the relevant declarations from the local
header file in drivers/pci to a more accessible locations so
that it can be used by the AMD IOMMU driver too.
The file is named pci-ats.h because support for the PCI PRI
capability will also be added there in a later patch-set.

Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Acked-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: fix pci_resource_alignment prototype

This fixes the prototype for both pci_resource_alignment() and
pci_sriov_resource_alignment().

Patch started as debugging effort from Cam Macdonell.

Cc: Cam Macdonell <cam@cs.ualberta.ca>
Cc: Avi Kivity <avi@redhat.com>
[chrisw: add iov bits]
Signed-off-by: Chris Wright <chrisw@sous-sol.org>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h

percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.

2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).

* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.

Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>

pci: Add SR-IOV convenience functions and macros

Add and export pci_num_vf to allow other subsystems to determine how many
virtual function devices are associated with an SR-IOV physical function
device.
Add macros dev_is_pci, dev_is_ps, and dev_num_vf to make it easier for
non-PCI specific code to determine SR-IOV capabilities.

Signed-off-by: Mitch Williams <mitch.a.williams@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

PCI: use pci_is_pcie() in pci core

Change for PCI core to use pci_is_pcie() instead of checking
pci_dev->is_pcie.

Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI SR-IOV: correct broken resource alignment calculations

An SR-IOV capable device includes an SR-IOV PCIe capability which
describes the Virtual Function (VF) BAR requirements. A typical SR-IOV
device can support multiple VFs whose BARs must be in a contiguous region,
effectively an array of VF BARs. The BAR reports the size requirement
for a single VF. We calculate the full range needed by simply multiplying
the VF BAR size with the number of possible VFs and create a resource
spanning the full range.

This all seems sane enough except it artificially inflates the alignment
requirement for the VF BAR. The VF BAR need only be aligned to the size
of a single BAR not the contiguous range of VF BARs. This can cause us
to fail to allocate resources for the BAR despite the fact that we
actually have enough space.

This patch adds a thin PCI specific layer over the generic
resource_alignment() function which is aware of the special nature of
VF BARs and does sorting and allocation based on the smaller alignment
requirement.

I recognize that while resource_alignment is generic, it's basically a
PCI helper. An alternative to this patch is to add PCI VF BAR specific
information to struct resource. I opted for the extra layer rather than
adding such PCI specific information to struct resource. This does
have the slight downside that we don't cache the BAR size and re-read
for each alignment query (happens a small handful of times during boot
for each VF BAR).

Signed-off-by: Chris Wright <chrisw@sous-sol.org>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Matthew Wilcox <matthew@wil.cx>
Cc: Yu Zhao <yu.zhao@intel.com>
Cc: stable@kernel.org
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: cleanup Function Level Reset

This patch enhances the FLR functions:
1) remove disable_irq() so the shared IRQ won't be disabled.
2) replace the 1s wait with 100, 200 and 400ms wait intervals
for the Pending Transaction.
3) replace mdelay() with msleep().
4) add might_sleep().
5) lock the device to prevent PM suspend from accessing the CSRs
during the reset.
6) coding style fixes.

Reviewed-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Yu Zhao <yu.zhao@intel.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: fix SR-IOV function dependency link problem

PCIe root complex integrated endpoint does not implement ARI, so this
kind of endpoint uses 3-bit function number. The function dependency
link of the integrated endpoint should be calculated using the device
number plus the value from function dependency link register.

Normal endpoint always implements ARI and the function dependency link
register contains 8-bit function number (i.e. `devfn' from software's
perspective).

Signed-off-by: Yu Zhao <yu.zhao@intel.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: handle Virtual Function ATS enabling

The SR-IOV spec requires that the Smallest Translation Unit and
the Invalidate Queue Depth fields in the Virtual Function ATS
capability are hardwired to 0. If a function is a Virtual Function,
then and set its Physical Function's STU before enabling the ATS.

Signed-off-by: Yu Zhao <yu.zhao@intel.com>
Acked-by: Jesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>

PCI: support the ATS capability

The PCIe ATS capability makes the Endpoint be able to request the
DMA address translation from the IOMMU and cache the translation
in the device side, thus alleviate IOMMU pressure and improve the
hardware performance in the I/O virtualization environment.

Signed-off-by: Yu Zhao <yu.zhao@intel.com>
Acked-by: Jesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>

PCI-IOV: fix missing kernel-doc

Fix PCI iov kernel-doc warning:

Warning(drivers/pci/iov.c:638): No description found for parameter 'nr_virtfn'

Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: handle SR-IOV Virtual Function Migration

Add or remove a Virtual Function after receiving a Migrate In or Out
Request.

Reviewed-by: Matthew Wilcox <willy@linux.intel.com>
Signed-off-by: Yu Zhao <yu.zhao@intel.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: add SR-IOV API for Physical Function driver

Add or remove the Virtual Function when the SR-IOV is enabled or
disabled by the device driver. This can happen anytime rather than
only at the device probe stage.

Reviewed-by: Matthew Wilcox <willy@linux.intel.com>
Signed-off-by: Yu Zhao <yu.zhao@intel.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: reserve bus range for SR-IOV device

Reserve the bus number range used by the Virtual Function when
pcibios_assign_all_busses() returns true.

Reviewed-by: Matthew Wilcox <willy@linux.intel.com>
Signed-off-by: Yu Zhao <yu.zhao@intel.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: restore saved SR-IOV state

Restore the volatile registers in the SR-IOV capability after the
D3->D0 transition.

Reviewed-by: Matthew Wilcox <willy@linux.intel.com>
Signed-off-by: Yu Zhao <yu.zhao@intel.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: initialize and release SR-IOV capability

If a device has the SR-IOV capability, initialize it (set the ARI
Capable Hierarchy in the lowest numbered PF if necessary; calculate
the System Page Size for the VF MMIO, probe the VF Offset, Stride
and BARs). A lock for the VF bus allocation is also initialized if
a PF is the lowest numbered PF.

Reviewed-by: Matthew Wilcox <willy@linux.intel.com>
Signed-off-by: Yu Zhao <yu.zhao@intel.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>