PCI/ASPM: Update save_state when configuration changes

Many PCIe device drivers save the configuration state of their device
during probe and restore it when their .slot_reset() hook is called during
PCIe error recovery.

If the ASPM configuration is changed after the driver's probe is called and
before an error event occurs, .slot_reset() restores the ASPM configuration
to what it was at the time of probe, not to what it was just before the
occurrence of the error event. This leads to a mismatch in ASPM
configuration between the device and its upstream device.

Update the saved configuration of the device when the ASPM configuration
changes.

Link: https://lore.kernel.org/r/20240222174436.3565146-1-vidyas@nvidia.com
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
[bhelgaas: commit log, rebase to pci/aspm, rename to
pci_update_aspm_saved_state() since it updates only LNKCTL, update only
ASPMC and CLKREQ_EN in LNKCTL]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Reviewed-by: David E. Box <david.e.box@linux.intel.com>

PCI/ASPM: Disable L1 before configuring L1 Substates

Per PCIe r6.1, sec 5.5.4, L1 must be disabled while setting ASPM L1 PM
Substates enable bits. Previously this was enforced by clearing
PCI_EXP_LNKCTL_ASPMC before calling pci_restore_aspm_l1ss_state().

Move the L1 (and L0s, although that doesn't seem required) disable into
pci_restore_aspm_l1ss_state() itself so it's closer to the code that
depends on it.

Link: https://lore.kernel.org/r/20240223213733.GA115410@bhelgaas
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Save L1 PM Substates Capability for suspend/resume

4ff116d0d5fd ("PCI/ASPM: Save L1 PM Substates Capability for
suspend/resume") restored the L1 PM Substates Capability after resume,
which reduced power consumption by making the ASPM L1.x states work after
resume.

a7152be79b62 ("Revert "PCI/ASPM: Save L1 PM Substates Capability for
suspend/resume"") reverted 4ff116d0d5fd because resume failed on some
systems, so power consumption after resume increased again.

a7152be79b62 mentioned that we restore L1 PM substate configuration even
though ASPM L1 may already be enabled. This is due the fact that the
pci_restore_aspm_l1ss_state() was called before pci_restore_pcie_state().

Save and restore the L1 PM Substates Capability, following PCIe r6.1, sec
5.5.4 more closely by:

1) Do not restore ASPM configuration in pci_restore_pcie_state() but
do that after PCIe capability is restored in pci_restore_aspm_state()
following PCIe r6.1, sec 5.5.4.

2) If BIOS reenables L1SS, particularly L1.2, we need to clear the
enables in the right order, downstream before upstream. Defer
restoring the L1SS config until we are at the downstream component.
Then update the config for both ends of the link in the prescribed
order.

3) Program ASPM L1 PM substate configuration before L1 enables.

4) Program ASPM L1 PM substate enables last, after rest of the fields
in the capability are programmed.

[bhelgaas: commit log, squash L1SS-related patches, do both LNKCTL restores
in pci_restore_pcie_state()]

Link: https://lore.kernel.org/r/20240128233212.1139663-3-david.e.box@linux.intel.com
Link: https://lore.kernel.org/r/20240128233212.1139663-4-david.e.box@linux.intel.com
Link: https://lore.kernel.org/r/20240223205851.114931-5-helgaas@kernel.org
Closes: https://bugzilla.kernel.org/show_bug.cgi?id=217321
Link: https://bugzilla.kernel.org/show_bug.cgi?id=216782
Link: https://bugzilla.kernel.org/show_bug.cgi?id=216877
Co-developed-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Co-developed-by: David E. Box <david.e.box@linux.intel.com>
Reported-by: Koba Ko <koba.ko@canonical.com>
Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Signed-off-by: David E. Box <david.e.box@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Tested-by: Tasev Nikola <tasev.stefanoska@skynet.be> # Asus UX305FA
Cc: Mark Enriquez <enriquezmark36@gmail.com>
Cc: Thomas Witt <kernel@witt.link>
Cc: Werner Sembach <wse@tuxedocomputers.com>
Cc: Vidya Sagar <vidyas@nvidia.com>

PCI/ASPM: Move pci_save_ltr_state() to aspm.c

Even when CONFIG_PCIEASPM is not set, we save and restore the LTR
Capability so that if ASPM L1.2 and LTR were configured by the platform,
ASPM L1.2 will still work after suspend/resume, when that platform
configuration may be lost. See dbbfadf23190 ("PCI/ASPM: Save LTR Capability
for suspend/resume").

Since ASPM L1.2 depends on the LTR Capability, move the save/restore code
to the part of aspm.c that is always compiled regardless of
CONFIG_PCIEASPM. No functional change intended.

Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20240128233212.1139663-5-david.e.box@linux.intel.com
[bhelgaas: commit log, reorder to make this a pure move]
Link: https://lore.kernel.org/r/20240223205851.114931-4-helgaas@kernel.org
Signed-off-by: David E. Box <david.e.box@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Always build aspm.c

Some ASPM-related tasks, such as save and restore of LTR and L1SS
capabilities, still need to be performed when CONFIG_PCIEASPM is not
enabled. To prepare for these changes, wrap the current code in aspm.c
with an #ifdef and always build the file.

Link: https://lore.kernel.org/r/20240128233212.1139663-2-david.e.box@linux.intel.com
[bhelgaas: split build change from function moves]
Link: https://lore.kernel.org/r/20240223205851.114931-3-helgaas@kernel.org
Signed-off-by: David E. Box <david.e.box@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Move pci_configure_ltr() to aspm.c

The Latency Tolerance Reporting (LTR) mechanism supports the ASPM L1.2
state and is only configured when CONFIG_PCIEASPM is set.

Move pci_configure_ltr() and pci_bridge_reconfigure_ltr() into aspm.c since
they only build when CONFIG_PCIEASPM is set. No functional change
intended.

Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20240128233212.1139663-2-david.e.box@linux.intel.com
[bhelgaas: commit log, split build change from function moves]
Link: https://lore.kernel.org/r/20240223205851.114931-2-helgaas@kernel.org
Signed-off-by: David E. Box <david.e.box@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Fix deadlock when enabling ASPM

A last minute revert in 6.7-final introduced a potential deadlock when
enabling ASPM during probe of Qualcomm PCIe controllers as reported by
lockdep:

============================================
WARNING: possible recursive locking detected
6.7.0 #40 Not tainted
--------------------------------------------
kworker/u16:5/90 is trying to acquire lock:
ffffacfa78ced000 (pci_bus_sem){++++}-{3:3}, at: pcie_aspm_pm_state_change+0x58/0xdc

but task is already holding lock:
ffffacfa78ced000 (pci_bus_sem){++++}-{3:3}, at: pci_walk_bus+0x34/0xbc

other info that might help us debug this:
Possible unsafe locking scenario:

CPU0
----
lock(pci_bus_sem);
lock(pci_bus_sem);

*** DEADLOCK ***

Call trace:
print_deadlock_bug+0x25c/0x348
__lock_acquire+0x10a4/0x2064
lock_acquire+0x1e8/0x318
down_read+0x60/0x184
pcie_aspm_pm_state_change+0x58/0xdc
pci_set_full_power_state+0xa8/0x114
pci_set_power_state+0xc4/0x120
qcom_pcie_enable_aspm+0x1c/0x3c [pcie_qcom]
pci_walk_bus+0x64/0xbc
qcom_pcie_host_post_init_2_7_0+0x28/0x34 [pcie_qcom]

The deadlock can easily be reproduced on machines like the Lenovo ThinkPad
X13s by adding a delay to increase the race window during asynchronous
probe where another thread can take a write lock.

Add a new pci_set_power_state_locked() and associated helper functions that
can be called with the PCI bus semaphore held to avoid taking the read lock
twice.

Link: https://lore.kernel.org/r/ZZu0qx2cmn7IwTyQ@hovoldconsulting.com
Link: https://lore.kernel.org/r/20240130100243.11011-1-johan+linaro@kernel.org
Fixes: f93e71aea6c6 ("Revert "PCI/ASPM: Remove pcie_aspm_pm_state_change()"")
Signed-off-by: Johan Hovold <johan+linaro@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: <stable@vger.kernel.org> # 6.7

PCI: Move pci_clear_and_set_dword() helper to PCI header

The clear and set pattern is commonly used for accessing PCI config,
move the helper pci_clear_and_set_dword() from aspm.c into PCI header.
In addition, rename to pci_clear_and_set_config_dword() to retain the
"config" information and match the other accessors.

No functional change intended.

Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Tested-by: Ilkka Koskinen <ilkka@os.amperecomputing.com>
Link: https://lore.kernel.org/r/20231208025652.87192-4-xueshuai@linux.alibaba.com
Signed-off-by: Will Deacon <will@kernel.org>

Revert "PCI/ASPM: Remove pcie_aspm_pm_state_change()"

This reverts commit 08d0cc5f34265d1a1e3031f319f594bd1970976c.

Michael reported that when attempting to resume from suspend to RAM on ASUS
mini PC PN51-BB757MDE1 (DMI model: MINIPC PN51-E1), 08d0cc5f3426
("PCI/ASPM: Remove pcie_aspm_pm_state_change()") caused a 12-second delay
with no output, followed by a reboot.

Workarounds include:

- Reverting 08d0cc5f3426 ("PCI/ASPM: Remove pcie_aspm_pm_state_change()")
- Booting with "pcie_aspm=off"
- Booting with "pcie_aspm.policy=performance"
- "echo 0 | sudo tee /sys/bus/pci/devices/0000:03:00.0/link/l1_aspm"
before suspending
- Connecting a USB flash drive

Link: https://lore.kernel.org/r/20240102232550.1751655-1-helgaas@kernel.org
Fixes: 08d0cc5f3426 ("PCI/ASPM: Remove pcie_aspm_pm_state_change()")
Reported-by: Michael Schaller <michael@5challer.de>
Link: https://lore.kernel.org/r/76c61361-b8b4-435f-a9f1-32b716763d62@5challer.de
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: <stable@vger.kernel.org>

PCI/ASPM: Add pci_disable_link_state_locked() lockdep assert

Add a lockdep assert to pci_disable_link_state_locked() which should only
be called with a pci_bus_sem read lock held.

Link: https://lore.kernel.org/r/20231128081512.19387-7-johan+linaro@kernel.org
Signed-off-by: Johan Hovold <johan+linaro@kernel.org>
[bhelgaas: include function name in subject, commit log]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI/ASPM: Clean up __pci_disable_link_state() 'sem' parameter

Replace the current 'sem' parameter to the __pci_disable_link_state()
helper with a more descriptive 'locked' parameter, which indicates whether
a pci_bus_sem read lock is already held.

Link: https://lore.kernel.org/r/20231128081512.19387-6-johan+linaro@kernel.org
Signed-off-by: Johan Hovold <johan+linaro@kernel.org>
[bhelgaas: include function name in subject, commit log]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

PCI/ASPM: Add pci_enable_link_state_locked()

Add pci_enable_link_state_locked() for enabling link states that can be
used in contexts where a pci_bus_sem read lock is already held (e.g. from
pci_walk_bus()).

This helper will be used to fix a couple of potential deadlocks where
the current helper is called with the lock already held, hence the CC
stable tag.

Fixes: f492edb40b54 ("PCI: vmd: Add quirk to configure PCIe ASPM and LTR")
Link: https://lore.kernel.org/r/20231128081512.19387-2-johan+linaro@kernel.org
Signed-off-by: Johan Hovold <johan+linaro@kernel.org>
[bhelgaas: include helper name in subject, commit log]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Cc: <stable@vger.kernel.org> # 6.3
Cc: Michael Bottini <michael.a.bottini@linux.intel.com>
Cc: David E. Box <david.e.box@linux.intel.com>

PCI: Simplify pcie_capability_clear_and_set_word() to ..._clear_word()

When using pcie_capability_clear_and_set_word() but not actually *setting*
anything, use pcie_capability_clear_word() instead.

Link: https://lore.kernel.org/r/20231026121924.2164-1-ilpo.jarvinen@linux.intel.com
Link: https://lore.kernel.org/r/20231026121924.2164-2-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
[bhelgaas: squash]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Fix L1 substate handling in aspm_attr_store_common()

aspm_attr_store_common(), which handles sysfs control of ASPM, has the same
problem as fb097dcd5a28 ("PCI/ASPM: Disable only ASPM_STATE_L1 when driver
disables L1"): disabling L1 adds only ASPM_L1 (but not any of the L1.x
substates) to the "aspm_disable" mask.

Enabling one substate, e.g., L1.1, via sysfs removes ASPM_L1 from the
disable mask. Since disabling L1 via sysfs doesn't add any of the
substates to the disable mask, enabling L1.1 actually enables *all* the
substates.

In this scenario:

- Write 0 to "l1_aspm" to disable L1
- Write 1 to "l1_1_aspm" to enable L1.1

the intention is to disable L1 and all L1.x substates, then enable just
L1.1, but in fact, *all* L1.x substates are enabled.

Fix this by explicitly disabling all the L1.x substates when disabling L1.

Fixes: 72ea91afbfb0 ("PCI/ASPM: Add sysfs attributes for controlling ASPM link states")
Link: https://lore.kernel.org/r/6ba7dd79-9cfe-4ed0-a002-d99cb842f361@gmail.com
Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
[bhelgaas: commit log]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: stable@vger.kernel.org

Revert "PCI/ASPM: Disable only ASPM_STATE_L1 when driver, disables L1"

This reverts commit fb097dcd5a28c0a2325632405c76a66777a6bed9.

After fb097dcd5a28 ("PCI/ASPM: Disable only ASPM_STATE_L1 when driver
disables L1"), disabling L1 via pci_disable_link_state(PCIE_LINK_STATE_L1),
then enabling one substate, e.g., L1.1, via sysfs actually enables *all*
the substates.

For example, r8169 disables L1 because of hardware issues on a number of
systems, which implicitly disables the L1.1 and L1.2 substates.

On some systems, L1 and L1.1 work fine, but L1.2 causes missed rx packets.
Enabling L1.1 via the sysfs "aspm_l1_1" attribute unexpectedly enables L1.2
as well as L1.1.

After fb097dcd5a28, pci_disable_link_state(PCIE_LINK_STATE_L1) adds only
ASPM_L1 (but not any of the L1.x substates) to the "aspm_disable" mask:

--- Before fb097dcd5a28
+++ After fb097dcd5a28

# r8169 disables L1:
pci_disable_link_state(PCIE_LINK_STATE_L1)
- disable |= ASPM_L1 | ASPM_L1_1 | ASPM_L1_2 | ... # disable L1, L1.x
+ disable |= ASPM_L1 # disable L1 only

# write "1" to sysfs "aspm_l1_1" attribute:
l1_1_aspm
aspm_attr_store_common(state = ASPM_L1_1)
disable &= ~ASPM_L1_1 # enable L1.1
if (state & (ASPM_L1_1 | ...)) # if enabling any substate
disable &= ~ASPM_L1 # enable L1

# final state:
- disable = ASPM_L1_2 | ... # L1, L1.1 enabled; L1.2 disabled
+ disable = 0 # L1, L1.1, L1.2 all enabled

Enabling an L1.x substate removes the substate and L1 from the
"aspm_disable" mask. After fb097dcd5a28, the substates were not added to
the mask when disabling L1, so enabling one substate implicitly enables all
of them.

Revert fb097dcd5a28 so enabling one substate doesn't enable the others.

Link: https://lore.kernel.org/r/c75931ac-7208-4200-9ca1-821629cf5e28@gmail.com
Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
[bhelgaas: work through example in commit log]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: stable@vger.kernel.org

PCI/ASPM: Convert printk() to pr_*() and add include

Convert printk(KERN_INFO ...) to pr_info() and add the correct include
for it.

Link: https://lore.kernel.org/r/20230915155752.84640-8-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Remove unnecessary includes

aspm.c does not use anything from delay.h nor jiffies.h so remove the
includes.

Link: https://lore.kernel.org/r/20230915155752.84640-7-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Use FIELD_MAX() instead of literals

Convert 0x3ff literals in encode_l12_threshold() to
FIELD_MAX(PCI_L1SS_CTL1_LTR_L12_TH_VALUE) that explains the purpose of
the literal.

Link: https://lore.kernel.org/r/20230915155752.84640-6-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Use time constants

Use defined constants to convert between time units.

Link: https://lore.kernel.org/r/20230915155752.84640-5-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Return U32_MAX instead of bit magic construct

Instead of returning a bit obscure -1U, make code's intent of returning
the maximum representable value more obvious by returning U32_MAX.

Link: https://lore.kernel.org/r/20230915155752.84640-4-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Use FIELD_GET/PREP() to access PCIe capability fields

Replace open-coded variants to access PCIe capability registers fields
with FIELD_GET/PREP().

Link: https://lore.kernel.org/r/20230915155752.84640-3-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Use RMW accessors for changing LNKCTL

Don't assume that the device is fully under the control of ASPM and use RMW
capability accessors which do proper locking to avoid losing concurrent
updates to the register values.

If configuration fails in pcie_aspm_configure_common_clock(), the
function attempts to restore the old PCI_EXP_LNKCTL_CCC settings. Store
only the old PCI_EXP_LNKCTL_CCC bit for the relevant devices rather
than the content of the whole LNKCTL registers. It aligns better with
how pcie_lnkctl_clear_and_set() expects its parameter and makes the
code more obvious to understand.

Suggested-by: Lukas Wunner <lukas@wunner.de>
Fixes: 2a42d9dba784 ("PCIe: ASPM: Break out of endless loop waiting for PCI config bits to switch")
Fixes: 7d715a6c1ae5 ("PCI: add PCI Express ASPM support")
Link: https://lore.kernel.org/r/20230717120503.15276-5-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: "Rafael J. Wysocki" <rafael@kernel.org>

PCI/ASPM: Avoid link retraining race

PCIe r6.0.1, sec 7.5.3.7, recommends setting the link control parameters,
then waiting for the Link Training bit to be clear before setting the
Retrain Link bit.

This avoids a race where the LTSSM may not use the updated parameters if it
is already in the midst of link training because of other normal link
activity.

Wait for the Link Training bit to be clear before toggling the Retrain Link
bit to ensure that the LTSSM uses the updated link control parameters.

[bhelgaas: commit log, return 0 (success)/-ETIMEDOUT instead of bool for
both pcie_wait_for_retrain() and the existing pcie_retrain_link()]
Suggested-by: Lukas Wunner <lukas@wunner.de>
Fixes: 7d715a6c1ae5 ("PCI: add PCI Express ASPM support")
Link: https://lore.kernel.org/r/20230502083923.34562-1-ilpo.jarvinen@linux.intel.com
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Lukas Wunner <lukas@wunner.de>
Cc: stable@vger.kernel.org

PCI/ASPM: Factor out pcie_wait_for_retrain()

Factor pcie_wait_for_retrain() out from pcie_retrain_link(). No functional
change intended.

[bhelgaas: split out from
https://lore.kernel.org/r/20230502083923.34562-1-ilpo.jarvinen@linux.intel.com]
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Return 0 or -ETIMEDOUT from pcie_retrain_link()

"pcie_retrain_link" is not a question with a true/false answer, so "bool"
isn't quite the right return type. Return 0 for success or -ETIMEDOUT if
the retrain failed. No functional change intended.

[bhelgaas: based on Ilpo's patch below]
Link: https://lore.kernel.org/r/20230502083923.34562-1-ilpo.jarvinen@linux.intel.com
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Add support for polling DLLLA to pcie_retrain_link()

Let the caller of pcie_retrain_link() specify whether they want to use the
LT bit or the DLLLA bit of the Link Status Register to determine if link
training has completed. It is up to the caller to verify whether the use
of the DLLLA bit, the implementation of which is optional, is valid for the
device requested.

Link: https://lore.kernel.org/r/alpine.DEB.2.21.2306110310540.64925@angie.orcam.me.uk
Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Export pcie_retrain_link() for use outside ASPM

Export pcie_retrain_link() for link retrain needs outside ASPM. Struct
pcie_link_state is local to ASPM and only used by pcie_retrain_link() to
get at the associated PCI device, so change the operand and adjust the lone
call site accordingly. Document the interface. No functional change at
this point.

Link: https://lore.kernel.org/r/alpine.DEB.2.21.2306110229010.64925@angie.orcam.me.uk
Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Export PCIe link retrain timeout

Convert LINK_RETRAIN_TIMEOUT from jiffies to milliseconds, accordingly
rename to PCIE_LINK_RETRAIN_TIMEOUT_MS, and make available via "pci.h" for
the PCI core to use. Use in pcie_wait_for_link_delay().

Link: https://lore.kernel.org/r/alpine.DEB.2.21.2305310030280.59226@angie.orcam.me.uk
Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Factor out waiting for link training to complete

Move code polling for the Link Training bit to clear into a function of its
own.

[bhelgaas: reorder to clean up before exposing to PCI core]
Link: https://lore.kernel.org/r/alpine.DEB.2.21.2306111605060.64925@angie.orcam.me.uk
Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Avoid unnecessary pcie_link_state use

[bhelgaas: extract from expose patch, reorder to clean up before exposing]
Link: https://lore.kernel.org/r/alpine.DEB.2.21.2306110229010.64925@angie.orcam.me.uk
Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Use distinct local vars in pcie_retrain_link()

Use separate local variables to hold the respective values retrieved from
the Link Control Register and the Link Status Register. Improves
readability and it makes it possible for the compiler to detect actual
uninitialised use should this code change in the future.

[bhelgaas: reorder to clean up before exposing to PCI core]
Link: https://lore.kernel.org/r/alpine.DEB.2.21.2306110252260.64925@angie.orcam.me.uk
Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Remove unnecessary ASPM_STATE_L1SS check

Previously aspm_l1ss_init() checked if ASPM_STATE_L1SS is supported before
calling aspm_calc_l12_info(), only for that function to return if
ASPM_STATE_L1_2_MASK is not supported. Simplify the logic by directly
checking for ASPM_STATE_L1_2_MASK.

Link: https://lore.kernel.org/r/20230504111301.229358-6-ajayagarwal@google.com
Signed-off-by: Ajay Agarwal <ajayagarwal@google.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Rename L1.2-specific functions from 'l1ss' to 'l12'

The functions aspm_calc_l1ss_info() and calc_l1ss_pwron() perform
calculations and register programming specific to L1.2 state. Rename them
to aspm_calc_l12_info() and calc_l12_pwron() respectively.

Link: https://lore.kernel.org/r/20230504111301.229358-5-ajayagarwal@google.com
Signed-off-by: Ajay Agarwal <ajayagarwal@google.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Set ASPM_STATE_L1 when driver enables L1.1 or L1.2

Previously pci_enable_link_state(PCIE_LINK_STATE_L1_1) enabled only
ASPM_STATE_L1_1 and did not enable ASPM_STATE_L1. The L1.1 state only
works when L1 is enabled, so enable ASPM_STATE_L1 in addition, and do the
same for L1.2.

The only current caller is vmd_pm_enable_quirk(), which enables *all* ASPM
states, so this should have no functional effect.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20230504111301.229358-4-ajayagarwal@google.com
Signed-off-by: Ajay Agarwal <ajayagarwal@google.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Set only ASPM_STATE_L1 when driver enables L1

Previously pci_enable_link_state(PCIE_LINK_STATE_L1) enabled L1SS as well
as L1. Enable only ASPM_STATE_L1 when the caller enables L1.

The only current caller is vmd_pm_enable_quirk(), which enables *all* ASPM
states, so this should have no functional effect.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20230504111301.229358-3-ajayagarwal@google.com
Signed-off-by: Ajay Agarwal <ajayagarwal@google.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

PCI/ASPM: Disable only ASPM_STATE_L1 when driver disables L1

Previously pci_disable_link_state(PCIE_LINK_STATE_L1) disabled L1SS as well
as L1. This is unnecessary since pcie_config_aspm_link() takes care that
L1SS is not enabled if L1 is disabled.

Disable only ASPM_STATE_L1 when the caller disables L1. No functional
changes intended.

This is consistent with aspm_attr_store_common(), which disables only L1,
not L1SS, when L1 is disabled via the sysfs "l1_aspm" file.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20230504111301.229358-2-ajayagarwal@google.com
Signed-off-by: Ajay Agarwal <ajayagarwal@google.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

PCI/ASPM: Disable ASPM on MFD function removal to avoid use-after-free

Struct pcie_link_state->downstream is a pointer to the pci_dev of function
0. Previously we retained that pointer when removing function 0, and
subsequent ASPM policy changes dereferenced it, resulting in a
use-after-free warning from KASAN, e.g.:

# echo 1 > /sys/bus/pci/devices/0000:03:00.0/remove
# echo powersave > /sys/module/pcie_aspm/parameters/policy

BUG: KASAN: slab-use-after-free in pcie_config_aspm_link+0x42d/0x500
Call Trace:
kasan_report+0xae/0xe0
pcie_config_aspm_link+0x42d/0x500
pcie_aspm_set_policy+0x8e/0x1a0
param_attr_store+0x162/0x2c0
module_attr_store+0x3e/0x80

PCIe spec r6.0, sec 7.5.3.7, recommends that software program the same ASPM
Control value in all functions of multi-function devices.

Disable ASPM and free the pcie_link_state when any child function is
removed so we can discard the dangling pcie_link_state->downstream pointer
and maintain the same ASPM Control configuration for all functions.

[bhelgaas: commit log and comment]
Debugged-by: Zongquan Qin <qinzongquan@sangfor.com.cn>
Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Fixes: b5a0a9b59c81 ("PCI/ASPM: Read and set up L1 substate capabilities")
Link: https://lore.kernel.org/r/20230507034057.20970-1-dinghui@sangfor.com.cn
Signed-off-by: Ding Hui <dinghui@sangfor.com.cn>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Add pci_enable_link_state()

Add pci_enable_link_state() to allow devices to change the default BIOS
configured states. Clears the BIOS default settings then sets the new
states and reconfigures the link under the semaphore. Also add
PCIE_LINK_STATE_ALL macro for convenience for callers that want to enable
all link states.

Link: https://lore.kernel.org/r/20230120031522.2304439-2-david.e.box@linux.intel.com
Signed-off-by: Michael Bottini <michael.a.bottini@linux.intel.com>
Signed-off-by: David E. Box <david.e.box@linux.intel.com>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>

Revert "PCI/ASPM: Refactor L1 PM Substates Control Register programming"

This reverts commit 5e85eba6f50dc288c22083a7e213152bcc4b8208.

Thomas Witt reported that 5e85eba6f50d ("PCI/ASPM: Refactor L1 PM Substates
Control Register programming") broke suspend/resume on a Tuxedo
Infinitybook S 14 v5, which seems to use a Clevo L140CU Mainboard.

The main symptom is:

iwlwifi 0000:02:00.0: Unable to change power state from D3hot to D0, device inaccessible
nvme 0000:03:00.0: Unable to change power state from D3hot to D0, device inaccessible

and the machine is only partially usable after resume. It can't run dmesg
and can't do a clean reboot. This happens on every suspend/resume cycle.

Revert 5e85eba6f50d until we can figure out the root cause.

Fixes: 5e85eba6f50d ("PCI/ASPM: Refactor L1 PM Substates Control Register programming")
Link: https://bugzilla.kernel.org/show_bug.cgi?id=216877
Reported-by: Thomas Witt <kernel@witt.link>
Tested-by: Thomas Witt <kernel@witt.link>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: stable@vger.kernel.org # v6.1+
Cc: Vidya Sagar <vidyas@nvidia.com>

Revert "PCI/ASPM: Save L1 PM Substates Capability for suspend/resume"

This reverts commit 4ff116d0d5fd8a025604b0802d93a2d5f4e465d1.

Tasev Nikola and Mark Enriquez reported that resume from suspend was broken
in v6.1-rc1. Tasev bisected to a47126ec29f5 ("PCI/PTM: Cache PTM
Capability offset"), but we can't figure out how that could be related.

Mark saw the same symptoms and bisected to 4ff116d0d5fd ("PCI/ASPM: Save L1
PM Substates Capability for suspend/resume"), which does have a connection:
it restores L1 Substates configuration while ASPM L1 may be enabled:

pci_restore_state
pci_restore_aspm_l1ss_state
aspm_program_l1ss
pci_write_config_dword(PCI_L1SS_CTL1, ctl1) # L1SS restore
pci_restore_pcie_state
pcie_capability_write_word(PCI_EXP_LNKCTL, cap[i++]) # L1 restore

which is a problem because PCIe r6.0, sec 5.5.4, requires that:

If setting either or both of the enable bits for ASPM L1 PM
Substates, both ports must be configured as described in this
section while ASPM L1 is disabled.

Separately, Thomas Witt reported that 5e85eba6f50d ("PCI/ASPM: Refactor L1
PM Substates Control Register programming") broke suspend/resume, and it
depends on 4ff116d0d5fd.

Revert 4ff116d0d5fd ("PCI/ASPM: Save L1 PM Substates Capability for
suspend/resume") to fix the resume issue and enable revert of 5e85eba6f50d
to fix the issue Thomas reported.

Note that reverting 4ff116d0d5fd means L1 Substates config may be lost on
suspend/resume. As far as we know the system will use more power but will
still *work* correctly.

Fixes: 4ff116d0d5fd ("PCI/ASPM: Save L1 PM Substates Capability for suspend/resume")
Link: https://bugzilla.kernel.org/show_bug.cgi?id=216782
Link: https://bugzilla.kernel.org/show_bug.cgi?id=216877
Reported-by: Tasev Nikola <tasev.stefanoska@skynet.be>
Reported-by: Mark Enriquez <enriquezmark36@gmail.com>
Reported-by: Thomas Witt <kernel@witt.link>
Tested-by: Mark Enriquez <enriquezmark36@gmail.com>
Tested-by: Thomas Witt <kernel@witt.link>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: stable@vger.kernel.org # v6.1+
Cc: Vidya Sagar <vidyas@nvidia.com>

PCI/ASPM: Correct LTR_L1.2_THRESHOLD computation

80d7d7a904fa ("PCI/ASPM: Calculate LTR_L1.2_THRESHOLD from device
characteristics") replaced a fixed value (163840ns) with one computed from
T_POWER_OFF, Common_Mode_Restore_Time, etc., but it encoded the
LTR_L1.2_THRESHOLD value incorrectly.

This is especially a problem for small thresholds, e.g., 63ns fell into the
"threshold_ns < 1024" case and was encoded as 32ns:

LTR_L1.2_THRESHOLD_Scale = 1 (multiplier is 32ns)
LTR_L1.2_THRESHOLD_Value = 63 >> 5 = 1
LTR_L1.2_THRESHOLD = multiplier * value = 32ns * 1 = 32ns

Correct the algorithm to encode all times of 1023ns (0x3ff) or smaller
exactly and larger times conservatively (the encoded threshold is never
smaller than was requested). This reduces the chance of entering L1.2
when the device can't tolerate the exit latency.

Fixes: 80d7d7a904fa ("PCI/ASPM: Calculate LTR_L1.2_THRESHOLD from device characteristics")
Link: https://lore.kernel.org/r/20221005025809.2247547-4-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

PCI/ASPM: Ignore L1 PM Substates if device lacks capability

187f91db8237 ("PCI/ASPM: Remove struct aspm_register_info.l1ss_cap")
inadvertently removed a check for existence of the L1 PM Substates (L1SS)
Capability before reading it.

If there is no L1SS Capability, this means we mistakenly read PCI_COMMAND
and PCI_STATUS (config address 0x04) and interpret that as the PCI_L1SS_CAP
register, so we may incorrectly configure L1SS.

Make sure the L1SS Capability exists before trying to read it.

Fixes: 187f91db8237 ("PCI/ASPM: Remove struct aspm_register_info.l1ss_cap")
Link: https://lore.kernel.org/r/20221005025809.2247547-3-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

PCI/ASPM: Factor out L1 PM Substates configuration

Move L1 PM Substates configuration from pcie_aspm_cap_init() to a new
aspm_l1ss_init() function. No functional change intended.

Link: https://lore.kernel.org/r/20221005025809.2247547-2-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

PCI/ASPM: Save L1 PM Substates Capability for suspend/resume

Previously the L1 PM Substates Control Registers (CTL1 and CTL2) weren't
saved and restored during suspend/resume leading to the L1 PM Substates
configuration being lost post-resume.

Save the L1 PM Substates Control Registers so that the configuration is
retained post-resume.

[bhelgaas: drop pci_is_pcie() testing; we can rely on pci_configure_ltr()
having already done that]
Link: https://lore.kernel.org/r/20220913131822.16557-3-vidyas@nvidia.com
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Refactor L1 PM Substates Control Register programming

Refactor the code to extract the common code to program Control
Registers 1 and 2 of the L1 PM Substates capability to a new function
aspm_program_l1ss() and call it for both parent and child devices.

[bhelgaas: squash in update to preserve fields we're not updating from
https://lore.kernel.org/r/36fa13c5-e0f8-022f-77f7-7908e4df98b8@nvidia.com]
Link: https://lore.kernel.org/r/20220913131822.16557-2-vidyas@nvidia.com
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Unexport pcie_aspm_support_enabled()

pcie_aspm_support_enabled() is used only by the acpi/pci_root.c driver,
which cannot be built as a module, so it does not need to be exported.
Unexport it.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PCI/ASPM: Remove pcie_aspm_pm_state_change()

pcie_aspm_pm_state_change() was introduced at the inception of PCIe ASPM
code, but it can cause some issues. For instance, when ASPM config is
changed via sysfs, those changes won't persist across power state change
because pcie_aspm_pm_state_change() overwrites them.

Also, if the driver restores L1SS [1] after system resume, the restored
state will also be overwritten by pcie_aspm_pm_state_change().

Remove pcie_aspm_pm_state_change(). If there's any hardware that really
needs it to function, a quirk can be used instead.

[1] https://lore.kernel.org/linux-pci/20220201123536.12962-1-vidyas@nvidia.com/
Link: https://lore.kernel.org/r/20220509073639.2048236-1-kai.heng.feng@canonical.com
[bhelgaas: remove additional pcie_aspm_pm_state_change() call in
pci_set_low_power_state(), added by
10aa5377fc8a ("PCI/PM: Split pci_raw_set_power_state()") and moved by
7957d201456f ("PCI/PM: Relocate pci_set_low_power_state()")]
Signed-off-by: Kai-Heng Feng <kai.heng.feng@canonical.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Use DWORD accesses for LTR, L1 SS to avoid erratum

Some devices have an erratum such that they only support DWORD accesses to
some registers. E.g., this Bayhub O2 device ([VID:DID] = [0x1217:0x8621])
only supports DWORD accesses to LTR latency registers and L1 PM substates
control registers:

https://github.com/rajatxjain/public_shared/blob/main/OZ711LV2_appnote.pdf

The L1 PM substate control registers are DWORD sized, and hence their
access in the kernel is already DWORD sized, so we don't need to do
anything for them.

However, the LTR registers being WORD sized, are in need of a solution.
Convert the WORD sized accesses to these registers into DWORD sized
accesses while saving and restoring them.

Link: https://lore.kernel.org/r/20211222012105.3438916-1-rajatja@google.com
Signed-off-by: Rajat Jain <rajatja@google.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Remove struct aspm_latency

The struct aspm_latency is now used only inside pcie_aspm_check_latency().

Replace struct aspm_latency variables with u32 variables and remove struct
aspm_latency.

Link: https://lore.kernel.org/r/20211119193732.12343-5-refactormyself@gmail.com
Signed-off-by: Saheed O. Bolarinwa <refactormyself@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

PCI/ASPM: Stop caching device L0s, L1 acceptable exit latencies

Previously we calculated the device's acceptable L0s and L1 exit latencies
in pcie_aspm_cap_init() and cached them in struct pcie_link_state.

These values are only used in pcie_aspm_check_latency() where they are
compared with the actual exit latencies of the link. This path is used
when removing or changing the D state of the device, so it's relatively low
frequency.

To reduce the amount of per-link data we store, remove the acceptable[]
arrays from struct pcie_link_state and calculate them directly from the
already-cached Device Capabilities register when needed.

[bhelgaas: use endpoint->devcap instead of reading it again]
Link: https://lore.kernel.org/r/20211119193732.12343-4-refactormyself@gmail.com
Signed-off-by: Saheed O. Bolarinwa <refactormyself@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Stop caching link L0s, L1 exit latencies

Previously we calculated the upstream and downstream L0s and L1 exit
latencies of the link in pcie_aspm_cap_init() and cached them in struct
pcie_link_state.latency_*.

These values are only used in pcie_aspm_check_latency() where they are
compared with the acceptable latencies on the link. This path is used when
removing or changing the D state of the device, so it's relatively low
frequency.

To reduce the amount of per-link data we store, remove the latency_*
entries from struct pcie_link_state and calculate the latencies directly
where they are needed.

Link: https://lore.kernel.org/r/20211119193732.12343-3-refactormyself@gmail.com
Signed-off-by: Saheed O. Bolarinwa <refactormyself@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Move pci_function_0() upward

Move pci_function_0() earlier so we can use it from other functions.

Link: https://lore.kernel.org/r/20211119193732.12343-2-refactormyself@gmail.com
Signed-off-by: Bolarinwa O. Saheed <refactormyself@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

PCI: Use kstrtobool() directly, sans strtobool() wrapper

strtobool() is a wrapper around kstrtobool() that has been added for
backward compatibility.

There is no reason to use the old API, so use kstrtobool() directly.

Related: ef951599074b ("lib: move strtobool() to kstrtobool()")

Link: https://lore.kernel.org/r/20210915230127.2495723-3-kw@linux.com
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/sysfs: Use sysfs_emit() and sysfs_emit_at() in "show" functions

The sysfs_emit() and sysfs_emit_at() functions were introduced to make
it less ambiguous which function is preferred when writing to the output
buffer in a device attribute's "show" callback [1].

Convert the PCI sysfs object "show" functions from sprintf(), snprintf()
and scnprintf() to sysfs_emit() and sysfs_emit_at() accordingly, as the
latter is aware of the PAGE_SIZE buffer and correctly returns the number
of bytes written into the buffer.

No functional change intended.

[1] Documentation/filesystems/sysfs.rst

Related commit: ad025f8e46f3 ("PCI/sysfs: Use sysfs_emit() and
sysfs_emit_at() in "show" functions").

Link: https://lore.kernel.org/r/20210603000112.703037-2-kw@linux.com
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>

Revert "PCI/ASPM: Save/restore L1SS Capability for suspend/resume"

This reverts commit 4257f7e008ea394fcecc050f1569c3503b8bcc15.

Kenneth reported that after 4257f7e008ea, he sees a torrent of disk I/O
errors on his NVMe device after suspend/resume until a reboot.

Link: https://lore.kernel.org/linux-pci/20201228040513.GA611645@bjorn-Precision-5520/
Reported-by: Kenneth R. Crudup <kenny@panix.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Save/restore L1SS Capability for suspend/resume

Previously ASPM L1 Substates control registers (CTL1 and CTL2) weren't
saved and restored during suspend/resume leading to L1 Substates
configuration being lost post-resume.

Save the L1 Substates control registers so that the configuration is
retained post-resume.

Link: https://lore.kernel.org/r/20201024190442.871-1-vidyas@nvidia.com
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Remove struct pcie_link_state.l1ss

Previously we computed L1.2 parameters in the enumeration path, saved them
in struct pcie_link_state.l1ss, and programmed them into the devices
whenever we enabled or disabled L1.2 on the link. But these parameters are
constant and don't need to be updated when enabling/disabling L1.2.

Compute and program the L1.2 parameters once during enumeration and remove
the struct pcie_link_state.l1ss member. No functional change intended.

[bhelgaas: rework to program L1.2 parameters during enumeration]
Link: https://lore.kernel.org/r/20201015193039.12585-13-helgaas@kernel.org
Signed-off-by: Saheed O. Bolarinwa <refactormyself@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Remove struct aspm_register_info.l1ss_cap

Previously we stored the L1SS Capabilities value in the struct
aspm_register_info.

We only need this information in one place, so read it there and remove
struct aspm_register_info completely, since it's now empty. No functional
change intended.

[bhelgaas: split up, don't cache l1ss_cap in pci_dev]
Link: https://lore.kernel.org/r/20201015193039.12585-12-helgaas@kernel.org
Signed-off-by: Saheed O. Bolarinwa <refactormyself@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Pass L1SS Capabilities value, not struct aspm_register_info

aspm_calc_l1ss_info() needs only the L1SS Capabilities. It doesn't need
anything else from struct aspm_register_info, so pass only the Capabilities
value. No functional change intended.

Link: https://lore.kernel.org/r/20201015193039.12585-11-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Remove struct aspm_register_info.l1ss_ctl1

Previously we stored the L1SS Control 1 register in the struct
aspm_register_info.

We only need this information in one place, so read it there and remove it
from struct aspm_register_info. No functional change intended.

[bhelgaas: split ctl1/ctl2]
Link: https://lore.kernel.org/r/20201015193039.12585-10-helgaas@kernel.org
Signed-off-by: Saheed O. Bolarinwa <refactormyself@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Remove struct aspm_register_info.l1ss_ctl2 (unused)

We never use the aspm_register_info.l1ss_ctl2 value, so remove it. No
functional change intended.

Link: https://lore.kernel.org/r/20201015193039.12585-9-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Remove struct aspm_register_info.l1ss_cap_ptr

Save the L1 Substates Capability pointer in struct pci_dev. Then we don't
have to keep track of it in the struct aspm_register_info and struct
pcie_link_state, which makes the code easier to read. No functional change
intended.

[bhelgaas: split to a separate patch]
Link: https://lore.kernel.org/r/20201015193039.12585-8-helgaas@kernel.org
Signed-off-by: Saheed O. Bolarinwa <refactormyself@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Remove struct aspm_register_info.latency_encoding

Previously we stored L0s and L1 Exit Latency information from the Link
Capabilities register in the struct aspm_register_info.

We only need these latencies when we already have the Link Capabilities
values, so use those directly and remove the latencies from struct
aspm_register_info. No functional change intended.

Link: https://lore.kernel.org/r/20201015193039.12585-7-helgaas@kernel.org
Signed-off-by: Saheed O. Bolarinwa <refactormyself@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Remove struct aspm_register_info.enabled

Previously we stored the "ASPM Control" bits from the Link Control register
in the struct aspm_register_info.

Read PCI_EXP_LNKCTL directly when needed. This means we can use the
PCI_EXP_LNKCTL_ASPM_* bits directly instead of the similar but different
PCIE_LINK_STATE_* bits. No functional change intended.

[bhelgaas: drop get_aspm_enable() and read LNKCTL once directly]
Link: https://lore.kernel.org/r/20201015193039.12585-6-helgaas@kernel.org
Signed-off-by: Saheed O. Bolarinwa <refactormyself@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Remove struct aspm_register_info.support

Previously we stored the "ASPM Support" field from the Link Capabilities
register in the struct aspm_register_info.

Read the Link Capabilities directly when needed and remove it from the
struct aspm_register_info. No functional change intended.

[bhelgaas: remove pci_dev cached copy since LNKCAP isn't truly read-only,
add PCI_EXP_LNKCAP_ASPM_L0S & PCI_EXP_LNKCAP_ASPM_L1, check them directly
instead of adding aspm_support()]
Link: https://lore.kernel.org/r/20201015193039.12585-5-helgaas@kernel.org
Signed-off-by: Saheed O. Bolarinwa <refactormyself@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Use 'parent' and 'child' for readability

Other users of link->pdev and link->downstream, e.g., pcie_aspm_cap_init(),
pcie_config_aspm_l1ss(), and pcie_config_aspm_link(), use "parent" and
"child" as local names.

Do the same in aspm_calc_l1ss_info() for readability. No functional change
intended.

Link: https://lore.kernel.org/r/20201015193039.12585-4-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Move LTR path check to where it's used

pcie_get_aspm_reg() mostly reads ASPM-related registers, but in some cases
it also updates the value read from PCI_L1SS_CAP based on LTR properties.

Move this update to the point where the value is used to make the code more
readable.

No functional change intended, although previously we could clear
PCI_L1SS_CAP_ASPM_L1_2 for both ends of the link, and now we'll only do it
for the downstream end of a link. This shouldn't matter because we always
test that bit by ANDing l1ss_cap for the upstream and downstream ends.

Link: https://lore.kernel.org/r/20201015193039.12585-3-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Move pci_clear_and_set_dword() earlier

Move pci_clear_and_set_dword() earlier in file to prepare for future patch.
No functional change intended.

Link: https://lore.kernel.org/r/20201015193039.12585-2-helgaas@kernel.org
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Add missing newline in sysfs 'policy'

When I cat ASPM parameter 'policy' by sysfs, it displays as follows. Add a
newline for easy reading. Other sysfs attributes already include a
newline.

[root@localhost ~]# cat /sys/module/pcie_aspm/parameters/policy
[default] performance powersave powersupersave [root@localhost ~]#

Fixes: 7d715a6c1ae5 ("PCI: add PCI Express ASPM support")
Link: https://lore.kernel.org/r/1594972765-10404-1-git-send-email-wangxiongfeng2@huawei.com
Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Allow ASPM on links to PCIe-to-PCI/PCI-X Bridges

7d715a6c1ae5 ("PCI: add PCI Express ASPM support") added the ability for
Linux to enable ASPM, but for some undocumented reason, it didn't enable
ASPM on links where the downstream component is a PCIe-to-PCI/PCI-X Bridge.

Remove this exclusion so we can enable ASPM on these links.

The Dell OptiPlex 7080 mentioned in the bugzilla has a TI XIO2001
PCIe-to-PCI Bridge. Enabling ASPM on the link leading to it allows the
Intel SoC to enter deeper Package C-states, which is a significant power
savings.

[bhelgaas: commit log]
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=207571
Link: https://lore.kernel.org/r/20200505173423.26968-1-kai.heng.feng@canonical.com
Signed-off-by: Kai-Heng Feng <kai.heng.feng@canonical.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com>

PCI/ASPM: Reduce severity of common clock config message

When the UEFI/BIOS or bootloader has not initialised a PCIe device we would
get the following message:

kern.warning: pci 0000:00:01.0: ASPM: current common clock configuration is broken, reconfiguring

"warning" and "broken" are slightly misleading. On an embedded system it is
quite possible for the bootloader to avoid configuring PCIe devices if they
are not needed.

Downgrade the message to pci_info() and change "broken" to "inconsistent"
since we fix up the inconsistency in the code immediately following the
message (and emit an error if that fails).

Link: https://lore.kernel.org/r/20200323035530.11569-1-chris.packham@alliedtelesis.co.nz
Signed-off-by: Chris Packham <chris.packham@alliedtelesis.co.nz>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Clear the correct bits when enabling L1 substates

In pcie_config_aspm_l1ss(), we cleared the wrong bits when enabling ASPM L1
Substates. Instead of the L1.x enable bits (PCI_L1SS_CTL1_L1SS_MASK, 0xf), we
cleared the Link Activation Interrupt Enable bit (PCI_L1SS_CAP_L1_PM_SS,
0x10).

Clear the L1.x enable bits before writing the new L1.x configuration.

[bhelgaas: changelog]
Fixes: aeda9adebab8 ("PCI/ASPM: Configure L1 substate settings")
Link: https://lore.kernel.org/r/1584093227-1292-1-git-send-email-yangyicong@hisilicon.com
Signed-off-by: Yicong Yang <yangyicong@hisilicon.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
CC: stable@vger.kernel.org # v4.11+

PCI/ASPM: Remove PCIEASPM_DEBUG Kconfig option and related code

Previously, CONFIG_PCIEASPM_DEBUG enabled "link_state" and "clk_ctl" sysfs
files that controlled ASPM. We believe these files were rarely if ever
used.

We recently added sysfs ASPM controls that are always present, so the debug
code is no longer needed. Removing this debug code has been discussed for
quite some time, see e.g. [0].

Remove PCIEASPM_DEBUG and the related code.

[0] https://lore.kernel.org/lkml/20180727202619.GD173328@bhelgaas-glaptop.roam.corp.google.com/
Link: https://lore.kernel.org/r/ec935d8e-c084-3938-f1d1-748617596b25@gmail.com
Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Add sysfs attributes for controlling ASPM link states

Add sysfs attributes to Endpoints and other Upstream Ports to control ASPM,
Clock PM, and L1 PM Substates. The new attributes are:

/sys/devices/pci*/.../link/clkpm
/sys/devices/pci*/.../link/l0s_aspm
/sys/devices/pci*/.../link/l1_aspm
/sys/devices/pci*/.../link/l1_1_aspm
/sys/devices/pci*/.../link/l1_2_aspm
/sys/devices/pci*/.../link/l1_1_pcipm
/sys/devices/pci*/.../link/l1_2_pcipm

An attribute is only visible if both ends of the Link leading to the device
support the state. Writing y/1/on to the file enables the state; n/0/off
disables it.

These attributes can be used to tune the power/performance tradeoff for
individual devices.

[bhelgaas: commit log, rename directory to "link"]
Link: https://lore.kernel.org/r/b1c83f8a-9bf6-eac5-82d0-cf5b90128fbf@gmail.com
Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Add pcie_aspm_get_link()

Factor out getting the link associated with a pci_dev and use this helper
where appropriate. In addition this helper will be used in a subsequent
patch of this series.

Link: https://lore.kernel.org/r/19d33770-29de-a9af-4d85-f2b30269d383@gmail.com
Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Allow re-enabling Clock PM

Previously Clock PM could not be re-enabled after being disabled by
pci_disable_link_state() because clkpm_capable was reset. Change this by
adding a clkpm_disable field similar to aspm_disable.

Link: https://lore.kernel.org/r/4e8a66db-7d53-4a66-c26c-f0037ffaa705@gmail.com
Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Add L1 PM substate support to pci_disable_link_state()

Add support for disabling states L1.1 and L1.2 to pci_disable_link_state().
Allow separate control of ASPM and PCI PM L1 substates.

Link: https://lore.kernel.org/r/d81f8036-c236-6463-48e7-ebcdcda85bba@gmail.com
Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Remove pcie_aspm_enabled() unnecessary locking

The lifetime of the link_state structure (bridge->link_state) is not the
same as the lifetime of "bridge" itself. The link_state is allocated by
pcie_aspm_init_link_state() after children of the bridge have been
enumerated, and it is deallocated by pcie_aspm_exit_link_state() after all
children of the bridge (but not the bridge itself) have been removed.

Previously pcie_aspm_enabled() acquired aspm_lock to ensure that
link_state was not deallocated while we're looking at it. But the fact
that the caller of pcie_aspm_enabled() holds a reference to @pdev means
there's always at least one child of the bridge, which means link_state
can't be deallocated.

Remove the unnecessary locking in pcie_aspm_enabled().

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PCI: Get rid of dev->has_secondary_link flag

In some systems, the Device/Port Type in the PCI Express Capabilities
register incorrectly identifies upstream ports as downstream ports.

d0751b98dfa3 ("PCI: Add dev->has_secondary_link to track downstream PCIe
links") addressed this by adding pci_dev.has_secondary_link, which is set
for downstream ports. But this is confusing because pci_pcie_type()
sometimes gives the wrong answer, and it's not obvious that we should use
pci_dev.has_secondary_link instead.

Reduce the confusion by correcting the type of the port itself so that
pci_pcie_type() returns the actual type regardless of what the Device/Port
Type register claims it is. Update the users to call pci_pcie_type() and
pcie_downstream_port() accordingly, and remove pci_dev.has_secondary_link
completely.

Link: https://lore.kernel.org/linux-pci/20190703133953.GK128603@google.com/
Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20190822085553.62697-2-mika.westerberg@linux.intel.com
Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

PCI: Move ASPM declarations to linux/pci.h

Move ASPM definitions and function prototypes from include/linux/pci-aspm.h
to include/linux/pci.h so users only need to include <linux/pci.h>:

PCIE_LINK_STATE_L0S
PCIE_LINK_STATE_L1
PCIE_LINK_STATE_CLKPM
pci_disable_link_state()
pci_disable_link_state_locked()
pcie_no_aspm()

No functional changes intended.

Link: https://lore.kernel.org/r/20190827095620.11213-1-kw@linux.com
Signed-off-by: Krzysztof Wilczynski <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Add pcie_aspm_enabled()

Add a function checking whether or not PCIe ASPM has been enabled for
a given device.

It will be used by the NVMe driver to decide how to handle the
device during system suspend.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Keith Busch <keith.busch@intel.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: let pci_disable_link_state propagate errors

Drivers may rely on pci_disable_link_state() having disabled certain
ASPM link states. If OS can't control ASPM then pci_disable_link_state()
turns into a no-op w/o informing the caller. The driver therefore may
falsely assume the respective ASPM link states are disabled.
Let pci_disable_link_state() propagate errors to the caller, enabling
the caller to react accordingly.

Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

PCI: Rework pcie_retrain_link() wait loop

Transform wait code to a "do {} while (time_before())" loop as recommended
by reviewer. No functional change intended.

Signed-off-by: Stefan Mätje <stefan.maetje@esd.eu>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

PCI: Work around Pericom PCIe-to-PCI bridge Retrain Link erratum

Due to an erratum in some Pericom PCIe-to-PCI bridges in reverse mode
(conventional PCI on primary side, PCIe on downstream side), the Retrain
Link bit needs to be cleared manually to allow the link training to
complete successfully.

If it is not cleared manually, the link training is continuously restarted
and no devices below the PCI-to-PCIe bridge can be accessed. That means
drivers for devices below the bridge will be loaded but won't work and may
even crash because the driver is only reading 0xffff.

See the Pericom Errata Sheet PI7C9X111SLB_errata_rev1.2_102711.pdf for
details. Devices known as affected so far are: PI7C9X110, PI7C9X111SL,
PI7C9X130.

Add a new flag, clear_retrain_link, in struct pci_dev. Quirks for affected
devices set this bit.

Note that pcie_retrain_link() lives in aspm.c because that's currently the
only place we use it, but this erratum is not specific to ASPM, and we may
retrain links for other reasons in the future.

Signed-off-by: Stefan Mätje <stefan.maetje@esd.eu>
[bhelgaas: apply regardless of CONFIG_PCIEASPM]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
CC: stable@vger.kernel.org

PCI: Factor out pcie_retrain_link() function

Factor out pcie_retrain_link() to use for Pericom Retrain Link quirk. No
functional change intended.

Signed-off-by: Stefan Mätje <stefan.maetje@esd.eu>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
CC: stable@vger.kernel.org

PCI/ASPM: Remove unused lists from struct pcie_link_state

ASPM does not make use of the children or link LIST_HEADs declared in
struct pcie_link_state and defined in alloc_pcie_link_state(). Therefore,
remove these lists.

No functional change intended.

Signed-off-by: Frederick Lawler <fred@fredlawl.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Revert "PCI/ASPM: Do not initialize link state when aspm_disabled is set"

This reverts commit 17c91487364fb33797ed84022564ee7544ac4945.

Rafael found that this commit broke the SD card reader in his
Acer Aspire S5. Details of the problem are in the bugzilla below.

Fixes: 17c91487364f ("PCI/ASPM: Do not initialize link state when aspm_disabled is set")
Link: https://bugzilla.kernel.org/show_bug.cgi?id=201801
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Do not initialize link state when aspm_disabled is set

Now that ASPM is configured for *all* PCIe devices at boot, a problem is
seen with systems that set the FADT NO_ASPM bit. This bit indicates that
the OS should not alter the ASPM state, but when
pcie_aspm_init_link_state() runs it only checks for !aspm_support_enabled.
This misses the ACPI_FADT_NO_ASPM case because that is setting
aspm_disabled.

The result is systems may hang at boot after 1302fcf; avoidable if they
boot with pcie_aspm=off (sets !aspm_support_enabled).

Fix this by having aspm_init_link_state() check for either
!aspm_support_enabled or acpm_disabled.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=201001
Fixes: 1302fcf0d03e ("PCI: Configure *all* devices, not just hot-added ones")
Signed-off-by: Patrick Talbert <ptalbert@redhat.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Fix link_state teardown on device removal

Upon removal of the last device on a bus, the link_state of the bridge
leading to that bus is sought to be torn down by having pci_stop_dev()
call pcie_aspm_exit_link_state().

When ASPM was originally introduced by commit 7d715a6c1ae5 ("PCI: add
PCI Express ASPM support"), it determined whether the device being
removed is the last one by calling list_empty() on the bridge's
subordinate devices list. That didn't work because the device is only
removed from the list slightly later in pci_destroy_dev().

Commit 3419c75e15f8 ("PCI: properly clean up ASPM link state on device
remove") attempted to fix it by calling list_is_last(), but that's not
correct either because it checks whether the device is at the *end* of
the list, not whether it's the last one *left* in the list. If the user
removes the device which happens to be at the end of the list via sysfs
but other devices are preceding the device in the list, the link_state
is torn down prematurely.

The real fix is to move the invocation of pcie_aspm_exit_link_state() to
pci_destroy_dev() and reinstate the call to list_empty(). Remove a
duplicate check for dev->bus->self because pcie_aspm_exit_link_state()
already contains an identical check.

Fixes: 7d715a6c1ae5 ("PCI: add PCI Express ASPM support")
Signed-off-by: Lukas Wunner <lukas@wunner.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Shaohua Li <shaohua.li@intel.com>
Cc: stable@vger.kernel.org # v2.6.26

PCI/ASPM: Convert to use sysfs_match_string() helper

The sysfs_match_string() helper returns index of the matching string in an
array. Use it in pcie_aspm_set_policy() to simplify the code.

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
[bhelgaas: squash sysfs_match_string() fix into original patch for issue
Reported-by: Heiner Kallweit <hkallweit1@gmail.com>]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Disable ASPM L1.2 Substate if we don't have LTR

When in the ASPM L1.0 state (but not the PCI-PM L1.0 state), the most
recent LTR value and the LTR_L1.2_THRESHOLD determines whether the link
enters the L1.2 substate.

If we don't have LTR enabled, prevent the use of ASPM L1.2.

PCI-PM L1.2 may still be used because it doesn't depend on
LTR_L1.2_THRESHOLD (see PCIe r4.0, sec 5.5.1).

Tested-by: Srinath Mannam <srinath.mannam@broadcom.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PCI: Tidy comments

Remove pointless comments that tell us the file name, remove blank line
comments, follow multi-line comment conventions. No functional change
intended.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Don't warn if already in common clock mode

Previously we emitted a warning if we tried to configure common clock mode
the link was already configured to common clock mode by the UEFI BIOS.
Bail out silently in that case instead of emitting the warning:

pci 0004:00:00.0: ASPM: Could not configure common clock

Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
[bhelgaas: changelog]
Signed-off-by: Bjorn Helgaas <helgaas@kernel.org>

PCI/ASPM: Declare threshold_ns as u32, not u64

aspm_calc_l1ss_info() computes l1_2_threshold in microseconds as:

l1_2_threshold = 2 + 4 + t_common_mode + t_power_on;

where t_common_mode is at most 255us:

PCI_L1SS_CAP_CM_RESTORE_TIME 0x0000ff00 <-- 8 bits; <256us

and t_power_on is at most 31 * 100us = 3100us:

PCI_L1SS_CAP_P_PWR_ON_VALUE 0x00f80000 <-- 5 bits; <32
PCI_L1SS_CAP_P_PWR_ON_SCALE 0x00030000 <-- *2us, *10us, or *100us

So l1_2_threshold is at most 2 + 4 + 255 + 3100 = 3361, which means
threshold_ns is at most 3361 * 1000 = 3361000, which easily fits in a
u32.

Declare threshold_ns as u32, not u64. This fixes a Coverity warning.

Addresses-Coverity-ID: 1462501
Signed-off-by: Gustavo A. R. Silva <garsilva@embeddedor.com>
[bhelgaas: changelog]
Signed-off-by: Bjorn Helgaas <helgaas@kernel.org>
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>

PCI: Add wrappers for dev_printk()

Add PCI-specific dev_printk() wrappers and use them to simplify the code
slightly. No functional change intended.

Signed-off-by: Frederick Lawler <fred@fredlawl.com>
[bhelgaas: squash into one patch]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Calculate LTR_L1.2_THRESHOLD from device characteristics

Per PCIe r3.1, sec 5.5.1, LTR_L1.2_THRESHOLD determines whether we enter
the L1.2 Link state: if L1.2 is enabled and downstream devices have
reported that they can tolerate latency of at least LTR_L1.2_THRESHOLD, we
must enter L1.2 when CLKREQ# is de-asserted.

The implication is that LTR_L1.2_THRESHOLD is the time required to
transition the Link from L0 to L1.2 and back to L0, and per sec 5.5.3.3.1,
Figures 5-16 and 5-17, it appears that the absolute minimum time for those
transitions would be T(POWER_OFF) + T(L1.2) + T(POWER_ON) + T(COMMONMODE).

Therefore, compute LTR_L1.2_THRESHOLD as:

2us T(POWER_OFF)
+ 4us T(L1.2)
+ T(POWER_ON)
+ T(COMMONMODE)
= LTR_L1.2_THRESHOLD

Previously we set LTR_L1.2_THRESHOLD to a fixed value of 163840ns
(163.84us):

#define LTR_L1_2_THRESHOLD_BITS ((1 << 21) | (1 << 23) | (1 << 30))
((1 << 21) | (1 << 23) | (1 << 30)) = 0x40a00000
LTR_L1.2_THRESHOLD_Value = (0x40a00000 & 0x03ff0000) >> 16 = 0xa0 = 160
LTR_L1.2_THRESHOLD_Scale = (0x40a00000 & 0xe0000000) >> 29 = 0x2 (* 1024ns)
LTR_L1.2_THRESHOLD = 160 * 1024ns = 163840ns

Obviously this doesn't account for the circuit characteristics of different
implementations.

Note that while firmware may enable LTR, Linux itself currently does not
enable LTR. When L1.2 is enabled but LTR is not, LTR_L1.2_THRESHOLD is
ignored and we always enter L1.2 when it is enabled and CLKREQ# is
de-asserted. So this patch should not have any effect unless firmware
enables LTR.

Fixes: f1f0366dd6be ("PCI/ASPM: Calculate and save the L1.2 timing parameters")
Link: https://www.coreboot.org/pipermail/coreboot-gerrit/2015-March/021134.html
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Vidya Sagar <vidyas@nvidia.com>
Cc: Kenji Chen <kenji.chen@intel.com>
Cc: Patrick Georgi <pgeorgi@google.com>
Cc: Rajat Jain <rajatja@google.com>

PCI/ASPM: Add L1 Substates definitions

Add and use #defines for L1 Substate register fields instead of hard-coding
the masks. Also update comments to use names from the spec. No functional
change intended.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Vidya Sagar <vidyas@nvidia.com>

PCI/ASPM: Use correct capability pointer to program LTR_L1.2_THRESHOLD

Previously we programmed the LTR_L1.2_THRESHOLD in the parent (upstream)
device using the capability pointer of the *child* (downstream) device,
which corrupted some random word of the parent's config space.

Use the parent's L1 SS capability pointer to program its
LTR_L1.2_THRESHOLD.

Fixes: aeda9adebab8 ("PCI/ASPM: Configure L1 substate settings")
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Vidya Sagar <vidyas@nvidia.com>
CC: stable@vger.kernel.org # v4.11+
CC: Rajat Jain <rajatja@google.com>

PCI/ASPM: Account for downstream device's Port Common_Mode_Restore_Time

Every Port that supports the L1.2 substate advertises its Port
Common_Mode_Restore_Time, i.e., the time the Port requires to re-establish
common mode when exiting L1.2 (see PCIe r3.1, sec 7.33.2).

Per sec 5.5.3.3.1, when exiting L1.2, the Downstream Port (the device at
the upstream end of the link) must send TS1 training sequences for at least
T(COMMONMODE) after it detects electrical idle exit on the Link. We want
this to be long enough for both ends of the Link, so we should set it to
the maximum of the Port Common_Mode_Restore_Time for the upstream and
downstream components on the Link.

Previously we only looked at the Port Common_Mode_Restore_Time of the
upstream device, so if the downstream device required more time, we didn't
program the upstream device's T(COMMONMODE) correctly.

Fixes: f1f0366dd6be ("PCI/ASPM: Calculate and save the L1.2 timing parameters")
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Vidya Sagar <vidyas@nvidia.com>
Acked-by: Rajat Jain <rajatja@google.com>
CC: stable@vger.kernel.org # v4.11+

License cleanup: add SPDX GPL-2.0 license identifier to files with no license

Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.

By default all files without license information are under the default
license of the kernel, which is GPL version 2.

Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.

This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.

How this work was done:

Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,

Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.

The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.

The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.

Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).

All documentation files were explicitly excluded.

The following heuristics were used to determine which SPDX license
identifiers to apply.

- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.

For non */uapi/* files that summary was:

SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139

and resulted in the first patch in this series.

If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:

SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930

and resulted in the second patch in this series.

- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:

SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1

and that resulted in the third patch in this series.

- when the two scanners agreed on the detected license(s), that became
the concluded license(s).

- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.

- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).

- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.

- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.

In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.

Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.

Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.

In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.

Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct

This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.

These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.

Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

treewide: Fix function prototypes for module_param_call()

Several function prototypes for the set/get functions defined by
module_param_call() have a slightly wrong argument types. This fixes
those in an effort to clean up the calls when running under type-enforced
compiler instrumentation for CFI. This is the result of running the
following semantic patch:

@match_module_param_call_function@
declarer name module_param_call;
identifier _name, _set_func, _get_func;
expression _arg, _mode;
@@

module_param_call(_name, _set_func, _get_func, _arg, _mode);

@fix_set_prototype
depends on match_module_param_call_function@
identifier match_module_param_call_function._set_func;
identifier _val, _param;
type _val_type, _param_type;
@@

int _set_func(
-_val_type _val
+const char * _val
,
-_param_type _param
+const struct kernel_param * _param
) { ... }

@fix_get_prototype
depends on match_module_param_call_function@
identifier match_module_param_call_function._get_func;
identifier _val, _param;
type _val_type, _param_type;
@@

int _get_func(
-_val_type _val
+char * _val
,
-_param_type _param
+const struct kernel_param * _param
) { ... }

Two additional by-hand changes are included for places where the above
Coccinelle script didn't notice them:

drivers/platform/x86/thinkpad_acpi.c
fs/lockd/svc.c

Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Jessica Yu <jeyu@kernel.org>

PCI/ASPM: Deal with missing root ports in link state handling

Even though it is unconventional, some PCIe host implementations omit the
root ports entirely, and simply consist of a host bridge (which is not
modeled as a device in the PCI hierarchy) and a link.

When the downstream device is an endpoint, our current code does not seem
to mind this unusual configuration. However, when PCIe switches are
involved, the ASPM code assumes that any downstream switch port has a
parent, and blindly dereferences the bus->parent->self field of the pci_dev
struct to chain the downstream link state to the link state of the root
port. Given that the root port is missing, the link is not modeled at all,
and nor is the link state, and attempting to access it results in a NULL
pointer dereference and a crash.

Avoid this by allowing the link state chain to terminate at the downstream
port if no root port exists.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Always set link->downstream to avoid NULL dereference on remove

We call pcie_aspm_exit_link_state() when we remove a device. If the device
is the last PCIe function to be removed below a bridge and the bridge has
an ASPM link_state struct, we disable ASPM on the link. Disabling ASPM
requires link->downstream (used in pcie_config_aspm_link()).

We previously set link->downstream in pcie_aspm_cap_init(), but only if the
device was not blacklisted. Removing the blacklisted device caused a NULL
pointer dereference in the pcie_aspm_exit_link_state() ->
pcie_config_aspm_link() path:

# echo 1 > /sys/bus/pci/devices/0000\:0b\:00.0/remove
...
BUG: unable to handle kernel NULL pointer dereference at 0000000000000080
IP: pcie_config_aspm_link+0x5d/0x2b0
Call Trace:
pcie_aspm_exit_link_state+0x75/0x130
pci_stop_bus_device+0xa4/0xb0
pci_stop_and_remove_bus_device_locked+0x1a/0x30
remove_store+0x50/0x70
dev_attr_store+0x18/0x30
sysfs_kf_write+0x44/0x60
kernfs_fop_write+0x10e/0x190
__vfs_write+0x28/0x110
? rcu_read_lock_sched_held+0x5d/0x80
? rcu_sync_lockdep_assert+0x2c/0x60
? __sb_start_write+0x173/0x1a0
? vfs_write+0xb3/0x180
vfs_write+0xc4/0x180
SyS_write+0x49/0xa0
do_syscall_64+0xa6/0x1c0
entry_SYSCALL64_slow_path+0x25/0x25
---[ end trace bd187ee0267df5d9 ]---

To avoid this, set link->downstream in alloc_pcie_link_state(), so every
pcie_link_state structure has a valid link->downstream pointer.

[bhelgaas: changelog]
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Rajat Jain <rajatja@google.com>
CC: stable@vger.kernel.org

PCI/ASPM: Add comment about L1 substate latency

Since the exit latencies for L1 substates are not advertised by a device,
it is not clear in spec how to do a L1 substate exit latency check. We
assume that the L1 exit latencies advertised by a device include L1
substate latencies (and hence do not do any check). If that is not true,
we should do some sort of check here.

(I'm not clear about what that check should like currently. I'd be glad to
take up any suggestions).

Signed-off-by: Rajat Jain <rajatja@google.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Configure L1 substate settings

Configure the L1 substate settings on the upstream and downstream devices,
while taking care of the rules dictated by the PCIe spec.

[bhelgaas: drop "inline"]
Signed-off-by: Rajat Jain <rajatja@google.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Calculate and save the L1.2 timing parameters

Calculate and save the timing parameters that need to be programmed if we
need to enable L1.2 substates later.

We use the same logic (and a constant value for 1 of the parameters) as
used by Intel's coreboot:

https://www.coreboot.org/pipermail/coreboot-gerrit/2015-March/021134.html
https://review.coreboot.org/#/c/8832/

Signed-off-by: Rajat Jain <rajatja@google.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Read and set up L1 substate capabilities

The PCIe spec (r3.1, sec 7.33) says the L1 PM Substates Capability may be
implemented only in function 0.

Read the L1 substate capability structures of upstream and downstream
components of the link and set it up in the device structure.

[bhelgaas: add specific spec reference]
Signed-off-by: Rajat Jain <rajatja@google.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Add support for L1 substates

Add support for ASPM L1 substates. For details about L1 substates, see the
PCIe r3.1 spec, which includes the ECN below in secs 5.5 and 7.33.

Add macros for the 4 new L1 substates, and add a new ASPM "POWER_SUPERSAVE"
policy that can be used to enable L1 substates on a system if desired. The
new policy is in a sense, a superset of the existing POWERSAVE policy. The
4 policies are now:

DEFAULT: Reads and uses whatever ASPM states BIOS enabled
PERFORMANCE: Everything except L0 disabled.
POWERSAVE: L0s and L1 enabled (but not L1 substates)
POWER_SUPERSAVE: L0s + L1 + L1 substates also enabled

[bhelgaas: add PCIe r3.1 spec reference]
Link: https://pcisig.com/sites/default/files/specification_documents/ECN_L1_PM_Substates_with_CLKREQ_31_May_2013_Rev10a.pdf
Signed-off-by: Rajat Jain <rajatja@google.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Handle PCI-to-PCIe bridges as roots of PCIe hierarchies

In a struct pcie_link_state, link->root points to the pcie_link_state of
the root of the PCIe hierarchy. For the topmost link, this points to
itself (link->root = link). For others, we copy the pointer from the
parent (link->root = link->parent->root).

Previously we recognized that Root Ports originated PCIe hierarchies, but
we treated PCI/PCI-X to PCIe Bridges as being in the middle of the
hierarchy, and when we tried to copy the pointer from link->parent->root,
there was no parent, and we dereferenced a NULL pointer:

BUG: unable to handle kernel NULL pointer dereference at 0000000000000090
IP: [<ffffffff9e424350>] pcie_aspm_init_link_state+0x170/0x820

Recognize that PCI/PCI-X to PCIe Bridges originate PCIe hierarchies just
like Root Ports do, so link->root for these devices should also point to
itself.

Fixes: 51ebfc92b72b ("PCI: Enumerate switches below PCI-to-PCIe bridges")
Link: https://bugzilla.kernel.org/show_bug.cgi?id=193411
Link: https://bugzilla.opensuse.org/show_bug.cgi?id=1022181
Tested-by: lists@ssl-mail.com
Tested-by: Jayachandran C. <jnair@caviumnetworks.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
CC: stable@vger.kernel.org # v4.2+

PCI/ASPM: Don't retrain link if ASPM not possible

Some (defective) PCIe devices are not able to reliably do link retraining.

Check to see if ASPM is possible between link partners before configuring
common clocking, and doing the resulting link retraining. If ASPM is not
possible, there is no reason to risk losing access to a device due to an
unnecessary link retraining.

Signed-off-by: David Daney <david.daney@cavium.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Use permission-specific DEVICE_ATTR variants

Use DEVICE_ATTR_RW for read-write attributes. This simplifies the source
code, improves readability, and reduces the chance of inconsistencies.

The semantic patch that makes this change is as follows:
(http://coccinelle.lip6.fr/)

// <smpl>
@rw@
declarer name DEVICE_ATTR;
identifier x,x_show,x_store;
@@

DEVICE_ATTR(x, \(0644\|S_IRUGO|S_IWUSR\), x_show, x_store);

@script:ocaml@
x << rw.x;
x_show << rw.x_show;
x_store << rw.x_store;
@@

if not (x^"_show" = x_show && x^"_store" = x_store)
then Coccilib.include_match false

@@
declarer name DEVICE_ATTR_RW;
identifier rw.x,rw.x_show,rw.x_store;
@@

- DEVICE_ATTR(x, \(0644\|S_IRUGO|S_IWUSR\), x_show, x_store);
+ DEVICE_ATTR_RW(x);
// </smpl>

Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Remove redundant check of pcie_set_clkpm

Without supporting clock PM capable, if we want to disable clkpm, we don't
need this extra check as it must already be zero for the enable argument.
And it's the same for enabling clkpm here. So let's remove this check.

Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Make sysfs link_state_store() consistent with link_state_show()

If CONFIG_PCIEASPM_DEBUG is set, then PCI devices have a link_state
attribute. Reading that attribute shows the state as a bit mask: 1
means L0S upstream, 2 means L0S downstream, and 4 means L1.

Oddly, writing to link_state is inconsistent and gets translated, leading
to mysterious results in which the value you store isn't comparable the
value you load back out.

Fix it by making link_state_store() match link_state_show().

[bhelgaas: Check "aspm_disabled" *before* validating input. When
"aspm_disabled" is set, this changes the error for invalid input from
-EINVAL to -EPERM.]

Signed-off-by: Andy Lutomirski <luto@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Simplify Clock Power Management setting

Update the Link Control Enable Clock Power Management bit the same
way we update the ASPM Control bits, with a single call of
pcie_capability_clear_and_set_word().

No functional change; this just makes both paths use the same style.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Use dev->has_secondary_link to find downstream links

We allocate pcie_link_state for the component at the upstream end of a
Link. Previously we did this by allocating pcie_link_state for Root Ports
and Downstream Ports. This works fine for the typical topology:

00:1c.0 Root Port [bridge to bus 02]
02:00.0 Upstream Port [bridge to bus 03]
03:00.0 Downstream Port [bridge to bus 04]
04:00.0 Endpoint or Switch Port

However, it is possible to have a Root Port connected to a Downstream Port
instead of an Upstream Port, as in Robert White's ATCA system:

00:1c.0 Root Port [bridge to bus 02]
02:00.0 Downstream Port [bridge to bus 03]
03:01.0 Downstream Port [bridge to bus 04]
04:00.0 Endpoint or Switch Port

In this topology, we wrongly allocated pcie_link_state for the 02:00.0
Downstream Port, which is actually the *downstream* end of a link. This
led to the following NULL pointer dereference when we tried to connect this
link into the tree of links starting at the 00:1c.0 Root Port:

BUG: unable to handle kernel NULL pointer dereference at 0000000000000088
IP: [<ffffffff81550324>] pcie_aspm_init_link_state+0x744/0x850
Hardware name: Kontron B3001/B3001, BIOS 4.6.3 08/07/2012
Call Trace:
[<ffffffff8153b865>] pci_scan_slot+0xd5/0x120
[<ffffffff8153ca1d>] pci_scan_child_bus+0x2d/0xd0
...

Instead of relying on the component type to identify the upstream end of a
link, use the "dev->has_secondary_link" field.

This means it's now possible for an Upstream Port to have a link on its
secondary side, so alloc_pcie_link_state() needs to connect links
originating from both Upstream and Downstream Ports into the tree.

[bhelgaas: changelog, add comment]
Link: https://bugzilla.kernel.org/show_bug.cgi?id=94361
Link: http://lkml.kernel.org/r/54EB81B2.4050904@pobox.com
Reported-by: Robert White <rwhite@pobox.com>
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Remove redundant PCIe port type checking

We decide in alloc_pcie_link_state() whether to allocate a pcie_link_state
for a device. After that, it's sufficient to check pdev->link_state. We
don't need to check the PCIe port type again.

Remove the redundant PCIe port type checking.

[bhelgaas: changelog]
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Drop __pci_disable_link_state() useless "force" parameter

After 387d37577fdd ("PCI: Don't clear ASPM bits when the FADT declares it's
unsupported"), the "force" parameter to __pci_disable_link_state() is
always "false".

Remove the "force" parameter and assume it's always false.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Don't clear ASPM bits when the FADT declares it's unsupported

Communications with a hardware vendor confirm that the expected behaviour
on systems that set the FADT ASPM disable bit but which still grant full
PCIe control is for the OS to leave any BIOS configuration intact and
refuse to touch the ASPM bits. This mimics the behaviour of Windows.

Signed-off-by: Matthew Garrett <mjg59@coreos.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Use standard parsing functions for sysfs setters

The functions link_state_store() and clk_ctl_store() had just subtracted
ASCII '0' from input which could lead to undesired results. Instead, use
Linux string functions to safely parse input.

[bhelgaas: check kstrtouint() return value]
Signed-off-by: Chris J Arges <chris.j.arges@canonical.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Remove unused pcie_aspm_enabled()

My philosophy is unused code is dead code. And dead code is subject to bit
rot and is a likely source of bugs. Use it or lose it.

This reverts part of 3e1b16002af2 ("ACPI/PCI: PCIe ASPM _OSC support
capabilities called when root bridge added"), removing this interface:

pcie_aspm_enabled()

[bhelgaas: split to separate patch]
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
CC: Andrew Patterson <andrew.patterson@hp.com>

PCI: Fix whitespace, capitalization, and spelling errors

Fix whitespace, capitalization, and spelling errors. No functional change.
I know "busses" is not an error, but "buses" was more common, so I used it
consistently.

Signed-off-by: Marta Rybczynska <rybczynska@gmail.com> (pci_reset_bridge_secondary_bus())
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PCI: Fix kerneldoc for pci_disable_link_state()

Fix kerneldoc for pci_disable_link_state().

[bhelgaas: expand comment, fix typos]
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Warn when driver asks to disable ASPM, but we can't do it

Some devices have hardware problems related to using ASPM. Drivers for
these devices use pci_disable_link_state() to prevent their device from
entering L0s or L1. But on platforms where the OS doesn't have permission
to manage ASPM, pci_disable_link_state() doesn't actually disable ASPM.

Windows has a similar mechanism ("PciASPMOptOut"), and when the OS doesn't
have control of ASPM, it doesn't actually disable ASPM either.

This patch just adds a warning in dmesg about the fact that
pci_disable_link_state() is doing nothing.

Reported-by: Emmanuel Grumbach <egrumbach@gmail.com>
Reference: https://lkml.kernel.org/r/CANUX_P3F5YhbZX3WGU-j1AGpbXb_T9Bis2ErhvKkFMtDvzatVQ@mail.gmail.com
Reference: https://bugzilla.kernel.org/show_bug.cgi?id=57331
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Don't touch ASPM if forcibly disabled

Don't allocate and track PCIe ASPM state when "pcie_aspm=off" is specified
on the kernel command line.

Based-on-patch-from: Matthew Garrett <mjg59@srcf.ucam.org>
Signed-off-by: Joe Lawrence <joe.lawrence@stratus.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: David Bulkow <david.bulkow@stratus.com>
Acked-by: Myron Stowe <myron.stowe@redhat.com>

PCI/ASPM: Deallocate upstream link state even if device is not PCIe

On PCI bus hotplug removal, pcie_aspm_exit_link_state() can potentially
skip parent devices that have link_state allocated. Instead of exiting
early if a given device is not PCIe, check whether or not the device's
parent has link_state allocated. This enables pcie_aspm_exit_link_state()
to properly clean up parent link_state when the last function in a slot
might not be PCIe.

Reported-by: Joe Lawrence <joe.lawrence@stratus.com>
Tested-by: Joe Lawrence <joe.lawrence@stratus.com>
Signed-off-by: Myron Stowe <myron.stowe@redhat.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Allow pcie_aspm=force even when FADT indicates it is unsupported

Right now using pcie_aspm=force will not enable ASPM if the FADT indicates
ASPM is unsupported. However, the semantics of force should probably allow
for this, especially as they did before 3c076351c4 ("PCI: Rework ASPM
disable code")

This patch just skips the clearing of any ASPM setup that the firmware has
carried out on this bus if pcie_aspm=force is being used.

Reference: http://bugs.launchpad.net/bugs/962038
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
CC: stable@vger.kernel.org

PCI: Add standard PCIe Capability Link ASPM field names

Add standard #defines for ASPM fields in PCI Express Link Capability and
Link Control registers.

Previously we used PCIE_LINK_STATE_L0S and PCIE_LINK_STATE_L1 directly, but
these are defined for the Linux ASPM interfaces, e.g.,
pci_disable_link_state(), and only coincidentally match the actual register
bits. PCIE_LINK_STATE_CLKPM, also part of that interface, does not match
the register bit.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Acked-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>

PCI: Convert dev_printk(KERN_<LEVEL> to dev_<level>(

dev_<level> calls take less code than dev_printk(KERN_<LEVEL>
and reducing object size is good.
Coalesce formats for easier grep.

Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI/ASPM: Use PCI Express Capability accessors

Use PCI Express Capability access functions to simplify PCIe ASPM.

Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

PCI: Introduce pci_pcie_type(dev) to replace pci_dev->pcie_type

Introduce an inline function pci_pcie_type(dev) to extract PCIe
device type from pci_dev->pcie_flags_reg field, and prepare for
removing pci_dev->pcie_type.

Signed-off-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

ASPM: Fix pcie devices with non-pcie children

Since 3.2.12 and 3.3, some systems are failing to boot with a BUG_ON.
Some other systems using the pata_jmicron driver fail to boot because no
disks are detected. Passing pcie_aspm=force on the kernel command line
works around it.

The cause: commit 4949be16822e ("PCI: ignore pre-1.1 ASPM quirking when
ASPM is disabled") changed the behaviour of pcie_aspm_sanity_check() to
always return 0 if aspm is disabled, in order to avoid cases where we
changed ASPM state on pre-PCIe 1.1 devices.

This skipped the secondary function of pcie_aspm_sanity_check which was
to avoid us enabling ASPM on devices that had non-PCIe children, causing
trouble later on. Move the aspm_disabled check so we continue to honour
that scenario.

Addresses https://bugzilla.kernel.org/show_bug.cgi?id=42979 and
http://bugs.debian.org/665420

Reported-by: Romain Francoise <romain@orebokech.com> # kernel panic
Reported-by: Chris Holland <bandidoirlandes@gmail.com> # disk detection trouble
Signed-off-by: Matthew Garrett <mjg@redhat.com>
Cc: stable@vger.kernel.org
Tested-by: Hatem Masmoudi <hatem.masmoudi@gmail.com> # Dell Latitude E5520
Tested-by: janek <jan0x6c@gmail.com> # pata_jmicron with JMB362/JMB363
[jn: with more symptoms in log message]
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

PCI: ignore pre-1.1 ASPM quirking when ASPM is disabled

Right now we won't touch ASPM state if ASPM is disabled, except in the case
where we find a device that appears to be too old to reliably support ASPM.
Right now we'll clear it in that case, which is almost certainly the wrong
thing to do. The easiest way around this is just to disable the blacklisting
when ASPM is disabled.

Signed-off-by: Matthew Garrett <mjg@redhat.com>
Cc: stable@vger.kernel.org
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: pcie: Add support for setting default ASPM policy

Distributions may wish to provide different defaults for PCIE ASPM
depending on their target audience. Provide a configuration option for
choosing the default policy.

Signed-off-by: Matthew Garrett <mjg@redhat.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: Rework ASPM disable code

Right now we forcibly clear ASPM state on all devices if the BIOS indicates
that the feature isn't supported. Based on the Microsoft presentation
"PCI Express In Depth for Windows Vista and Beyond", I'm starting to think
that this may be an error. The implication is that unless the platform
grants full control via _OSC, Windows will not touch any PCIe features -
including ASPM. In that case clearing ASPM state would be an error unless
the platform has granted us that control.

This patch reworks the ASPM disabling code such that the actual clearing
of state is triggered by a successful handoff of PCIe control to the OS.
The general ASPM code undergoes some changes in order to ensure that the
ability to clear the bits isn't overridden by ASPM having already been
disabled. Further, this theoretically now allows for situations where
only a subset of PCIe roots hand over control, leaving the others in the
BIOS state.

It's difficult to know for sure that this is the right thing to do -
there's zero public documentation on the interaction between all of these
components. But enough vendors enable ASPM on platforms and then set this
bit that it seems likely that they're expecting the OS to leave them alone.

Measured to save around 5W on an idle Thinkpad X220.

Signed-off-by: Matthew Garrett <mjg@redhat.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: Rework ASPM disable code

Right now we forcibly clear ASPM state on all devices if the BIOS indicates
that the feature isn't supported. Based on the Microsoft presentation
"PCI Express In Depth for Windows Vista and Beyond", I'm starting to think
that this may be an error. The implication is that unless the platform
grants full control via _OSC, Windows will not touch any PCIe features -
including ASPM. In that case clearing ASPM state would be an error unless
the platform has granted us that control.

This patch reworks the ASPM disabling code such that the actual clearing
of state is triggered by a successful handoff of PCIe control to the OS.
The general ASPM code undergoes some changes in order to ensure that the
ability to clear the bits isn't overridden by ASPM having already been
disabled. Further, this theoretically now allows for situations where
only a subset of PCIe roots hand over control, leaving the others in the
BIOS state.

It's difficult to know for sure that this is the right thing to do -
there's zero public documentation on the interaction between all of these
components. But enough vendors enable ASPM on platforms and then set this
bit that it seems likely that they're expecting the OS to leave them alone.

Measured to save around 5W on an idle Thinkpad X220.

Signed-off-by: Matthew Garrett <mjg@redhat.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCIe ASPM: forcedly -> forcibly

Merriam-Webster tells us that the word exists. However ...

* Google suggests `forcibly' because it doesn't recognize `forcedly'.
* Google lists 494 thousand results for `forcedly'.
* Google lists 13.7 million results for `forcibly'.
* Linus's repo contains 1 occurrence of `forcedly' ( 0 after my change).
* Linus's repo contains 60 occurrences of `forcibly' (61 after my change).

Signed-off-by: Michael Witten <mfwitten@gmail.com>
Signed-off-by: Jiri Kosina <jkosina@suse.cz>

PCI/e1000e: Add and use pci_disable_link_state_locked()

Need to use it in _e1000e_disable_aspm. This routine is used for error
recovery, where the pci_bus_sem is already held, and we don't want
pci_disable_link_state to try to take it again. So add a locked variant
for use in cases like this.

Found lock up:

[ 2374.654557] kworker/32:1 D ffff881027f6b0f0 0 6075 2 0x00000000
[ 2374.654816] ffff88503f099a68 0000000000000046 ffff88503f098000 0000000000004000
[ 2374.654837] 00000000001d1ec0 ffff88503f099fd8 00000000001d1ec0 ffff88503f099fd8
[ 2374.654860] 0000000000004000 00000000001d1ec0 ffff88503dcc8000 ffff88503f090000
[ 2374.654880] Call Trace:
[ 2374.654898] [<ffffffff810b1302>] ? __lock_acquired+0x3a/0x224
[ 2374.654914] [<ffffffff81c2b59c>] ? _raw_spin_unlock_irq+0x30/0x36
[ 2374.654925] [<ffffffff810b069d>] ? trace_hardirqs_on_caller+0x1f/0x178
[ 2374.654936] [<ffffffff81c2ab24>] rwsem_down_failed_common+0xd3/0x103
[ 2374.654945] [<ffffffff810b158f>] ? __lock_contended+0x3a/0x2a2
[ 2374.654955] [<ffffffff81c2ab7b>] rwsem_down_read_failed+0x12/0x14
[ 2374.654967] [<ffffffff813371e4>] call_rwsem_down_read_failed+0x14/0x30
[ 2374.654981] [<ffffffff8135df20>] ? pci_disable_link_state+0x5f/0xf5
[ 2374.654990] [<ffffffff81c2a0e6>] ? down_read+0x7e/0x91
[ 2374.654999] [<ffffffff8135df20>] ? pci_disable_link_state+0x5f/0xf5
[ 2374.655008] [<ffffffff8135df20>] pci_disable_link_state+0x5f/0xf5
[ 2374.655024] [<ffffffff81661796>] e1000e_disable_aspm+0x55/0x5a
[ 2374.655037] [<ffffffff816677eb>] e1000_io_slot_reset+0x59/0xea
[ 2374.655048] [<ffffffff8135fe0d>] ? report_mmio_enabled+0x5d/0x5d
[ 2374.655057] [<ffffffff8135fe3b>] report_slot_reset+0x2e/0x5d
[ 2374.655072] [<ffffffff8135369e>] pci_walk_bus+0x8a/0xb7
[ 2374.655081] [<ffffffff8135fe0d>] ? report_mmio_enabled+0x5d/0x5d
[ 2374.655091] [<ffffffff813603be>] broadcast_error_message+0xa4/0xb2
[ 2374.655101] [<ffffffff81352c71>] ? pci_bus_read_config_dword+0x72/0x80
[ 2374.655110] [<ffffffff813606df>] do_recovery+0x9e/0xf9
[ 2374.655120] [<ffffffff81360786>] handle_error_source+0x4c/0x51
[ 2374.655129] [<ffffffff81360974>] aer_isr_one_error+0x1e9/0x21a
[ 2374.655138] [<ffffffff81360a6c>] aer_isr+0xc7/0xcc
[ 2374.655147] [<ffffffff813609a5>] ? aer_isr_one_error+0x21a/0x21a
[ 2374.655159] [<ffffffff81096d9f>] process_one_work+0x237/0x3ec
[ 2374.655168] [<ffffffff81096d10>] ? process_one_work+0x1a8/0x3ec
[ 2374.655178] [<ffffffff8109728d>] worker_thread+0x17c/0x240
[ 2374.655186] [<ffffffff810b0803>] ? trace_hardirqs_on+0xd/0xf
[ 2374.655196] [<ffffffff81097111>] ? manage_workers+0xab/0xab
[ 2374.655209] [<ffffffff8109c8ed>] kthread+0xa0/0xa8
[ 2374.655223] [<ffffffff81c332d4>] kernel_thread_helper+0x4/0x10
[ 2374.655232] [<ffffffff81c2b880>] ? retint_restore_args+0xe/0xe
[ 2374.655243] [<ffffffff8109c84d>] ? __init_kthread_worker+0x5b/0x5b
[ 2374.655252] [<ffffffff81c332d0>] ? gs_change+0xb/0xb

when aer happens,
pci_walk_bus already have down_read(&pci_bus_sem)...
then report_slot_reset
==> e1000_io_slot_reset
==> e1000e_disable_aspm
==> pci_disable_link_state...

We can not use pci_disable_link_state, and it will try to hold pci_bus_sem again.

Try to have __pci_disable_link_state that will not need to hold pci_bus_sem.

-v2: change name to pci_disable_link_state_locked() according to Jesse.

[jbarnes: make sure new function is exported for modules]

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: Enable ASPM state clearing regardless of policy

Commit 2f671e2d allowed us to clear ASPM state when the FADT
tells us it isn't supported, but we don't put this into effect
if the aspm_policy is set to POLICY_POWERSAVE. Enable the
state to be cleared regardless of policy.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: Changing ASPM policy, via /sys, to POWERSAVE could cause NMIs

v3 -> v2: Modified the text that describes the problem
v2 -> v1: Returned -EPERM
v1 : http://marc.info/?l=linux-pci&m=130013194803727&w=2

For servers whose hardware cannot handle ASPM the BIOS ought to set the
FADT bit shown below:
In Sec 5.2.9.3 (IA-PC Boot Arch. Flags) of ACPI4.0a Specification, please
see Table 5-11:
PCIe ASPM Controls: If set, indicates to OSPM that it must not enable
OPSM ASPM control on this platform.

However there are shipping servers whose BIOS did not set this bit. (An
example is the HP ProLiant DL385 G6. A Maintenance BIOS will fix that).
For such servers even if a call is made via pci_no_aspm(), based on _OSC
support in the BIOS, it may be too late because the ASPM code may have
already allocated and filled its "link_list".

So if a user sets the ASPM "policy" to "powersave" via /sys then
pcie_aspm_set_policy() will run through the "link_list" and re-configure
ASPM policy on devices that advertise ASPM L0s/L1 capability:
# echo powersave > /sys/module/pcie_aspm/parameters/policy
# cat /sys/module/pcie_aspm/parameters/policy
default performance [powersave]

That can cause NMIs since the hardware doesn't play well with ASPM:
[ 1651.906015] NMI: PCI system error (SERR) for reason b1 on CPU 0.
[ 1651.906015] Dazed and confused, but trying to continue

Ideally, the BIOS should have set that FADT bit in the first place but we
could be more robust - especially given the fact that Windows doesn't
cause NMIs in the above scenario.

There should be a sanity check to not allow a user to modify ASPM policy
when aspm_disabled is set.

Signed-off-by: Naga Chumbalkar <nagananda.chumbalkar@hp.com>
Acked-by: Rafael J. Wysocki <rjw@sisk.pl>
Cc: Matthew Garrett <mjg59@srcf.ucam.org>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: PCIe links may not get configured for ASPM under POWERSAVE mode

v3 -> v2: Moved ASPM enabling logic to pci_set_power_state()
v2 -> v1: Preserved the logic in pci_raw_set_power_state()
: Added ASPM enabling logic after scanning Root Bridge
: http://marc.info/?l=linux-pci&m=130046996216391&w=2
v1 : http://marc.info/?l=linux-pci&m=130013164703283&w=2

The assumption made in commit 41cd766b065970ff6f6c89dd1cf55fa706c84a3d
(PCI: Don't enable aspm before drivers have had a chance to veto it) that
pci_enable_device() will result in re-configuring ASPM when aspm_policy is
POWERSAVE is no longer valid. This is due to commit
97c145f7c87453cec90e91238fba5fe2c1561b32 (PCI: read current power state
at enable time) which resets dev->current_state to D0. Due to this the
call to pcie_aspm_pm_state_change() is never made. Note the equality check
(below) that returns early:
./drivers/pci/pci.c: pci_raw_set_pci_power_state()
546 /* Check if we're already there */
547 if (dev->current_state == state)
548 return 0;

Therefore OSPM never configures the PCIe links for ASPM to turn them "on".

Fix it by configuring ASPM from the pci_enable_device() code path. This
also allows a driver such as the e1000e networking driver a chance to
disable ASPM (L0s, L1), if need be, prior to enabling the device. A
driver may perform this action if the device is known to mis-behave
wrt ASPM.

Signed-off-by: Naga Chumbalkar <nagananda.chumbalkar@hp.com>
Acked-by: Rafael J. Wysocki <rjw@sisk.pl>
Cc: Matthew Garrett <mjg59@srcf.ucam.org>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI/ACPI: Report ASPM support to BIOS if not disabled from command line

We need to distinguish the situation in which ASPM support is
disabled from the command line or through .config from the situation
in which it is disabled, because the hardware or BIOS can't handle
it. In the former case we should not report ASPM support to the BIOS
through ACPI _OSC, but in the latter case we should do that.

Introduce pcie_aspm_support_enabled() that can be used by
acpi_pci_root_add() to determine whether or not it should report ASPM
support to the BIOS through _OSC.

Cc: stable@kernel.org
References: https://bugzilla.kernel.org/show_bug.cgi?id=29722
References: https://bugzilla.kernel.org/show_bug.cgi?id=20232
Reported-and-tested-by: Ortwin Glück <odi@odi.ch>
Reviewed-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Tested-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: Disable ASPM if BIOS asks us to

We currently refuse to touch the ASPM registers if the BIOS tells us that
ASPM isn't supported. This can cause problems if the BIOS has (for any
reason) enabled ASPM on some devices anyway. Change the code such that we
explicitly clear ASPM if the FADT indicates that ASPM isn't supported,
and make sure we tidy up appropriately on device removal in order to deal
with the hotplug case. If ASPM is disabled because the BIOS doesn't hand
over control then we won't touch the registers.

Signed-off-by: Matthew Garrett <mjg@redhat.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: Don't enable aspm before drivers have had a chance to veto it

The aspm code will currently set the configured aspm policy before drivers
have had an opportunity to indicate that their hardware doesn't support it.
Unfortunately, putting some hardware in L0 or L1 can result in the hardware
no longer responding to any requests, even after aspm is disabled. It makes
more sense to leave aspm policy at the BIOS defaults at initial setup time,
reconfiguring it after pci_enable_device() is called. This allows the
driver to blacklist individual devices beforehand.

Reviewed-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Matthew Garrett <mjg@redhat.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: change PCI nomenclature in drivers/pci/ (comment changes)

Changing occurrences of variants of PCI-X and PCIe to the PCI-SIG
terms listed in the "Trademark and Logo Usage Guidelines".
http://www.pcisig.com/developers/procedures/logos/Trademark_and_Logo_Usage_Guidelines_updated_112206.pdf

Patch is limited to drivers/pci/ and changes concern comments only.

Signed-off-by: Stefan Assmann <sassmann@redhat.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: fix BUG_ON triggered by logical PCIe root port removal

This problem happened when removing PCIe root port using PCI logical
hotplug operation.

The immediate cause of this problem is that the pointer to invalid
data structure is passed to pcie_update_aspm_capable() by
pcie_aspm_exit_link_state(). When pcie_aspm_exit_link_state() received
a pointer to root port link, it unconfigures the root port link and
frees its data structure at first. At this point, there are not links
to configure under the root port and the data structure for root port
link is already freed. So pcie_aspm_exit_link_state() must not call
pcie_update_aspm_capable() and pcie_config_aspm_path().

This patch fixes the problem by changing pcie_aspm_exit_link_state()
not to call pcie_update_aspm_capable() and pcie_config_aspm_path() if
the specified link is root port link.

------------[ cut here ]------------
kernel BUG at drivers/pci/pcie/aspm.c:606!
invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
last sysfs file: /sys/devices/pci0000:40/0000:40:13.0/remove
CPU 1
Modules linked in: shpchp
Pid: 9345, comm: sysfsd Not tainted 2.6.32-rc5 #98 ProLiant DL785 G6
RIP: 0010:[<ffffffff811df69b>] [<ffffffff811df69b>] pcie_update_aspm_capable+0x15/0xbe
RSP: 0018:ffff88082a2f5ca0 EFLAGS: 00010202
RAX: 0000000000000e77 RBX: ffff88182cc3e000 RCX: ffff88082a33d006
RDX: 0000000000000001 RSI: ffffffff811dff4a RDI: ffff88182cc3e000
RBP: ffff88082a2f5cc0 R08: ffff88182cc3e000 R09: 0000000000000000
R10: ffff88182fc00180 R11: ffff88182fc00198 R12: ffff88182cc3e000
R13: 0000000000000000 R14: ffff88182cc3e000 R15: ffff88082a2f5e20
FS: 00007f259a64b6f0(0000) GS:ffff880864600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00007feb53f73da0 CR3: 000000102cc94000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process sysfsd (pid: 9345, threadinfo ffff88082a2f4000, task ffff88082a33cf00)
Stack:
ffff88182cc3e000 ffff88182cc3e000 0000000000000000 ffff88082a33cf00
<0> ffff88082a2f5cf0 ffffffff811dff52 ffff88082a2f5cf0 ffff88082c525168
<0> ffff88402c9fd2f8 ffff88402c9fd2f8 ffff88082a2f5d20 ffffffff811d7db2
Call Trace:
[<ffffffff811dff52>] pcie_aspm_exit_link_state+0xf5/0x11e
[<ffffffff811d7db2>] pci_stop_bus_device+0x76/0x7e
[<ffffffff811d7d67>] pci_stop_bus_device+0x2b/0x7e
[<ffffffff811d7e4f>] pci_remove_bus_device+0x15/0xb9
[<ffffffff811dcb8c>] remove_callback+0x29/0x3a
[<ffffffff81135aeb>] sysfs_schedule_callback_work+0x15/0x6d
[<ffffffff81072790>] worker_thread+0x19d/0x298
[<ffffffff8107273b>] ? worker_thread+0x148/0x298
[<ffffffff81135ad6>] ? sysfs_schedule_callback_work+0x0/0x6d
[<ffffffff810765c0>] ? autoremove_wake_function+0x0/0x38
[<ffffffff810725f3>] ? worker_thread+0x0/0x298
[<ffffffff8107629e>] kthread+0x7d/0x85
[<ffffffff8102eafa>] child_rip+0xa/0x20
[<ffffffff8102e4bc>] ? restore_args+0x0/0x30
[<ffffffff81076221>] ? kthread+0x0/0x85
[<ffffffff8102eaf0>] ? child_rip+0x0/0x20
Code: 89 e5 8a 50 48 31 c0 c0 ea 03 83 e2 07 e8 b2 de fe ff c9 48 98 c3 55 48 89 e5 41 56 49 89 fe 41 55 41 54 53 48 83 7f 10 00 74 04 <0f> 0b eb fe 48 8b 05 da 7d 63 00 4c 8d 60 e8 4c 89 e1 eb 24 4c
RIP [<ffffffff811df69b>] pcie_update_aspm_capable+0x15/0xbe
RSP <ffff88082a2f5ca0>
---[ end trace 6ae0f65bdeab8555 ]---

Reported-by: Alex Chiang <achiang@hp.com>
Tested-by: Alex Chiang <achiang@hp.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCIe ASPM: use pci_is_pcie()

Change for PCIe ASPM driver to use pci_is_pcie() instead of checking
pci_dev->is_pcie.

Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCIe ASPM: use pci_pcie_cap()

Use pci_pcie_cap() instead of pci_find_capability() to get PCIe capability
offset in PCIe ASPM driver. This avoids unnecessary search in PCI
configuration space.

Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: fix oops on root port removal

Fix the following BUG_ON() problem reported by Alex Chiang.

This problem happened when removing PCIe root port using PCI logical
hotplug operation.

The immediate cause of this problem is that the pointer to invalid
data structure is passed to pcie_update_aspm_capable() by
pcie_aspm_exit_link_state(). When pcie_aspm_exit_link_state() received
a pointer to root port link, it unconfigures the root port link and
frees its data structure at first. At this point, there are not links
to configure under the root port and the data structure for root port
link is already freed. So pcie_aspm_exit_link_state() must not call
pcie_update_aspm_capable() and pcie_config_aspm_path().

This patch fixes the problem by changing pcie_aspm_exit_link_state()
not to call pcie_update_aspm_capable() and pcie_config_aspm_path() if
the specified link is root port link.

------------[ cut here ]------------
kernel BUG at drivers/pci/pcie/aspm.c:606!
invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
last sysfs file: /sys/devices/pci0000:40/0000:40:13.0/remove
CPU 1
Modules linked in: shpchp
Pid: 9345, comm: sysfsd Not tainted 2.6.32-rc5 #98 ProLiant DL785 G6
RIP: 0010:[<ffffffff811df69b>] [<ffffffff811df69b>] pcie_update_aspm_capable+0x15/0xbe
RSP: 0018:ffff88082a2f5ca0 EFLAGS: 00010202
RAX: 0000000000000e77 RBX: ffff88182cc3e000 RCX: ffff88082a33d006
RDX: 0000000000000001 RSI: ffffffff811dff4a RDI: ffff88182cc3e000
RBP: ffff88082a2f5cc0 R08: ffff88182cc3e000 R09: 0000000000000000
R10: ffff88182fc00180 R11: ffff88182fc00198 R12: ffff88182cc3e000
R13: 0000000000000000 R14: ffff88182cc3e000 R15: ffff88082a2f5e20
FS: 00007f259a64b6f0(0000) GS:ffff880864600000(0000) knlGS:0000000000000000
CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00007feb53f73da0 CR3: 000000102cc94000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process sysfsd (pid: 9345, threadinfo ffff88082a2f4000, task ffff88082a33cf00)
Stack:
ffff88182cc3e000 ffff88182cc3e000 0000000000000000 ffff88082a33cf00
<0> ffff88082a2f5cf0 ffffffff811dff52 ffff88082a2f5cf0 ffff88082c525168
<0> ffff88402c9fd2f8 ffff88402c9fd2f8 ffff88082a2f5d20 ffffffff811d7db2
Call Trace:
[<ffffffff811dff52>] pcie_aspm_exit_link_state+0xf5/0x11e
[<ffffffff811d7db2>] pci_stop_bus_device+0x76/0x7e
[<ffffffff811d7d67>] pci_stop_bus_device+0x2b/0x7e
[<ffffffff811d7e4f>] pci_remove_bus_device+0x15/0xb9
[<ffffffff811dcb8c>] remove_callback+0x29/0x3a
[<ffffffff81135aeb>] sysfs_schedule_callback_work+0x15/0x6d
[<ffffffff81072790>] worker_thread+0x19d/0x298
[<ffffffff8107273b>] ? worker_thread+0x148/0x298
[<ffffffff81135ad6>] ? sysfs_schedule_callback_work+0x0/0x6d
[<ffffffff810765c0>] ? autoremove_wake_function+0x0/0x38
[<ffffffff810725f3>] ? worker_thread+0x0/0x298
[<ffffffff8107629e>] kthread+0x7d/0x85
[<ffffffff8102eafa>] child_rip+0xa/0x20
[<ffffffff8102e4bc>] ? restore_args+0x0/0x30
[<ffffffff81076221>] ? kthread+0x0/0x85
[<ffffffff8102eaf0>] ? child_rip+0x0/0x20
Code: 89 e5 8a 50 48 31 c0 c0 ea 03 83 e2 07 e8 b2 de fe ff c9 48 98 c3 55 48 89 e5 41 56 49 89 fe 41 55 41 54 53 48 83 7f 10 00 74 04 <0f> 0b eb fe 48 8b 05 da 7d 63 00 4c 8d 60 e8 4c 89 e1 eb 24 4c
RIP [<ffffffff811df69b>] pcie_update_aspm_capable+0x15/0xbe
RSP <ffff88082a2f5ca0>
---[ end trace 6ae0f65bdeab8555 ]---

Reported-by: Alex Chiang <achiang@hp.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Tested-by: Alex Chiang <achiang@hp.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: support L1 only

The definition of the ASPM support field in the Link Capabilities
Register had been changed by the "ASPM optionality ECN" as follows:

<Before>
00b Reserved
01b L0s Supported
10b Reserved
11b L0s and L1 Supported

<After>
00b No ASPM Support
01b L0s Supported
10b L1 Supported
11b L0s and L1 Supported

Current linux ASPM driver doesn't enable ASPM if the support field is
00b or 10b. So there is no impact about 00b. But current linux ASPM
driver doesn't enable L1 if the support field is 10b. With this patch,
10b (L1 support) is handled properly.

Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: support per direction l0s management

The L0s state can be managed separately for each direction (upstream
direction and downstream direction) of the link. But in the current
implementation, those are mixed up. With this patch, L0s for each
direction are managed separately.

To maintain three states (upstream direction L0s, downstream L0s and
L1), 'aspm_support', 'aspm_enabled', 'aspm_capable', 'aspm_disable'
and 'aspm_default' fields in struct pcie_link_state are changed to
3-bit from 2-bit. The 'latency' field is separated to two 'latency_up'
and 'latency_dw' fields to maintain exit latencies for each direction
of the link. For L0, 'latency_up.l0' and 'latency_dw.l0' are used to
configure upstream direction L0s and downstream direction L0s
respectively. For L1, larger value of 'latency_up.l1' and
'latency_dw.l1' is considered as L1 exit latency.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: support partial aspm enablement

In the current implementation, ASPM L0s/L1 is disabled for all links
in the hierarchy if one of the link doesn't meet latency requirement.
But we can partially enable ASPM L0s/L1 on sub-tree in the hierarchy.
This patch allows partial L0s/L1 enablement in the hierarchy. And it
also reduce the calculation cost of ASPM configuration very much.

In the previous implementation, all links were enabled with the same
state. With this patch, enabled state for each link is determined
simply as follows (the 'requested' is from policy_to_aspm_state()).

enabled = requested & (link->aspm_capable & link->aspm_disable)

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: introduce capable flag

Introduce 'aspm_capable' field to maintain the capable ASPM setting of
the link. By the 'aspm_capable', we don't need to recheck latency
every time ASPM policy is changed.

Each bit in 'aspm_capable' is associated to ASPM state (L0S/L1). The
bit is set if the associated ASPM state is supported by the link and
it satisfies the latency requirement (i.e. exit latency < endpoint
acceptable latency). The 'aspm_capable' is updated when

- an endpoint device is added (boot time or hot-plug time)
- an endpoint device is removed (hot-unplug time)
- PCI power state is changed.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: introduce disable flag

Introduce 'aspm_disable' flag to manage disabled ASPM state more
robust way.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: fix possible null pointer dereference

Fix possible NULL dereference in pcie_aspm_exit_link_state(). This
patch also cleanup some code.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: remove redundant list check

Remove the following check in __pcie_aspm_config_link() because it
nerver be true.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: do not clear enabled field by support field

We must not clear bits in 'aspm_enabled' using 'aspm_support', or
'aspm_enabled' and 'aspm_default' might be different from the actual
state. In addtion, 'aspm_default' should be intialized even if
'aspm_support' is 0.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: remove get_root_port_link

By having a pointer to the root port link, we can remove loops in
get_root_port_link() to search the root port link.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: cleanup pcie_aspm_sanity_check

Minor cleanup for pcie_aspm_sanity_check().

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: remove has_switch field

We don't need the 'has_switch' field in the struct pcie_link_state.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: cleanup calc_Lx_latency

Cleanup for calc_L0S_latency() and calc_L1_latency().

- Separate exit latency and acceptable latency calculation.
- Some minor cleanups.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: cleanup pcie_aspm_get_cap_device

Minor cleanup for pcie_aspm_get_cap_device().

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: cleanup clkpm checks

In the current ASPM implementation, callers of pcie_set_clock_pm() check
Clock PM capability of the link or current Clock PM state of the link.
This check should be done in pcie_set_clock_pm() itself.

This patch moves those checks into pcie_set_clock_pm(). It also
introduces pcie_set_clkpm_nocheck() that is equivalent to old
pcie_set_clock_pm(), for the caller who wants to change Clocl PM state
regardless of the Clock PM capability or current Clock PM state. In
addition, this patch changes the function name from
pcie_set_clock_pm() to pcie_set_clkpm() for consistency.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: cleanup __pcie_aspm_check_state_one

Clean up and simplify __pcie_aspm_check_state_one().

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: cleanup initialization

Clean up ASPM initialization by refactoring some functionality, renaming
functions, and moving things around.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: cleanup change input argument of aspm functions

In the current ASPM implementation, there are many functions that
take a pointer to struct pci_dev corresponding to the upstream component
of the link as a parameter. But, since those functions handle PCI
express link state, a pointer to struct pcie_link_state is more
suitable than a pointer to struct pci_dev. Changing a parameter to a
pointer to struct pcie_link_state makes ASPM code much simpler and
easier to read. This patch also contains some minor cleanups. This patch
doesn't have any functional change.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: cleanup misc in struct pcie_link_state

Cleanup for some fields in pcie_link_state.

- Add comments.
- make "downstream_has_switch" field 1-bit.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: cleanup clkpm state in struct pcie_link_state

The "clk_pm_capable", "clk_pm_enable" and "bios_clk_state" fields in
the struct pcie_link_state only take 1-bit value. So those fields
don't need to be defined as unsigned int. This patch makes those
fields 1-bit, and cleans up some related code.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: cleanup latency field in struct pcie_link_state

Clean up latency related data structures for ASPM.

- Introduce struct acpi_latency for exit latency and acceptable
latency management. With this change, struct endpoint_state is no
longer needed.

- We don't need to hold both upstream latency and downstream latency
in the current implementation.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: cleanup aspm state field in struct pcie_link_state

The "support_state", "enabled_state" and "bios_aspm_state" fields in
the struct pcie_link_state take 2-bit value. So those fields don't
need to be defined as unsigned int. This patch makes those fields
2-bit, and cleans up some related code.

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI ASPM: fix typo in struct pcie_link_state

Fix a typo in struct pcie_link_state.

The "sibiling" field in the struct pcie_link_state should be
"sibling".

Acked-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: disable ASPM on VIA root-port-under-bridge configurations

VIA has a strange chipset, it has root port under a bridge. Disable ASPM
for such strange chipset.

Cc: stable@kernel.org
Tested-by: Wolfgang Denk <wd@denx.de>
Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: properly clean up ASPM link state on device remove

We only want to disable ASPM when the last function is removed from
the parent's device list. We determine this by checking to see if
the parent's device list is completely empty.

Unfortunately, we never hit that code because the parent is considered
an upstream port, and never had an ASPM link_state associated with it.

The early check for !link_state causes us to return early, we never
discover that our device list is empty, and thus we never remove the
downstream ports' link_state nodes.

Instead of checking to see if the parent's device list is empty, we can
check to see if we are the last device on the list, and if so, then we
know that we can clean up properly.

Cc: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Alex Chiang <achiang@hp.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: Use msleep instead of cpu_relax during ASPM link retraining

The cpu_relax() function can be a noop on certain architectures like
IA-64 when CPU threads are disabled, so use msleep instead during link
retraining busy/wait loop.

Introduce define LINK_RETRAIN_TIMEOUT instead of hard-coding timeout in
pcie_aspm_configure_common_clock.

Use time_after() to avoid jiffy wraparound when checking for expired
timeout.

After timeout expires, recheck link status register link training bit
instead of checking for expired timeout to avoid possible false
positive.

Note that Matthew Wilcox came up with the first rough version of this
patch.

Reviewed-by: Matthew Wilcox <willy@linux.intel.com>
Signed-off-by: Andrew Patterson <andrew.patterson@hp.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: keep ASPM link state consistent throughout PCIe hierarchy

In a PCIe hierarchy with a switch present, if the link state of an
endpoint device is changed, we must check the whole hierarchy from the
endpoint device to root port, and for each link in the hierarchy, the new
link state should be configured. Previously, the implementation checked
the state but forgot to configure the links between root port to switch.
Fixes Novell bz #448987.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Tested-by: Andrew Patterson <andrew.patterson@hp.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

ACPI/PCI: PCIe ASPM _OSC support capabilities called when root bridge added

The _OSC capabilities OSC_ACTIVE_STATE_PWR_SUPPORT and
OSC_CLOCK_PWR_CAPABILITY_SUPPORT are set when the root bridge is added
with pci_acpi_osc_support(), so we no longer need to do it in the ASPM
driver. Also add the function pcie_aspm_enabled, which returns true if
pcie_aspm=off is not on the kernel command-line.

Signed-off-by: Andrew Patterson <andrew.patterson@hp.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCIe: ASPM: Break out of endless loop waiting for PCI config bits to switch

Makes a Compaq 6735s boot reliably again. It used to hang in the loop
on some boots. Give the link one second to train, otherwise break out
of the loop and reset the previously set clock bits.

Cc: stable@vger.kernel.org
Signed-off-by: Thomas Renninger <trenn@suse.de>
Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Matthew Garrett <mjg59@srcf.ucam.org>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: probing debug message uniformization

This patch uniformizes PCI probing debug boot messages with dev_printk()
intead of manual printk()

It changes adress range output from [%llx, %llx] to [%#llx-%#llx], like
in pci_request_region().

For example, it goes from the mixed-style:

PCI: 0000:00:1b.0 reg 10 64bit mmio: [f4280000, f4283fff]
pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold

to uniform:

pci 0000:00:1b.0: reg 10 64bit mmio: [0xf4280000-0xf4283fff]
pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold

This patch has been runtime tested, boot log messages diffed, everything
looks OK.

Acked-by: Bjorn Helgaas <bjorn.helgaas@hp.com>
Signed-off-by: Vincent Legoll <vincent.legoll@gmail.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: Fix pcie_aspm=force

pcie_aspm=force did not work because aspm_force was being double negated
leading to the sanity check failing. Moving a bracket should fix this.

Acked-by: Alan Cox <alan@redhat.com>
Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: add an option to allow ASPM enabled forcibly

A new option, pcie_aspm=force, will force ASPM to be enabled, even on system
with PCIe 1.0 devices.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: disable ASPM on pre-1.1 PCIe devices

Disable ASPM on pre-1.1 PCIe devices, as many of them don't implement it
correctly.

Tested-by: Jack Howarth <howarth@bromo.msbb.uc.edu>
Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: disable ASPM per ACPI FADT setting

The ACPI FADT table includes an ASPM control bit. If the bit is set, do
not enable ASPM since it may indicate that the platform doesn't actually
support the feature.

Tested-by: Jack Howarth <howarth@bromo.msbb.uc.edu>
Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: don't enable ASPM on devices with mixed PCIe/PCI functions

The Slot 03:00.* of JMicron controller has two functions, but one is
PCIE endpoint the other isn't PCIE device, very strange. PCIE spec
defines all functions should have the same config for ASPM, so disable
ASPM for the whole slot in this case.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

PCI: add PCI Express ASPM support

PCI Express ASPM defines a protocol for PCI Express components in the D0
state to reduce Link power by placing their Links into a low power state
and instructing the other end of the Link to do likewise. This
capability allows hardware-autonomous, dynamic Link power reduction
beyond what is achievable by software-only controlled power management.
However, The device should be configured by software appropriately.
Enabling ASPM will save power, but will introduce device latency.

This patch adds ASPM support in Linux. It introduces a global policy for
ASPM, a sysfs file /sys/module/pcie_aspm/parameters/policy can control
it. The interface can be used as a boot option too. Currently we have
below setting:
-default, BIOS default setting
-powersave, highest power saving mode, enable all available ASPM
state and clock power management
-performance, highest performance, disable ASPM and clock power
management
By default, the 'default' policy is used currently.

In my test, power difference between powersave mode and performance mode
is about 1.3w in a system with 3 PCIE links.

Note: some devices might not work well with aspm, either because chipset
issue or device issue. The patch provide API (pci_disable_link_state),
driver can disable ASPM for specific device.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

Revert "PCI: PCIE ASPM support"

This reverts commit 6c723d5bd89f03fc3ef627d50f89ade054d2ee3b.

It caused build errors on non-x86 platforms, config file confusion, and
even some boot errors on some x86-64 boxes. All around, not quite ready
for prime-time :(

Cc: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

PCI: PCIE ASPM support

PCI Express ASPM defines a protocol for PCI Express components in the D0
state to reduce Link power by placing their Links into a low power state
and instructing the other end of the Link to do likewise. This
capability allows hardware-autonomous, dynamic Link power reduction
beyond what is achievable by software-only controlled power management.
However, The device should be configured by software appropriately.
Enabling ASPM will save power, but will introduce device latency.

This patch adds ASPM support in Linux. It introduces a global policy for
ASPM, a sysfs file /sys/module/pcie_aspm/parameters/policy can control
it. The interface can be used as a boot option too. Currently we have
below setting:
-default, BIOS default setting
-powersave, highest power saving mode, enable all available ASPM
state
and clock power management
-performance, highest performance, disable ASPM and clock power
management
By default, the 'default' policy is used currently.

In my test, power difference between powersave mode and performance mode
is about 1.3w in a system with 3 PCIE links.

Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>