PM: runtime: add tracepoint for runtime_status changes

Existing runtime PM ftrace events (`rpm_suspend`, `rpm_resume`,
`rpm_return_int`) offer limited visibility into the exact timing of device
runtime power state transitions, particularly when asynchronous operations
are involved. When the `rpm_suspend` or `rpm_resume` functions are invoked
with the `RPM_ASYNC` flag, a return value of 0 i.e., success merely
indicates that the device power state request has been queued, not that
the device has yet transitioned.

A new ftrace event, `rpm_status`, is introduced. This event directly logs
the `power.runtime_status` value of a device whenever it changes providing
granular tracking of runtime power state transitions regardless of
synchronous or asynchronous `rpm_suspend` / `rpm_resume` usage.

Signed-off-by: Vilas Bhat <vilasbhat@google.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: Simplify pm_runtime_get_if_active() usage

There are two ways to opportunistically increment a device's runtime PM
usage count, calling either pm_runtime_get_if_active() or
pm_runtime_get_if_in_use(). The former has an argument to tell whether to
ignore the usage count or not, and the latter simply calls the former with
ign_usage_count set to false. The other users that want to ignore the
usage_count will have to explicitly set that argument to true which is a
bit cumbersome.

To make this function more practical to use, remove the ign_usage_count
argument from the function. The main implementation is in a static
function called pm_runtime_get_conditional() and implementations of
pm_runtime_get_if_active() and pm_runtime_get_if_in_use() are moved to
runtime.c.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Alex Elder <elder@linaro.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Takashi Iwai <tiwai@suse.de> # sound/
Reviewed-by: Jacek Lawrynowicz <jacek.lawrynowicz@linux.intel.com> # drivers/accel/ivpu/
Acked-by: Rodrigo Vivi <rodrigo.vivi@intel.com> # drivers/gpu/drm/i915/
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com> # drivers/pci/
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: fix missing rculist.h dependency

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>

PM: runtime: Document that force_suspend() is incompatible with SMART_SUSPEND

pm_runtime_force_suspend() cannot be used with DPM_FLAG_SMART_SUSPEND, so
note this in the kerneldoc.

If DPM_FLAG_SMART_SUSPEND is set and the PM core cannot skip system resume
it will call pm_runtime_active() on the driver. This can lead to an
inconsistent state where:

pm_runtime_force_suspend() called ->runtime_suspend

but

device_resume_noirq() called pm_runtime_set_active()

This leaves the driver actually suspended but marked as active.

Signed-off-by: Richard Fitzgerald <rf@opensource.cirrus.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

cpuidle, ARM: OMAP2+: powerdomain: Remove trace_.*_rcuidle()

OMAP was the one and only user.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Tony Lindgren <tony@atomide.com>
Tested-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://lore.kernel.org/r/20230112195541.782536366@infradead.org

PM: runtime: Adjust white space in the core code

Some inconsistent usage of white space in the PM-runtime core code
causes that code to be somewhat harder to read that it would have
been otherwise, so adjust the white space in there to be more
consistent with the rest of the code.

No expected functional impact.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: Relocate rpm_callback() right after __rpm_callback()

Because rpm_callback() is a wrapper around __rpm_callback(), and the
only caller of it after the change eliminating an invocation of it
from rpm_idle(), move the former next to the latter to make the code
a bit easier to follow.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Adrian Hunter <adrian.hunter@intel.com>

PM: runtime: Do not call __rpm_callback() from rpm_idle()

Calling __rpm_callback() from rpm_idle() after adding device links
support to the former is a clear mistake.

Not only it causes rpm_idle() to carry out unnecessary actions, but it
is also against the assumption regarding the stability of PM-runtime
status across __rpm_callback() invocations, because rpm_suspend() and
rpm_resume() may run in parallel with __rpm_callback() when it is called
by rpm_idle() and the device's PM-runtime status can be updated by any
of them.

Fixes: 21d5c57b3726 ("PM / runtime: Use device links")
Link: https://lore.kernel.org/linux-pm/36aed941-a73e-d937-2721-4f0decd61ce0@quicinc.com
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Adrian Hunter <adrian.hunter@intel.com>

PM: runtime: Return -EINPROGRESS from rpm_resume() in the RPM_NOWAIT case

The prospective callers of rpm_resume() passing RPM_NOWAIT to it may
be confused when it returns 0 without actually resuming the device
which may happen if the device is suspending at the given time and it
will only resume when the suspend in progress has completed. To avoid
that confusion, return -EINPROGRESS from rpm_resume() in that case.

Since none of the current callers passing RPM_NOWAIT to rpm_resume()
check its return value, this change has no functional impact.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

PM: runtime: Extend support for wakeirq for force_suspend|resume

A driver that makes use of pm_runtime_force_suspend|resume() to support
system suspend/resume, currently needs to manage the wakeirq support
itself. To avoid the boilerplate code in the driver's system suspend/resume
callbacks in particular, let's extend pm_runtime_force_suspend|resume() to
deal with the wakeirq.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: Fix supplier device management during consumer probe

Because pm_runtime_get_suppliers() bumps up the rpm_active counter
of each device link to a supplier of the given device in addition
to bumping up the supplier's PM-runtime usage counter, a runtime
suspend of the consumer device may case the latter to go down to 0
when pm_runtime_put_suppliers() is running on a remote CPU. If that
happens after pm_runtime_put_suppliers() has released power.lock for
the consumer device, and a runtime resume of that device takes place
immediately after it, before pm_runtime_put() is called for the
supplier, that pm_runtime_put() call may cause the supplier to be
suspended even though the consumer is active.

To prevent that from happening, modify pm_runtime_get_suppliers() to
call pm_runtime_get_sync() for the given device's suppliers without
touching the rpm_active counters of the involved device links
Accordingly, modify pm_runtime_put_suppliers() to call pm_runtime_put()
for the given device's suppliers without looking at the rpm_active
counters of the device links at hand. [This is analogous to what
happened before commit 4c06c4e6cf63 ("driver core: Fix possible
supplier PM-usage counter imbalance").]

Since pm_runtime_get_suppliers() sets supplier_preactivated for each
device link where the supplier's PM-runtime usage counter has been
incremented and pm_runtime_put_suppliers() calls pm_runtime_put() for
the suppliers whose device links have supplier_preactivated set, the
PM-runtime usage counter is balanced for each supplier and this is
independent of the runtime suspend and resume of the consumer device.

However, in case a device link with DL_FLAG_PM_RUNTIME set is dropped
during the consumer device probe, so pm_runtime_get_suppliers() bumps
up the supplier's PM-runtime usage counter, but it cannot be dropped by
pm_runtime_put_suppliers(), make device_link_release_fn() take care of
that.

Fixes: 4c06c4e6cf63 ("driver core: Fix possible supplier PM-usage counter imbalance")
Reported-by: Peter Wang <peter.wang@mediatek.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Cc: 5.1+ <stable@vger.kernel.org> # 5.1+

PM: runtime: Redefine pm_runtime_release_supplier()

Instead of passing an extra bool argument to pm_runtime_release_supplier(),
make its callers take care of triggering a runtime-suspend of the
supplier device as needed.

No expected functional impact.

Suggested-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: 5.1+ <stable@vger.kernel.org> # 5.1+

PM: runtime: Allow to call __pm_runtime_set_status() from atomic context

The only two users of __pm_runtime_set_status() are pm_runtime_set_active()
and pm_runtime_set_suspended(). These are widely used and should be called
from non-atomic context to work as expected. However, it would be
convenient to allow them be called from atomic context too, as shown from a
subsequent change, so let's add support for this.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Tested-by: Maulik Shah <quic_mkshah@quicinc.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: Avoid device usage count underflows

A PM-runtime device usage count underflow is potentially critical,
because it may cause a device to be suspended when it is expected to
be operational. It is also a programming problem that would be good
to catch and warn about.

For this reason, (1) make rpm_check_suspend_allowed() return an error
when the device usage count is negative to prevent devices from being
suspended in that case, (2) introduce rpm_drop_usage_count() that will
detect device usage count underflows, warn about them and fix them up,
and (3) use it to drop the usage count in a few places instead of
atomic_dec_and_test().

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

PM: runtime: Have devm_pm_runtime_enable() handle pm_runtime_dont_use_autosuspend()

The PM Runtime docs say:

Drivers in ->remove() callback should undo the runtime PM changes done
in ->probe(). Usually this means calling pm_runtime_disable(),
pm_runtime_dont_use_autosuspend() etc.

From grepping code, it's clear that many people aren't aware of the
need to call pm_runtime_dont_use_autosuspend().

When brainstorming solutions, one idea that came up was to leverage
the new-ish devm_pm_runtime_enable() function. The idea here is that:

* When the devm action is called we know that the driver is being
removed. It's the perfect time to undo the use_autosuspend.

* The code of pm_runtime_dont_use_autosuspend() already handles the
case of being called when autosuspend wasn't enabled.

Suggested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Douglas Anderson <dianders@chromium.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: Simplify locking in pm_runtime_put_suppliers()

Notice that pm_runtime_put_suppliers() cannot be called with
disabled interrupts, because it may sleep (due to the device
links read locking in the non-SRCU case), and so it can use
spin_lock_irq() and spin_unlock_irq() for the locking.

Update the function accordingly and while at it move the "put"
local variable in it into the inner block where it is used.

This change is not expected to have any visible functional impact.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

PM: runtime: Add safety net to supplier device release

Because refcount_dec_not_one() returns true if the target refcount
becomes saturated, it is generally unsafe to use its return value as
a loop termination condition, but that is what happens when a device
link's supplier device is released during runtime PM suspend
operations and on device link removal.

To address this, introduce pm_runtime_release_supplier() to be used
in the above cases which will check the supplier device's runtime
PM usage counter in addition to the refcount_dec_not_one() return
value, so the loop can be terminated in case the rpm_active refcount
value becomes invalid, and update the code in question to use it as
appropriate.

This change is not expected to have any visible functional impact.

Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>

PM: runtime: Capture device status before disabling runtime PM

In some cases (for example, during system-wide suspend and resume of
devices) it is useful to know whether or not runtime PM has ever been
enabled for a given device and, if so, what the runtime PM status of
it had been right before runtime PM was disabled for it last time.

For this reason, introduce a new struct dev_pm_info field called
last_status that will be used for capturing the runtime PM status of
the device when its power.disable_depth counter changes from 0 to 1.

The new field will be set to RPM_INVALID to start with and whenever
power.disable_depth changes from 1 to 0, so it will be valid only
when runtime PM of the device is currently disabled, but it has been
enabled at least once.

Immediately use power.last_status in rpm_resume() to make it handle
the case when PM runtime is disabled for the device, but its runtime
PM status is RPM_ACTIVE more consistently. Namely, make it return 1
if power.last_status is also equal to RPM_ACTIVE in that case (the
idea being that if the status was RPM_ACTIVE last time when
power.disable_depth was changing from 0 to 1 and it is still
RPM_ACTIVE, it can be assumed to reflect what happened to the device
last time when it was using runtime PM) and -EACCES otherwise.

Update the documentation to provide a description of last_status and
change the description of pm_runtime_resume() in it to reflect the
new behavior of rpm_active().

While at it, rearrange the code in pm_runtime_enable() to be more
straightforward and replace the WARN() macro in it with a pr_warn()
invocation which is less disruptive.

Link: https://lore.kernel.org/linux-pm/20211026222626.39222-1-ulf.hansson@linaro.org/t/#u
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / wakeirq: support enabling wake-up irq after runtime_suspend called

When the dedicated wake IRQ is level trigger, and it uses the
device's low-power status as the wakeup source, that means if the
device is not in low-power state, the wake IRQ will be triggered
if enabled; For this case, need enable the wake IRQ after running
the device's ->runtime_suspend() which make it enter low-power state.

e.g.
Assume the wake IRQ is a low level trigger type, and the wakeup
signal comes from the low-power status of the device.
The wakeup signal is low level at running time (0), and becomes
high level when the device enters low-power state (runtime_suspend
(1) is called), a wakeup event at (2) make the device exit low-power
state, then the wakeup signal also becomes low level.

------------------
| ^ ^|
---------------- | | --------------
|<---(0)--->|<--(1)--| (3) (2) (4)

if enable the wake IRQ before running runtime_suspend during (0),
a wake IRQ will arise, it causes resume immediately;
it works if enable wake IRQ ( e.g. at (3) or (4)) after running
->runtime_suspend().

This patch introduces a new status WAKE_IRQ_DEDICATED_REVERSE to
optionally support enabling wake IRQ after running ->runtime_suspend().

Suggested-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Chunfeng Yun <chunfeng.yun@mediatek.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: add devm_pm_runtime_enable helper

A typical code pattern for pm_runtime_enable() call is to call it in the
_probe function and to call pm_runtime_disable() both from _probe error
path and from _remove function. For some drivers the whole remove
function would consist of the call to pm_remove_disable().

Add helper function to replace this bolierplate piece of code. Calling
devm_pm_runtime_enable() removes the need for calling
pm_runtime_disable() both in the probe()'s error path and in the
remove() function.

Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Link: https://lore.kernel.org/r/20210731195034.979084-2-dmitry.baryshkov@linaro.org
Acked-by: Rafael J. Wysocki <rafael@kernel.org>
Signed-off-by: Stephen Boyd <sboyd@kernel.org>

PM: runtime: Allow unassigned ->runtime_suspend|resume callbacks

We are currently allowing ->runtime_idle() callbacks to be unassigned
without returning an error code from rpm_idle(). This has been useful to
avoid boilerplate code in drivers. Let's take this approach a step further,
by allowing also unassigned ->runtime_suspend|resume() callbacks.

In this way, a consumer/supplier device link can be used to let a consumer
device be power managed through its supplier device, without requiring
assigned ->runtime_suspend|resume() callbacks for the consumer device, for
example.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: Improve path in rpm_idle() when no callback

When pm_runtime_no_callbacks() has been called for a struct device to set
the dev->power.no_callbacks flag for it, it enables rpm_idle() to take a
slightly quicker path by assuming that a ->runtime_idle() callback would
have returned 0 to indicate success.

A device that does not have the dev->power.no_callbacks flag set for it,
may still be missing a corresponding ->runtime_idle() callback, in which
case the slower path in rpm_idle() is taken. Let's improve the behaviour
for this case, by aligning code to the quicker path.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: Fix unpaired parent child_count for force_resume

As pm_runtime_need_not_resume() relies also on usage_count, it can return
a different value in pm_runtime_force_suspend() compared to when called in
pm_runtime_force_resume(). Different return values can happen if anything
calls PM runtime functions in between, and causes the parent child_count
to increase on every resume.

So far I've seen the issue only for omapdrm that does complicated things
with PM runtime calls during system suspend for legacy reasons:

omap_atomic_commit_tail() for omapdrm.0
dispc_runtime_get()
wakes up 58000000.dss as it's the dispc parent
dispc_runtime_resume()
rpm_resume() increases parent child_count
dispc_runtime_put() won't idle, PM runtime suspend blocked
pm_runtime_force_suspend() for 58000000.dss, !pm_runtime_need_not_resume()
__update_runtime_status()
system suspended
pm_runtime_force_resume() for 58000000.dss, pm_runtime_need_not_resume()
pm_runtime_enable() only called because of pm_runtime_need_not_resume()
omap_atomic_commit_tail() for omapdrm.0
dispc_runtime_get()
wakes up 58000000.dss as it's the dispc parent
dispc_runtime_resume()
rpm_resume() increases parent child_count
dispc_runtime_put() won't idle, PM runtime suspend blocked
...
rpm_suspend for 58000000.dss but parent child_count is now unbalanced

Let's fix the issue by adding a flag for needs_force_resume and use it in
pm_runtime_force_resume() instead of pm_runtime_need_not_resume().

Additionally omapdrm system suspend could be simplified later on to avoid
lots of unnecessary PM runtime calls and the complexity it adds. The
driver can just use internal functions that are shared between the PM
runtime and system suspend related functions.

Fixes: 4918e1f87c5f ("PM / runtime: Rework pm_runtime_force_suspend/resume()")
Signed-off-by: Tony Lindgren <tony@atomide.com>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Tested-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Cc: 4.16+ <stable@vger.kernel.org> # 4.16+
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: remove kernel-doc warnings

remove make W=1 warnings

drivers/base/power/runtime.c:926: warning: Function parameter or
member 'timer' not described in 'pm_suspend_timer_fn'

drivers/base/power/runtime.c:926: warning: Excess function parameter
'data' description in 'pm_suspend_timer_fn'

Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: Fix race getting/putting suppliers at probe

pm_runtime_put_suppliers() must not decrement rpm_active unless the
consumer is suspended. That is because, otherwise, it could suspend
suppliers for an active consumer.

That can happen as follows:

static int driver_probe_device(struct device_driver *drv, struct device *dev)
{
int ret = 0;

if (!device_is_registered(dev))
return -ENODEV;

dev->can_match = true;
pr_debug("bus: '%s': %s: matched device %s with driver %s\n",
drv->bus->name, __func__, dev_name(dev), drv->name);

pm_runtime_get_suppliers(dev);
if (dev->parent)
pm_runtime_get_sync(dev->parent);

At this point, dev can runtime suspend so rpm_put_suppliers() can run,
rpm_active becomes 1 (the lowest value).

pm_runtime_barrier(dev);
if (initcall_debug)
ret = really_probe_debug(dev, drv);
else
ret = really_probe(dev, drv);

Probe callback can have runtime resumed dev, and then runtime put
so dev is awaiting autosuspend, but rpm_active is 2.

pm_request_idle(dev);

if (dev->parent)
pm_runtime_put(dev->parent);

pm_runtime_put_suppliers(dev);

Now pm_runtime_put_suppliers() will put the supplier
i.e. rpm_active 2 -> 1, but consumer can still be active.

return ret;
}

Fix by checking the runtime status. For any status other than
RPM_SUSPENDED, rpm_active can be considered to be "owned" by
rpm_[get/put]_suppliers() and pm_runtime_put_suppliers() need do nothing.

Reported-by: Asutosh Das <asutoshd@codeaurora.org>
Fixes: 4c06c4e6cf63 ("driver core: Fix possible supplier PM-usage counter imbalance")
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: 5.1+ <stable@vger.kernel.org> # 5.1+
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: Fix ordering in pm_runtime_get_suppliers()

rpm_active indicates how many times the supplier usage_count has been
incremented. Consequently it must be updated after pm_runtime_get_sync() of
the supplier, not before.

Fixes: 4c06c4e6cf63 ("driver core: Fix possible supplier PM-usage counter imbalance")
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: 5.1+ <stable@vger.kernel.org> # 5.1+
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: Defer suspending suppliers

Because the PM-runtime status of the device is not updated in
__rpm_callback(), attempts to suspend the suppliers of the given
device triggered by the rpm_put_suppliers() call in there may
cause a supplier to be suspended completely before the status of
the consumer is updated to RPM_SUSPENDED, which is confusing.

To avoid that (1) modify __rpm_callback() to only decrease the
PM-runtime usage counter of each supplier and (2) make rpm_suspend()
try to suspend the suppliers after changing the consumer's status to
RPM_SUSPENDED, in analogy with the device's parent.

Link: https://lore.kernel.org/linux-pm/CAPDyKFqm06KDw_p8WXsM4dijDbho4bb6T4k50UqqvR1_COsp8g@mail.gmail.com/
Fixes: 21d5c57b3726 ("PM / runtime: Use device links")
Reported-by: elaine.zhang <zhangqing@rock-chips.com>
Diagnosed-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

Revert "PM: runtime: Update device status before letting suppliers suspend"

Revert commit 44cc89f76464 ("PM: runtime: Update device status
before letting suppliers suspend") that introduced a race condition
into __rpm_callback() which allowed a concurrent rpm_resume() to
run and resume the device prematurely after its status had been
changed to RPM_SUSPENDED by __rpm_callback().

Fixes: 44cc89f76464 ("PM: runtime: Update device status before letting suppliers suspend")
Link: https://lore.kernel.org/linux-pm/24dfb6fc-5d54-6ee2-9195-26428b7ecf8a@intel.com/
Reported-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: 4.10+ <stable@vger.kernel.org> # 4.10+
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

PM: runtime: Update device status before letting suppliers suspend

Because the PM-runtime status of the device is not updated in
__rpm_callback(), attempts to suspend the suppliers of the given
device triggered by rpm_put_suppliers() called by it may fail.

Fix this by making __rpm_callback() update the device's status to
RPM_SUSPENDED before calling rpm_put_suppliers() if the current
status of the device is RPM_SUSPENDING and the callback just invoked
by it has returned 0 (success).

While at it, modify the code in __rpm_callback() to always check
the device's PM-runtime status under its PM lock.

Link: https://lore.kernel.org/linux-pm/CAPDyKFqm06KDw_p8WXsM4dijDbho4bb6T4k50UqqvR1_COsp8g@mail.gmail.com/
Fixes: 21d5c57b3726 ("PM / runtime: Use device links")
Reported-by: Elaine Zhang <zhangqing@rock-chips.com>
Diagnosed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Tested-by: Elaine Zhang <zhangiqng@rock-chips.com>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Cc: 4.10+ <stable@vger.kernel.org> # 4.10+
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

drivers: base: power: Fix resposible -> responsible in runtime.c

s/resposible/responsible/

Acked-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Bhaskar Chowdhury <unixbhaskar@gmail.com>
Link: https://lore.kernel.org/r/20210120143312.3229181-1-unixbhaskar@gmail.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

PM: runtime: Fix resposible -> responsible in runtime.c

s/resposible/responsible/

Signed-off-by: Bhaskar Chowdhury <unixbhaskar@gmail.com>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
[ rjw: Subject edit ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: Drop pm_runtime_clean_up_links()

After commit d12544fb2aa9 ("PM: runtime: Remove link state checks in
rpm_get/put_supplier()") nothing prevents the consumer device's
runtime PM from acquiring additional references to the supplier
device after pm_runtime_clean_up_links() has run (or even while it
is running), so calling this function from __device_release_driver()
may be pointless (or even harmful).

Moreover, it ignores stateless device links, so the runtime PM
handling of managed and stateless device links is inconsistent
because of it, so better get rid of it entirely.

Fixes: d12544fb2aa9 ("PM: runtime: Remove link state checks in rpm_get/put_supplier()")
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: 5.1+ <stable@vger.kernel.org> # 5.1+
Tested-by: Xiang Chen <chenxiang66@hisilicon.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

PM: runtime: Drop runtime PM references to supplier on link removal

While removing a device link, drop the supplier device's runtime PM
usage counter as many times as needed to drop all of the runtime PM
references to it from the consumer in addition to dropping the
consumer's link count.

Fixes: baa8809f6097 ("PM / runtime: Optimize the use of device links")
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: 5.1+ <stable@vger.kernel.org> # 5.1+
Tested-by: Xiang Chen <chenxiang66@hisilicon.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

PM: runtime: Remove link state checks in rpm_get/put_supplier()

To support runtime PM for hisi SAS driver (the driver is in directory
drivers/scsi/hisi_sas), we add device link between scsi_device->sdev_gendev
(consumer device) and hisi_hba->dev(supplier device) with flags
DL_FLAG_PM_RUNTIME | DL_FLAG_RPM_ACTIVE.

After runtime suspended consumers and supplier, unload the dirver which
causes a hung.

We found that it called function device_release_driver_internal() to
release the supplier device (hisi_hba->dev), as the device link was
busy, it set the device link state to DL_STATE_SUPPLIER_UNBIND, and
then it called device_release_driver_internal() to release the consumer
device (scsi_device->sdev_gendev).

Then it would try to call pm_runtime_get_sync() to resume the consumer
device, but because consumer-supplier relation existed, it would try
to resume the supplier first, but as the link state was already
DL_STATE_SUPPLIER_UNBIND, so it skipped resuming the supplier and only
resumed the consumer which hanged (it sends IOs to resume scsi_device
while the SAS controller is suspended).

Simple flow is as follows:

device_release_driver_internal -> (supplier device)
if device_links_busy ->
device_links_unbind_consumers ->
...
WRITE_ONCE(link->status, DL_STATE_SUPPLIER_UNBIND)
device_release_driver_internal (consumer device)
pm_runtime_get_sync -> (consumer device)
...
__rpm_callback ->
rpm_get_suppliers ->
if link->state == DL_STATE_SUPPLIER_UNBIND -> skip the action of resuming the supplier
...
pm_runtime_clean_up_links
...

Correct suspend/resume ordering between a supplier device and its consumer
devices (resume the supplier device before resuming consumer devices, and
suspend consumer devices before suspending the supplier device) should be
guaranteed by runtime PM, but the state checks in rpm_get_supplier() and
rpm_put_supplier() break this rule, so remove them.

Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
[ rjw: Subject and changelog edits ]
Cc: All applicable <stable@vger.kernel.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: Improve kerneldoc of pm_runtime_get_if_active()

The kerneldoc comment of pm_runtime_get_if_active() doesn't list the
second argument of the function properly, so fix that and while at it
clarify that comment somewhat and add some markup to it.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Sakari Ailus <sakari.ailus@linux.intel.com>

PM: runtime: Make clear what we do when conditions are wrong in rpm_suspend()

rpm_suspend() simple bails out when conditions are wrong. But this is not
immediately obvious from the code. Make it clear what we do when conditions
are wrong in rpm_suspend().

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: runtime: Add pm_runtime_get_if_active()

pm_runtime_get_if_in_use() bumps up the PM-runtime usage count if it
is not equal to zero and the device's PM-runtime status is 'active'.
This works for drivers that do not use autoidle, but for those that
do, the function returns zero even when the device is active.

In order to maintain sane device state while the device is powered on
in the hope that it'll be needed, pm_runtime_get_if_active(dev, true)
returns a positive value if the device's PM-runtime status is 'active'
when it is called, in which case it also increments the device's usage
count.

If the second argument of pm_runtime_get_if_active() is 'false', the
function behaves just like pm_runtime_get_if_in_use(), so redefine
the latter as a wrapper around the former.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
[ rjw: Changelog ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM-runtime: add tracepoints for usage_count changes

Add tracepoints to remaining places where device's power.usage_count
is changed.

This helps debugging where and why autosuspend is prevented.

Signed-off-by: Michał Mirosław <mirq-linux@rere.qmqm.pl>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

driver/core: Convert to use built-in RCU list checking

This commit applies the consolidated hlist_for_each_entry_rcu() support
for lockdep conditions.

Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>

driver core: Remove device link creation limitation

If device_link_add() is called for a consumer/supplier pair with an
existing device link between them and the existing link's type is
not in agreement with the flags passed to that function by its
caller, NULL will be returned. That is seriously inconvenient,
because it forces the callers of device_link_add() to worry about
what others may or may not do even if that is not relevant to them
for any other reasons.

It turns out, however, that this limitation can be made go away
relatively easily.

The underlying observation is that if DL_FLAG_STATELESS has been
passed to device_link_add() in flags for the given consumer/supplier
pair at least once, calling either device_link_del() or
device_link_remove() to release the link returned by it should work,
but there are no other requirements associated with that flag. In
turn, if at least one of the callers of device_link_add() for the
given consumer/supplier pair has not passed DL_FLAG_STATELESS to it
in flags, the driver core should track the status of the link and act
on it as appropriate (ie. the link should be treated as "managed").
This means that DL_FLAG_STATELESS needs to be set for managed device
links and it should be valid to call device_link_del() or
device_link_remove() to drop references to them in certain
sutiations.

To allow that to happen, introduce a new (internal) device link flag
called DL_FLAG_MANAGED and make device_link_add() set it automatically
whenever DL_FLAG_STATELESS is not passed to it. Also make it take
additional references to existing device links that were previously
stateless (that is, with DL_FLAG_STATELESS set and DL_FLAG_MANAGED
unset) and will need to be managed going forward and initialize
their status (which has been DL_STATE_NONE so far).

Accordingly, when a managed device link is dropped automatically
by the driver core, make it clear DL_FLAG_MANAGED, reset the link's
status back to DL_STATE_NONE and drop the reference to it associated
with DL_FLAG_MANAGED instead of just deleting it right away (to
allow it to stay around in case it still needs to be released
explicitly by someone).

With that, since setting DL_FLAG_STATELESS doesn't mean that the
device link in question is not managed any more, replace all of the
status-tracking checks against DL_FLAG_STATELESS with analogous
checks against DL_FLAG_MANAGED and update the documentation to
reflect these changes.

While at it, make device_link_add() reject flags that it does not
recognize, including DL_FLAG_MANAGED.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Saravana Kannan <saravanak@google.com>
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Review-by: Saravana Kannan <saravanak@google.com>
Link: https://lore.kernel.org/r/2305283.AStDPdUUnE@kreacher
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

PM / QOS: Rename __dev_pm_qos_read_value() and dev_pm_qos_raw_read_value()

dev_pm_qos_read_value() will soon need to support more constraint types
(min/max frequency) and will have another argument to it, i.e. type of
the constraint. While that is fine for the existing users of
dev_pm_qos_read_value(), but not that optimal for the callers of
__dev_pm_qos_read_value() and dev_pm_qos_raw_read_value() as all the
callers of these two routines are only looking for resume latency
constraint.

Lets make these two routines care only about the resume latency
constraint and rename them to __dev_pm_qos_resume_latency() and
dev_pm_qos_raw_resume_latency().

Suggested-by: Rafael J. Wysocki <rjw@rjwysocki.net>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

drivers: base: power: add proper SPDX identifiers on files that did not have them.

There were a few files in the driver core power code that did not have
SPDX identifiers on them, so fix that up. At the same time, remove the
"free form" text that specified the license of the file, as that is
impossible for any tool to properly parse.

Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

PM-runtime: Call pm_runtime_active|suspended_time() from sysfs

Avoid the open-coding of the accounted time acquisition in
runtime_active|suspend_time_show() and make them call
pm_runtime_active|suspended_time() instead.

Note that this change also indirectly avoids holding dev->power.lock
around the do_div() computation and the sprintf() call which is an
additional improvement.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
[ rjw: Changelog ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM-runtime: Consolidate code to get active/suspended time

In a step to consolidate the code for fetching the PM-runtime
active/suspended time for a device, add a common function for that
and make the existing pm_runtime_suspended_time() call it.

Also add a corresponding pm_runtime_active_time() calling the new
common function.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
[ rjw: Changelog, function rename ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM-runtime: Fix deadlock when canceling hrtimer

When rpm_resume() desactivates the autosuspend timer, it should only
try to cancel hrtimer but not wait for the handler to finish, because
both rpm_resume() and pm_suspend_timer_fn() take the power.lock.

A deadlock is possible as follows:

CPU0 CPU1
rpm_resume()
spin_lock_irqsave
pm_suspend_timer_fn()
spin_lock_irqsave
pm_runtime_deactivate_timer()
hrtimer_cancel()

It is sufficient to call hrtimer_try_to_cancel() from
pm_runtime_deactivate_timer(), because dev->power.timer_expires
reset to 0 by it, so use that function instead of hrtimer_cancel().

Fixes: 8234f6734c5d ("PM-runtime: Switch autosuspend over to using hrtimers")
Reported-by: Sunzhaosheng Sun(Zhaosheng) <sunzhaosheng@hisilicon.com>
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
[ rjw: Changelog ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

driver core: Fix PM-runtime for links added during consumer probe

Commit 4c06c4e6cf63 ("driver core: Fix possible supplier PM-usage
counter imbalance") introduced a regression that causes suppliers
to be suspended prematurely for device links added during consumer
driver probe if the initial PM-runtime status of the consumer is
"suspended" and the consumer is resumed after adding the link and
before pm_runtime_put_suppliers() is called. In that case,
pm_runtime_put_suppliers() will drop the rpm_active refcount for
the link by one and (since rpm_active is equal to two after the
preceding consumer resume) the supplier's PM-runtime usage counter
will be decremented, which may cause the supplier to suspend even
though the consumer's PM-runtime status is "active".

For this reason, partially revert commit 4c06c4e6cf63 as the problem
it tried to fix needs to be addressed somewhat differently, and
change pm_runtime_get_suppliers() and pm_runtime_put_suppliers() so
that the latter only drops rpm_active references acquired by the
former. [This requires adding a new field to struct device_link,
but I coulnd't find a cleaner way to address the issue that would
work in all cases.]

This causes pm_runtime_put_suppliers() to effectively ignore device
links added during consumer probe, so device_link_add() doesn't need
to worry about ensuring that suppliers will remain active after
pm_runtime_put_suppliers() for links created with DL_FLAG_RPM_ACTIVE
set and it only needs to bump up rpm_active by one for those links,
so pm_runtime_active_link() is not necessary any more.

Fixes: 4c06c4e6cf63 ("driver core: Fix possible supplier PM-usage counter imbalance")
Reported-by: Jon Hunter <jonathanh@nvidia.com>
Tested-by: Jon Hunter <jonathanh@nvidia.com>
Tested-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Tested-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

driver core: Fix possible supplier PM-usage counter imbalance

If a stateless device link to a certain supplier with
DL_FLAG_PM_RUNTIME set in the flags is added and then removed by the
consumer driver's probe callback, the supplier's PM-runtime usage
counter will be nonzero after that which effectively causes the
supplier to remain "always on" going forward.

Namely, device_link_add() called to add the link invokes
device_link_rpm_prepare() which notices that the consumer driver is
probing, so it increments the supplier's PM-runtime usage counter
with the assumption that the link will stay around until
pm_runtime_put_suppliers() is called by driver_probe_device(),
but if the link goes away before that point, the supplier's
PM-runtime usage counter will remain nonzero.

To prevent that from happening, first rework pm_runtime_get_suppliers()
and pm_runtime_put_suppliers() to use the rpm_active refounts of device
links and make the latter only drop rpm_active and the supplier's
PM-runtime usage counter for each link by one, unless rpm_active is
one already for it. Next, modify device_link_add() to bump up the
new link's rpm_active refcount and the suppliers PM-runtime usage
counter by two, to prevent pm_runtime_put_suppliers(), if it is
called subsequently, from suspending the supplier prematurely (in
case its PM-runtime usage counter goes down to 0 in there).

Due to the way rpm_put_suppliers() works, this change does not
affect runtime suspend of the consumer ends of new device links (or,
generally, device links for which DL_FLAG_PM_RUNTIME has just been
set).

Fixes: e2f3cd831a28 ("driver core: Fix handling of runtime PM flags in device_link_add()")
Reported-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Tested-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

PM-runtime: Fix __pm_runtime_set_status() race with runtime resume

Commit 4080ab083000 ("PM-runtime: Take suppliers into account in
__pm_runtime_set_status()") introduced a race condition that may
trigger if __pm_runtime_set_status() is used incorrectly (that is,
if it is called when PM-runtime is enabled for the target device
and working).

In that case, if the original PM-runtime status of the device is
RPM_SUSPENDED, a runtime resume of the device may occur after
__pm_runtime_set_status() has dropped its power.lock spinlock
and before deactivating its suppliers, so the suppliers may be
deactivated while the device is PM-runtime-active which may lead
to functional issues.

To avoid that, modify __pm_runtime_set_status() to check whether
or not PM-runtime is enabled for the device before activating its
suppliers (if the new status is RPM_ACTIVE) and either return an
error if that's the case or increment the device's disable_depth
counter to prevent PM-runtime from being enabled for it while
the remaining part of the function is running (disable_depth is
then decremented on the way out).

Fixes: 4080ab083000 ("PM-runtime: Take suppliers into account in __pm_runtime_set_status()")
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

PM-runtime: Take suppliers into account in __pm_runtime_set_status()

If the target device has any suppliers, as reflected by device links
to them, __pm_runtime_set_status() does not take them into account,
which is not consistent with the other parts of the PM-runtime
framework and may lead to programming mistakes.

Modify __pm_runtime_set_status() to take suppliers into account by
activating them upfront if the new status is RPM_ACTIVE and
deactivating them on exit if the new status is RPM_SUSPENDED.

If the activation of one of the suppliers fails, the new status
will be RPM_SUSPENDED and the (remaining) suppliers will be
deactivated on exit (the child count of the device's parent
will be dropped too then).

Of course, adding device links locking to __pm_runtime_set_status()
means that it cannot be run fron interrupt context, so make it use
spin_lock_irq() and spin_unlock_irq() instead of spin_lock_irqsave()
and spin_unlock_irqrestore(), respectively.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

PM-runtime: update time accounting only when enabled

Update the accounting_timestamp field only when PM runtime is enabled
and don't forget to account the last state before disabling it.

Suggested-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
[ rjw: Minor cleanups ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM-runtime: Switch accounting over to ktime_get_mono_fast_ns()

Similar to what happened whith autosuspend, a deadlock has been
reported with PM-runtime accounting in the call path:

change_clocksource
...
write_seqcount_begin
...
timekeeping_update
...
sh_cmt_clocksource_enable
...
rpm_resume
update_pm_runtime_accounting
ktime_get
do
read_seqcount_begin
while read_seqcount_retry
....
write_seqcount_end

Make PM-runtime accounting use ktime_get_mono_fast_ns() to avoid this
problem.

With ktime_get_mono_fast_ns(), the timestamp is not guaranteed to be
monotonic across an update of timekeeping and as a result time can go
backward. Add a test to skip accounting for such situation which should
stay exceptional.

Fixes: a08c2a5a3194 ("PM-runtime: Replace jiffies-based accounting with ktime-based accounting")
Reported-by: Biju Das <biju.das@bp.renesas.com>
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
[ rjw: Subject, changelog, comment cleanup ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM-runtime: Optimize pm_runtime_autosuspend_expiration()

pm_runtime_autosuspend_expiration calls ktime_get_mono_fast_ns()
even when its returned value may be unused. Therefore get the
current time later and remove gotos while there.

Signed-off-by: Ladislav Michl <ladis@linux-mips.org>
Acked-by: Tony Lindgren <tony@atomide.com>
Acked-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

driver core: Do not call rpm_put_suppliers() in pm_runtime_drop_link()

Calling rpm_put_suppliers() from pm_runtime_drop_link() is excessive
as it affects all suppliers of the consumer device and not just the
one pointed to by the device link being dropped. Worst case it may
cause the consumer device to stop working unexpectedly. Moreover, in
principle it is racy with respect to runtime PM of the consumer
device.

To avoid these problems drop runtime PM references on the particular
supplier pointed to by the link in question only and do that after
the link has been dropped from the consumer device's list of links to
suppliers, which is in device_link_free().

Fixes: a0504aecba76 ("PM / runtime: Drop usage count for suppliers at device link removal")
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

driver core: Fix handling of runtime PM flags in device_link_add()

After commit ead18c23c263 ("driver core: Introduce device links
reference counting"), if there is a link between the given supplier
and the given consumer already, device_link_add() will refcount it
and return it unconditionally without updating its flags. It is
possible, however, that the second (or any subsequent) caller of
device_link_add() for the same consumer-supplier pair will pass
DL_FLAG_PM_RUNTIME, possibly along with DL_FLAG_RPM_ACTIVE, in flags
to it and the existing link may not behave as expected then.

First, if DL_FLAG_PM_RUNTIME is not set in the existing link's flags
at all, it needs to be set like during the original initialization of
the link.

Second, if DL_FLAG_RPM_ACTIVE is passed to device_link_add() in flags
(in addition to DL_FLAG_PM_RUNTIME), the existing link should to be
updated to reflect the "active" runtime PM configuration of the
consumer-supplier pair and extra care must be taken here to avoid
possible destructive races with runtime PM of the consumer.

To that end, redefine the rpm_active field in struct device_link
as a refcount, initialize it to 1 and make rpm_resume() (for the
consumer) and device_link_add() increment it whenever they acquire
a runtime PM reference on the supplier device. Accordingly, make
rpm_suspend() (for the consumer) and pm_runtime_clean_up_links()
decrement it and drop runtime PM references to the supplier
device in a loop until rpm_active becones 1 again.

Fixes: ead18c23c263 ("driver core: Introduce device links reference counting")
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

PM-runtime: Replace jiffies-based accounting with ktime-based accounting

Replace jiffies-based accounting for runtime_active_time and
runtime_suspended_time with ktime-based accounting. This makes the
runtime debug counters inline with genpd and other PM subsytems which
use ktime-based accounting.

Timekeeping is initialized before driver_init(). It's only at that time
that PM-runtime can be enabled.

Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
[switch from ktime to raw nsec]
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM-runtime: update accounting_timestamp on enable

Initializing accounting_timestamp to something different from 0 during
pm_runtime_init() doesn't make sense and puts an artificial ordering
constraint between timekeeping_init() and pm_runtime_init().

PM-runtime should start time accounting only when it is enabled and
discard the period when disabled.

Set accounting_timestamp to now when enabling PM-runtime.

Suggested-by: "Rafael J. Wysocki" <rjw@rjwysocki.net>
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
[ rjw: Subject & changelog ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM-runtime: Fix deadlock with ktime_get()

A deadlock has been seen when swicthing clocksources which use
PM-runtime. The call path is:

change_clocksource
...
write_seqcount_begin
...
timekeeping_update
...
sh_cmt_clocksource_enable
...
rpm_resume
pm_runtime_mark_last_busy
ktime_get
do
read_seqcount_begin
while read_seqcount_retry
....
write_seqcount_end

Although we should be safe because we haven't yet changed the
clocksource at that time, we can't do that because of seqcount
protection.

Use ktime_get_mono_fast_ns() instead which is lock safe for such
cases.

With ktime_get_mono_fast_ns, the timestamp is not guaranteed to be
monotonic across an update and as a result can goes backward.
According to update_fast_timekeeper() description: "In the worst
case, this can result is a slightly wrong timestamp (a few
nanoseconds)". For PM-runtime autosuspend, this means only that
the suspend decision may be slightly suboptimal.

Fixes: 8234f6734c5d ("PM-runtime: Switch autosuspend over to using hrtimers")
Reported-by: Biju Das <biju.das@bp.renesas.com>
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM-runtime: Add new interface to get accounted time

Some drivers (like i915/drm) needs to get the accounted suspended time.
pm_runtime_suspended_time() will return the suspended accounted time
in ns unit.

Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM-runtime: Fix autosuspend_delay on 32bits arch

Cast autosuspend_delay to u64 to make sure that the full computation
of 'expires' or slack will be done in u64, even on 32bits arch.

Otherwise, any delay greater than 2^31 nsec can overflow if signed
32bits is used when converting delay from msec to nsec.

Fixes: 8234f6734c5d (PM-runtime: Switch autosuspend over to using hrtimers)
Reported-by: Tony Lindgren <tony@atomide.com>
Tested-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM-runtime: Fix 'jiffies' in comments after switch to hrtimers

PM-runtime now uses the hrtimers infrastructure for autosuspend, however
comments still reference 'jiffies'.

Fixes: 8234f6734c5d (PM-runtime: Switch autosuspend over to using hrtimers)
Signed-off-by: Ladislav Michl <ladis@linux-mips.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM-runtime: Switch autosuspend over to using hrtimers

PM-runtime uses the timer infrastructure for autosuspend. This implies
that the minimum time before autosuspending a device is in the range
of 1 tick included to 2 ticks excluded
-On arm64 this means between 4ms and 8ms with default jiffies
configuration
-And on arm, it is between 10ms and 20ms

These values are quite high for embedded systems which sometimes want
the duration to be in the range of 1 ms.

It is possible to switch autosuspend over to using hrtimers to get
finer granularity for short durations and take advantage of slack to
retain some margins and get long timeouts with minimum wakeups.

On an arm64 platform that uses 1ms for autosuspending timeout of its
GPU, idle power is reduced by 10% with hrtimer.

The latency impact on arm64 hikey octo cores is:
- mark_last_busy: from 1.11 us to 1.25 us
- rpm_suspend: from 15.54 us to 15.38 us
[Only the code path of rpm_suspend() that starts hrtimer has been
measured.]

arm64 image (arm64 default defconfig) decreases by around 3KB
with following details:

$ size vmlinux-timer
text data bss dec hex filename
12034646 6869268 386840 19290754 1265a82 vmlinux

$ size vmlinux-hrtimer
text data bss dec hex filename
12030550 6870164 387032 19287746 1264ec2 vmlinux

The latency impact on arm 32bits snowball dual cores is :
- mark_last_busy: from 0.31 us usec to 0.77 us
- rpm_suspend: from 6.83 us to 6.67 usec

The increase of the image for snowball platform that I used for
testing performance impact, is neglictable (244B).

$ size vmlinux-timer
text data bss dec hex filename
7157961 2119580 264120 9541661 91981d build-ux500/vmlinux

size vmlinux-hrtimer
text data bss dec hex filename
7157773 2119884 264248 9541905 919911 vmlinux-hrtimer

And arm 32bits image (multi_v7_defconfig) increases by around 1.7KB
with following details:

$ size vmlinux-timer
text data bss dec hex filename
13304443 6803420 402768 20510631 138f7a7 vmlinux

$ size vmlinux-hrtimer
text data bss dec hex filename
13304299 6805276 402768 20512343 138fe57 vmlinux

Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

Revert "PM / runtime: Fixup reference counting of device link suppliers at probe"

Revert commit 1e8378619841 (PM / runtime: Fixup reference counting of
device link suppliers at probe), as it has introduced a regression
and the condition it was designed to address should be covered by the
existing code.

Reported-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / runtime: Drop usage count for suppliers at device link removal

In the case consumer device is runtime resumed, while the link to the
supplier is removed, the earlier call to pm_runtime_get_sync() made from
rpm_get_suppliers() does not get properly balanced with a corresponding
call to pm_runtime_put(). This leads to that suppliers remains to be
runtime resumed forever, while they don't need to.

Let's fix the behaviour by calling rpm_put_suppliers() when dropping a
device link. Not that, since rpm_put_suppliers() checks the
link->rpm_active flag, we can correctly avoid to call pm_runtime_put() in
cases when we shouldn't.

Reported-by: Todor Tomov <todor.tomov@linaro.org>
Fixes: 21d5c57b3726 (PM / runtime: Use device links)
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / runtime: Fixup reference counting of device link suppliers at probe

In the driver core, before it invokes really_probe() it runtime resumes the
suppliers for the device via calling pm_runtime_get_suppliers(), which also
increases the runtime PM usage count for each of the available supplier.

This makes sense, as to be able to allow the consumer device to be probed
by its driver. However, if the driver decides to add a new supplier link
during ->probe(), hence updating the list of suppliers, the following call
to pm_runtime_put_suppliers(), invoked after really_probe() in the driver
core, we get into trouble.

More precisely, pm_runtime_put() gets called also for the new supplier(s),
which is wrong as the driver core, didn't trigger pm_runtime_get_sync() to
be called for it in the first place. In other words, the new supplier may
be runtime suspended even in cases when it shouldn't.

Fix this behaviour, by runtime resume suppliers according to the same
conditions as managed by the runtime PM core, when runtime resume callbacks
are being invoked.

Additionally, don't try to runtime suspend any of the suppliers after
really_probe(), but instead rely on that to happen via the consumer device,
when it becomes runtime suspended.

Fixes: 21d5c57b3726 (PM / runtime: Use device links)
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / runtime: Allow no callbacks in pm_runtime_force_suspend|resume()

The pm_runtime_force_suspend|resume() helpers currently requires the device
to at some level (PM domain, bus, etc), have the ->runtime_suspend|resume()
callbacks assigned for it, else -ENOSYS is returned as an error.

However, there are no reason for this requirement, so let's simply remove
it by allowing these callbacks to be NULL.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / runtime: Check ignore_children in pm_runtime_need_not_resume()

Modify pm_runtime_need_not_resume() to make it avoid taking
power.child_count for devices with power.ignore_children which
is consistent with the runtime PM usage of these fields.

Suggested-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

PM / runtime: Rework pm_runtime_force_suspend/resume()

One of the limitations of pm_runtime_force_suspend/resume() is that
if a parent driver wants to use these functions, all of its child
drivers generally have to do that too because of the parent usage
counter manipulations necessary to get the correct state of the parent
during system-wide transitions to the working state (system resume).
However, that limitation turns out to be artificial, so remove it.

Namely, pm_runtime_force_suspend() only needs to update the children
counter of its parent (if there's is a parent) when the device can
stay in suspend after the subsequent system resume transition, as
that counter is correct already otherwise. Now, if the parent's
children counter is not updated, it is not necessary to increment
the parent's usage counter in that case any more, as long as the
children counters of devices are checked along with their usage
counters in order to decide whether or not the devices may be left
in suspend after the subsequent system resume transition.

Accordingly, modify pm_runtime_force_suspend() to only call
pm_runtime_set_suspended() for devices whose usage and children
counters are at the "no references" level (the runtime PM status
of the device needs to be updated to "suspended" anyway in case
this function is called once again for the same device during the
transition under way), drop the parent usage counter incrementation
from it and update pm_runtime_force_resume() to compensate for these
changes.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

PM / runtime: Fix handling of suppliers with disabled runtime PM

Prevent rpm_get_suppliers() from returning an error code if runtime
PM is disabled for one or more of the supplier devices it wants to
runtime-resume, so as to make runtime PM work for devices with links
to suppliers that don't use runtime PM (such links may be created
during device enumeration even before it is known whether or not
runtime PM will be enabled for the devices in question, for example).

Fixes: 21d5c57b3726 (PM / runtime: Use device links)
Reported-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Lukas Wunner <lukas@wunner.de>
Tested-by: Adrian Hunter <adrian.hunter@intel.com>

PM / runtime: Drop children check from __pm_runtime_set_status()

The check for "active" children in __pm_runtime_set_status(), when
trying to set the parent device status to "suspended", doesn't
really make sense, because in fact it is not invalid to set the
status of a device with runtime PM disabled to "suspended" in any
case. It is invalid to enable runtime PM for a device with its
status set to "suspended" while its child_count reference counter
is nonzero, but the check in __pm_runtime_set_status() doesn't
really cover that situation.

For this reason, drop the children check from __pm_runtime_set_status()
and add a check against child_count reference counters of "suspended"
devices to pm_runtime_enable().

Fixes: a8636c89648a (PM / Runtime: Don't allow to suspend a device with an active child)
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Johan Hovold <johan@kernel.org>

PM / QoS: Fix device resume latency framework

The special value of 0 for device resume latency PM QoS means
"no restriction", but there are two problems with that.

First, device resume latency PM QoS requests with 0 as the
value are always put in front of requests with positive
values in the priority lists used internally by the PM QoS
framework, causing 0 to be chosen as an effective constraint
value. However, that 0 is then interpreted as "no restriction"
effectively overriding the other requests with specific
restrictions which is incorrect.

Second, the users of device resume latency PM QoS have no
way to specify that *any* resume latency at all should be
avoided, which is an artificial limitation in general.

To address these issues, modify device resume latency PM QoS to
use S32_MAX as the "no constraint" value and 0 as the "no
latency at all" one and rework its users (the cpuidle menu
governor, the genpd QoS governor and the runtime PM framework)
to follow these changes.

Also add a special "n/a" value to the corresponding user space I/F
to allow user space to indicate that it cannot accept any resume
latencies at all for the given device.

Fixes: 85dc0b8a4019 (PM / QoS: Make it possible to expose PM QoS latency constraints)
Link: https://bugzilla.kernel.org/show_bug.cgi?id=197323
Reported-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Tested-by: Reinette Chatre <reinette.chatre@intel.com>
Tested-by: Geert Uytterhoeven <geert+renesas@glider.be>
Tested-by: Tero Kristo <t-kristo@ti.com>
Reviewed-by: Ramesh Thomas <ramesh.thomas@intel.com>

Revert "PM / QoS: Fix device resume latency PM QoS"

This reverts commit 0cc2b4e5a020 (PM / QoS: Fix device resume latency PM
QoS) as it introduced regressions on multiple systems and the fix-up
in commit 2a9a86d5c813 (PM / QoS: Fix default runtime_pm device resume
latency) does not address all of them.

The original problem that commit 0cc2b4e5a020 was attempting to fix
will be addressed later.

Fixes: 0cc2b4e5a020 (PM / QoS: Fix device resume latency PM QoS)
Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

locking/atomics: COCCINELLE/treewide: Convert trivial ACCESS_ONCE() patterns to READ_ONCE()/WRITE_ONCE()

Please do not apply this to mainline directly, instead please re-run the
coccinelle script shown below and apply its output.

For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't harmful, and changing them results in
churn.

However, for some features, the read/write distinction is critical to
correct operation. To distinguish these cases, separate read/write
accessors must be used. This patch migrates (most) remaining
ACCESS_ONCE() instances to {READ,WRITE}_ONCE(), using the following
coccinelle script:

----
// Convert trivial ACCESS_ONCE() uses to equivalent READ_ONCE() and
// WRITE_ONCE()

// $ make coccicheck COCCI=/home/mark/once.cocci SPFLAGS="--include-headers" MODE=patch

virtual patch

@ depends on patch @
expression E1, E2;
@@

- ACCESS_ONCE(E1) = E2
+ WRITE_ONCE(E1, E2)

@ depends on patch @
expression E;
@@

- ACCESS_ONCE(E)
+ READ_ONCE(E)
----

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: davem@davemloft.net
Cc: linux-arch@vger.kernel.org
Cc: mpe@ellerman.id.au
Cc: shuah@kernel.org
Cc: snitzer@redhat.com
Cc: thor.thayer@linux.intel.com
Cc: tj@kernel.org
Cc: viro@zeniv.linux.org.uk
Cc: will.deacon@arm.com
Link: http://lkml.kernel.org/r/1508792849-3115-19-git-send-email-paulmck@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>

PM / QoS: Fix device resume latency PM QoS

The special value of 0 for device resume latency PM QoS means
"no restriction", but there are two problems with that.

First, device resume latency PM QoS requests with 0 as the
value are always put in front of requests with positive
values in the priority lists used internally by the PM QoS
framework, causing 0 to be chosen as an effective constraint
value. However, that 0 is then interpreted as "no restriction"
effectively overriding the other requests with specific
restrictions which is incorrect.

Second, the users of device resume latency PM QoS have no
way to specify that *any* resume latency at all should be
avoided, which is an artificial limitation in general.

To address these issues, modify device resume latency PM QoS to
use S32_MAX as the "no constraint" value and 0 as the "no
latency at all" one and rework its users (the cpuidle menu
governor, the genpd QoS governor and the runtime PM framework)
to follow these changes.

Also add a special "n/a" value to the corresponding user space I/F
to allow user space to indicate that it cannot accept any resume
latencies at all for the given device.

Fixes: 85dc0b8a4019 (PM / QoS: Make it possible to expose PM QoS latency constraints)
Link: https://bugzilla.kernel.org/show_bug.cgi?id=197323
Reported-by: Reinette Chatre <reinette.chatre@intel.com>
Tested-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Alex Shi <alex.shi@linaro.org>
Cc: All applicable <stable@vger.kernel.org>

PM / core: Convert timers to use timer_setup()

In preparation for unconditionally passing the struct timer_list pointer to
all timer callbacks, switch to using the new timer_setup() and from_timer()
to pass the timer pointer explicitly. Removes test of .data field, since
that will be going away.

Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

sched/headers: Prepare to move the memalloc_noio_*() APIs to <linux/sched/mm.h>

Update the .c files that depend on these APIs.

Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>

PM / runtime: Avoid false-positive warnings from might_sleep_if()

The might_sleep_if() assertions in __pm_runtime_idle(),
__pm_runtime_suspend() and __pm_runtime_resume() may generate
false-positive warnings in some situations. For example, that
happens if a nested pm_runtime_get_sync()/pm_runtime_put() pair
is executed with disabled interrupts within an outer
pm_runtime_get_sync()/pm_runtime_put() section for the same device.
[Generally, pm_runtime_get_sync() may sleep, so it should not be
called with disabled interrupts, but in this particular case the
previous pm_runtime_get_sync() guarantees that the device will not
be suspended, so the inner pm_runtime_get_sync() will return
immediately after incrementing the device's usage counter.]

That started to happen in the i915 driver in 4.10-rc, leading to
the following splat:

BUG: sleeping function called from invalid context at drivers/base/power/runtime.c:1032
in_atomic(): 1, irqs_disabled(): 0, pid: 1500, name: Xorg
1 lock held by Xorg/1500:
#0: (&dev->struct_mutex){+.+.+.}, at:
[<ffffffffa0680c13>] i915_mutex_lock_interruptible+0x43/0x140 [i915]
CPU: 0 PID: 1500 Comm: Xorg Not tainted
Call Trace:
dump_stack+0x85/0xc2
___might_sleep+0x196/0x260
__might_sleep+0x53/0xb0
__pm_runtime_resume+0x7a/0x90
intel_runtime_pm_get+0x25/0x90 [i915]
aliasing_gtt_bind_vma+0xaa/0xf0 [i915]
i915_vma_bind+0xaf/0x1e0 [i915]
i915_gem_execbuffer_relocate_entry+0x513/0x6f0 [i915]
i915_gem_execbuffer_relocate_vma.isra.34+0x188/0x250 [i915]
? trace_hardirqs_on+0xd/0x10
? i915_gem_execbuffer_reserve_vma.isra.31+0x152/0x1f0 [i915]
? i915_gem_execbuffer_reserve.isra.32+0x372/0x3a0 [i915]
i915_gem_do_execbuffer.isra.38+0xa70/0x1a40 [i915]
? __might_fault+0x4e/0xb0
i915_gem_execbuffer2+0xc5/0x260 [i915]
? __might_fault+0x4e/0xb0
drm_ioctl+0x206/0x450 [drm]
? i915_gem_execbuffer+0x340/0x340 [i915]
? __fget+0x5/0x200
do_vfs_ioctl+0x91/0x6f0
? __fget+0x111/0x200
? __fget+0x5/0x200
SyS_ioctl+0x79/0x90
entry_SYSCALL_64_fastpath+0x23/0xc6

even though the code triggering it is correct.

Unfortunately, the might_sleep_if() assertions in question are
too coarse-grained to cover such cases correctly, so make them
a bit less sensitive in order to avoid the false-positives.

Reported-and-tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / wakeirq: Fix dedicated wakeirq for drivers not using autosuspend

I noticed some wakeirq flakeyness with consumer drivers not using
autosuspend. For drivers not using autosuspend, the wakeirq may never
get unmasked in rpm_suspend() because of irq desc->depth.

We are configuring dedicated wakeirqs to start with IRQ_NOAUTOEN as we
naturally don't want them running until rpm_suspend() is called.

However, when a consumer driver initially calls pm_runtime_get(), we
now wrongly start with disable_irq_nosync() call on the dedicated
wakeirq that is disabled to start with.

This causes desc->depth to toggle between 1 and 2 instead of the usual
0 and 1. This can prevent enable_irq() from unmasking the wakeirq as
that only happens at desc->depth 1.

This does not necessarily show up with drivers using autosuspend as
there is time for disable_irq_nosync() before rpm_suspend() gets called
after the autosuspend timeout.

Let's fix the issue by adding wirq->status that lazily gets set on
the first rpm_suspend(). We also need PM runtime core private functions
for dev_pm_enable_wake_irq_check() and dev_pm_disable_wake_irq_check()
so we can enable the dedicated wakeirq on the first rpm_suspend().

While at it, let's also fix the comments for dev_pm_enable_wake_irq()
and dev_pm_disable_wake_irq(). Those can still be used by the consumer
drivers as needed because the IRQ core manages the interrupt usecount
for us.

Fixes: 4990d4fe327b (PM / Wakeirq: Add automated device wake IRQ handling)
Signed-off-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / Runtime: Defer resuming of the device in pm_runtime_force_resume()

When the pm_runtime_force_suspend|resume() helpers were invented, we still
had CONFIG_PM_RUNTIME and CONFIG_PM_SLEEP as separate Kconfig options.

To make sure these helpers worked for all combinations and without
introducing too much of complexity, the device was always resumed in
pm_runtime_force_resume().

More precisely, when CONFIG_PM_SLEEP was set and CONFIG_PM_RUNTIME was
unset, we needed to resume the device as the subsystem/driver couldn't
rely on using runtime PM to do it.

As the CONFIG_PM_RUNTIME option was merged into CONFIG_PM a while ago, it
removed this combination, of using CONFIG_PM_SLEEP without the earlier
CONFIG_PM_RUNTIME.

For this reason we can now rely on the subsystem/driver to use runtime PM
to resume the device, instead of forcing that to be done in all cases. In
other words, let's defer the runtime resume to a later point when it's
actually needed.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Tested-by: Geert Uytterhoeven <geert+renesas@glider.be>
Acked-by: Kevin Hilman <khilman@baylibre.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / Runtime: Don't allow to suspend a device with an active child

When resuming a device in __pm_runtime_set_status(), the prerequisite is
that its parent must already be active, else an error code is returned and
the device's status remains suspended.

When suspending a device there is no similar constraints being validated.
Let's change this to make the behaviour consistent, by not allowing to
suspend a device with an active child, unless it has been explicitly set to
ignore its children.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / runtime: Optimize the use of device links

If the device has no links to suppliers that should be used for
runtime PM (links with DEVICE_LINK_PM_RUNTIME set), there is no
reason to walk the list of suppliers for that device during
runtime suspend and resume.

Add a simple mechanism to detect that case and possibly avoid the
extra unnecessary overhead.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

PM / runtime: Use device links

Modify the runtime PM framework to use device links to ensure that
supplier devices will not be suspended if any of their consumer
devices are active.

The idea is to reference count suppliers on the consumer's resume
and drop references to them on its suspend. The information on
whether or not the supplier has been reference counted by the
consumer's (runtime) resume is stored in a new field (rpm_active)
in the link object for each link.

It may be necessary to clean up those references when the
supplier is unbinding and that's why the links whose status is
DEVICE_LINK_SUPPLIER_UNBIND are skipped by the runtime suspend
and resume code.

The above means that if the consumer device is probed in the
runtime-active state, the supplier has to be resumed and reference
counted by device_link_add() so the code works as expected on its
(runtime) suspend. There is a new flag, DEVICE_LINK_RPM_ACTIVE,
to tell device_link_add() about that (in which case the caller
is responsible for making sure that the consumer really will
be runtime-active when runtime PM is enabled for it).

The other new link flag, DEVICE_LINK_PM_RUNTIME, tells the core
whether or not the link should be used for runtime PM at all.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

PM / Runtime: Clarify comment in rpm_resume() when resuming the parent

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / Runtime: Remove the exported function pm_children_suspended()

The exported function pm_children_suspended() has only one caller, which is
the runtime PM internal function, rpm_check_suspend_allowed().

Let's clean-up this code, by removing pm_children_suspended() altogether
and instead do the one-liner check directly in rpm_check_suspend_allowed().

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / runtime: Use _rcuidle for runtime suspend tracepoints

Further testing with false negatives suppressed by commit 293e2421fe25
("rcu: Remove superfluous versions of rcu_read_lock_sched_held()")
identified a few more unprotected uses of RCU from the idle loop.
Because RCU actively ignores idle-loop code (for energy-efficiency
reasons, among other things), using RCU from the idle loop can result
in too-short grace periods, in turn resulting in arbitrary misbehavior.

The affected function is rpm_suspend().

The resulting lockdep-RCU splat is as follows:

------------------------------------------------------------------------

Warning from omap3

===============================
[ INFO: suspicious RCU usage. ]
4.6.0-rc5-next-20160426+ #1112 Not tainted
-------------------------------
include/trace/events/rpm.h:63 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from idle CPU!
rcu_scheduler_active = 1, debug_locks = 0
RCU used illegally from extended quiescent state!
1 lock held by swapper/0/0:
#0: (&(&dev->power.lock)->rlock){-.-...}, at: [<c052ee24>] __pm_runtime_suspend+0x54/0x84

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6.0-rc5-next-20160426+ #1112
Hardware name: Generic OMAP36xx (Flattened Device Tree)
[<c0110308>] (unwind_backtrace) from [<c010c3a8>] (show_stack+0x10/0x14)
[<c010c3a8>] (show_stack) from [<c047fec8>] (dump_stack+0xb0/0xe4)
[<c047fec8>] (dump_stack) from [<c052d7b4>] (rpm_suspend+0x604/0x7e4)
[<c052d7b4>] (rpm_suspend) from [<c052ee34>] (__pm_runtime_suspend+0x64/0x84)
[<c052ee34>] (__pm_runtime_suspend) from [<c04bf3bc>] (omap2_gpio_prepare_for_idle+0x5c/0x70)
[<c04bf3bc>] (omap2_gpio_prepare_for_idle) from [<c01255e8>] (omap_sram_idle+0x140/0x244)
[<c01255e8>] (omap_sram_idle) from [<c0126b48>] (omap3_enter_idle_bm+0xfc/0x1ec)
[<c0126b48>] (omap3_enter_idle_bm) from [<c0601db8>] (cpuidle_enter_state+0x80/0x3d4)
[<c0601db8>] (cpuidle_enter_state) from [<c0183c74>] (cpu_startup_entry+0x198/0x3a0)
[<c0183c74>] (cpu_startup_entry) from [<c0b00c0c>] (start_kernel+0x354/0x3c8)
[<c0b00c0c>] (start_kernel) from [<8000807c>] (0x8000807c)

------------------------------------------------------------------------

Reported-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Tony Lindgren <tony@atomide.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
[ rjw: Subject ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / runtime: Add _rcuidle suffix to allow rpm_idle() use from idle

This commit appends a few _rcuidle suffixes to fix the following
RCU-used-from-idle bug:

> ===============================
> [ INFO: suspicious RCU usage. ]
> 4.6.0-rc5-next-20160426+ #1116 Not tainted
> -------------------------------
> include/trace/events/rpm.h:95 suspicious rcu_dereference_check() usage!
>
> other info that might help us debug this:
>
>
> RCU used illegally from idle CPU!
> rcu_scheduler_active = 1, debug_locks = 0
> RCU used illegally from extended quiescent state!
> 1 lock held by swapper/0/0:
> #0: (&(&dev->power.lock)->rlock){-.-...}, at: [<c052cc2c>] __rpm_callback+0x58/0x60
>
> stack backtrace:
> CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6.0-rc5-next-20160426+ #1116
> Hardware name: Generic OMAP36xx (Flattened Device Tree)
> [<c0110290>] (unwind_backtrace) from [<c010c3a8>] (show_stack+0x10/0x14)
> [<c010c3a8>] (show_stack) from [<c047fd68>] (dump_stack+0xb0/0xe4)
> [<c047fd68>] (dump_stack) from [<c052d5d0>] (rpm_suspend+0x580/0x768)
> [<c052d5d0>] (rpm_suspend) from [<c052ec58>] (__pm_runtime_suspend+0x64/0x84)
> [<c052ec58>] (__pm_runtime_suspend) from [<c04bf25c>] (omap2_gpio_prepare_for_idle+0x5c/0x70)
> [<c04bf25c>] (omap2_gpio_prepare_for_idle) from [<c0125568>] (omap_sram_idle+0x140/0x244)
> [<c0125568>] (omap_sram_idle) from [<c01269dc>] (omap3_enter_idle_bm+0xfc/0x1ec)
> [<c01269dc>] (omap3_enter_idle_bm) from [<c0601bdc>] (cpuidle_enter_state+0x80/0x3d4)
> [<c0601bdc>] (cpuidle_enter_state) from [<c0183b08>] (cpu_startup_entry+0x198/0x3a0)
> [<c0183b08>] (cpu_startup_entry) from [<c0b00c0c>] (start_kernel+0x354/0x3c8)
> [<c0b00c0c>] (start_kernel) from [<8000807c>] (0x8000807c)

In the immortal words of Steven Rostedt, "*Whack* *Whack* *Whack*!!!"

Reported-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Tony Lindgren <tony@atomide.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
WhACKED-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / runtime: Add _rcuidle suffix to allow rpm_resume() to be called from idle

This commit applies another _rcuidle suffix to fix an RCU use from
idle.

> ===============================
> [ INFO: suspicious RCU usage. ]
> 4.6.0-rc5-next-20160426+ #1122 Not tainted
> -------------------------------
> include/trace/events/rpm.h:69 suspicious rcu_dereference_check() usage!
>
> other info that might help us debug this:
>
>
> RCU used illegally from idle CPU!
> rcu_scheduler_active = 1, debug_locks = 0
> RCU used illegally from extended quiescent state!
> 1 lock held by swapper/0/0:
> #0: (&(&dev->power.lock)->rlock){-.-...}, at: [<c052e3dc>] __pm_runtime_resume+0x3c/0x64
>
> stack backtrace:
> CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6.0-rc5-next-20160426+ #1122
> Hardware name: Generic OMAP36xx (Flattened Device Tree)
> [<c0110290>] (unwind_backtrace) from [<c010c3a8>] (show_stack+0x10/0x14)
> [<c010c3a8>] (show_stack) from [<c047fd68>] (dump_stack+0xb0/0xe4)
> [<c047fd68>] (dump_stack) from [<c052e178>] (rpm_resume+0x5cc/0x7f4)
> [<c052e178>] (rpm_resume) from [<c052e3ec>] (__pm_runtime_resume+0x4c/0x64)
> [<c052e3ec>] (__pm_runtime_resume) from [<c04bf2c4>] (omap2_gpio_resume_after_idle+0x54/0x68)
> [<c04bf2c4>] (omap2_gpio_resume_after_idle) from [<c01269dc>] (omap3_enter_idle_bm+0xfc/0x1ec)
> [<c01269dc>] (omap3_enter_idle_bm) from [<c060198c>] (cpuidle_enter_state+0x80/0x3d4)
> [<c060198c>] (cpuidle_enter_state) from [<c0183b08>] (cpu_startup_entry+0x198/0x3a0)
> [<c0183b08>] (cpu_startup_entry) from [<c0b00c0c>] (start_kernel+0x354/0x3c8)
> [<c0b00c0c>] (start_kernel) from [<8000807c>] (0x8000807c)

Reported-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Tony Lindgren <tony@atomide.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / runtime: Asynchronous "idle" in pm_runtime_allow()

Arjan reports that it takes a relatively long time to enable runtime
PM for multiple devices at system startup, because all writes to the
"control" attribute in sysfs are handled synchronously and if the
device is suspended as a result of the write, it will block until
that operation is complete.

That may be avoided by passing the RPM_ASYNC flag to rpm_idle()
in pm_runtime_allow() which will make it execute the device's
"idle" callback asynchronously, so writes to "control" changing
it from "on" to "auto" will return without waiting.

Reported-by: Arjan van de Ven <arjan@linux.intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Kevin Hilman <khilman@baylibre.com>

PM / runtime: print error when activating a child to unactive parent

The code currently silently bails out with -EBUSY if you try to
activate a child to an inactive parent.

This typically happens when you have a runtime suspended parent
and runtime resume your child, but forgot to set .ignore_children
on the parent to true with pm_suspend_ignore_children(dev).

Silently ignoring this error is not good as it gives rise to
other strange behaviour like double-resume of devices after
silently bailing out of the .runtime_resume() callback.

Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / Runtime: Avoid resuming devices again in pm_runtime_force_resume()

If the runtime PM status of the device isn't RPM_SUSPENDED, prevent the
pm_runtime_force_resume() from invoking the ->runtime_resume() callback
for the device, as it's not the expected behaviour from the subsystem/driver.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Kevin Hilman <khilman@baylibre.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / Runtime: Fix error path in pm_runtime_force_resume()

As pm_runtime_set_active() may fail because the device's parent isn't
active, we can end up executing the ->runtime_resume() callback for the
device when it isn't allowed.

Fix this by invoking pm_runtime_set_active() before running the callback
and let's also deal with the error code.

Fixes: 37f204164dfb (PM: Add pm_runtime_suspend|resume_force functions)
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Cc: 3.15+ <stable@vger.kernel.org> # 3.15+
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / runtime: Add new helper for conditional usage count incrementation

Introduce a new runtime PM function, pm_runtime_get_if_in_use(),
that will increment the device's runtime PM usage counter and
return 1 if its status is RPM_ACTIVE and its usage counter
is greater than 0 at the same time (0 will be returned otherwise).

This is useful for things that should only be done if the device
is active (from the runtime PM perspective) and used by somebody
(as indicated by the usage counter) already and they are not worth
bothering otherwise.

Requested-by: Imre Deak <imre.deak@intel.com>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / runtime: Re-init runtime PM states at probe error and driver unbind

There are two common expectations among several subsystems/drivers that
deploys runtime PM support, but which isn't met by the driver core.

Expectation 1)
At ->probe() the subsystem/driver expects the runtime PM status of the
device to be RPM_SUSPENDED, which is the initial status being assigned at
device registration.

This expectation is especially common among some of those subsystems/
drivers that manages devices with an attached PM domain, as those requires
the ->runtime_resume() callback at the PM domain level to be invoked
during ->probe().

Moreover these subsystems/drivers entirely relies on runtime PM resources
being managed at the PM domain level, thus don't implement their own set
of runtime PM callbacks.

These are two scenarios that suffers from this unmet expectation.

i) A failed ->probe() sequence requests probe deferral:

->probe()
...
pm_runtime_enable()
pm_runtime_get_sync()
...

err:
pm_runtime_put()
pm_runtime_disable()
...

As there are no guarantees that such sequence turns the runtime PM status
of the device into RPM_SUSPENDED, the re-trying ->probe() may start with
the status in RPM_ACTIVE.

In such case the runtime PM core won't invoke the ->runtime_resume()
callback because of a pm_runtime_get_sync(), as it considers the device to
be already runtime resumed.

ii) A driver re-bind sequence:

At driver unbind, the subsystem/driver's >remove() callback invokes a
sequence of runtime PM APIs, to undo actions during ->probe() and to put
the device into low power state.

->remove()
...
pm_runtime_put()
pm_runtime_disable()
...

Similar as in the failing ->probe() case, this sequence don't guarantee
the runtime PM status of the device to turn into RPM_SUSPENDED.

Trying to re-bind the driver thus causes the same issue as when re-trying
->probe(), in the probe deferral scenario.

Expectation 2)
Drivers that invokes the pm_runtime_irq_safe() API during ->probe(),
triggers the runtime PM core to increase the usage count for the device's
parent and permanently make it runtime resumed.

The usage count is only dropped at device removal, which also allows it to
be runtime suspended again.

A re-trying ->probe() repeats the call to pm_runtime_irq_safe() and thus
once more triggers the usage count of the device's parent to be increased.

This leads to not only an imbalance issue of the usage count of the
device's parent, but also to keep it runtime resumed permanently even if
->probe() fails.

To address these issues, let's change the policy of the driver core to
meet these expectations. More precisely, at ->probe() failures and driver
unbind, restore the initial states of runtime PM.

Although to still allow subsystem's to control PM for devices that doesn't
->probe() successfully, don't restore the initial states unless runtime PM
is disabled.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Kevin Hilman <khilman@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / Wakeirq: Add automated device wake IRQ handling

Turns out we can automate the handling for the device_may_wakeup()
quite a bit by using the kernel wakeup source list as suggested
by Rafael J. Wysocki <rjw@rjwysocki.net>.

And as some hardware has separate dedicated wake-up interrupt
in addition to the IO interrupt, we can automate the handling by
adding a generic threaded interrupt handler that just calls the
device PM runtime to wake up the device.

This allows dropping code from device drivers as we currently
are doing it in multiple ways, and often wrong.

For most drivers, we should be able to drop the following
boilerplate code from runtime_suspend and runtime_resume
functions:

...
device_init_wakeup(dev, true);
...
if (device_may_wakeup(dev))
enable_irq_wake(irq);
...
if (device_may_wakeup(dev))
disable_irq_wake(irq);
...
device_init_wakeup(dev, false);
...

We can replace it with just the following init and exit
time code:

...
device_init_wakeup(dev, true);
dev_pm_set_wake_irq(dev, irq);
...
dev_pm_clear_wake_irq(dev);
device_init_wakeup(dev, false);
...

And for hardware with dedicated wake-up interrupts:

...
device_init_wakeup(dev, true);
dev_pm_set_dedicated_wake_irq(dev, irq);
...
dev_pm_clear_wake_irq(dev);
device_init_wakeup(dev, false);
...

Signed-off-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / Runtime: Update last_busy in rpm_resume

If we don't update last_busy in rpm_resume, devices can go back
to sleep immediately after resume. This happens at least in
cases where the device has been powered off and does not have
any interrupt pending until there's something in the FIFO.

Signed-off-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: Drop CONFIG_PM_RUNTIME from the driver core

After commit b2b49ccbdd54 (PM: Kconfig: Set PM_RUNTIME if PM_SLEEP is
selected) PM_RUNTIME is always set if PM is set, so quite a few
depend on CONFIG_PM or even may be dropped entirely in some cases.

Replace CONFIG_PM_RUNTIME with CONFIG_PM in the PM core code.

Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Acked-by: Kevin Hilman <khilman@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / Runtime: Rework RPM get callback routines

PM uses three separate functions to fetch RPM callbacks.
These functions uses quite complicated macro in their body.
The patch replaces these routines with one small macro and
one helper function.

Signed-off-by: Andrzej Hajda <a.hajda@samsung.com>
Acked-by: Pavel Machek <pavel@ucw.cz>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Acked-by: Kevin Hilman <khilman@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / Runtime: Add missing "it" in comment

Signed-off-by: Geert Uytterhoeven <geert+renesas@linux-m68k.org>
Acked-by: Pavel Machek <pavel@ucw.cz>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM: Add pm_runtime_suspend|resume_force functions

This patch provides two new runtime PM helper functions which intend to
be used from system suspend/resume callbacks, to make sure devices are
put into low power state during system suspend and brought back to full
power at system resume.

The prerequisite is to have all levels of a device's runtime PM
callbacks to be defined through the SET_PM_RUNTIME_PM_OPS macro, which
means these are available for CONFIG_PM.

By using the new runtime PM helper functions especially the two
scenarios below will be addressed.

1) The PM core prevents .runtime_suspend callbacks from being invoked
during system suspend. That means even for a runtime PM centric
subsystem and driver, the device needs to be put into low power state
from a system suspend callback. Otherwise it may very well be left in
full power state (runtime resumed) while the system is suspended. By
using the new helper functions, we make sure to walk the hierarchy of
a device's power domain, subsystem and driver.

2) Subsystems and drivers need to cope with all the combinations of
CONFIG_PM_SLEEP and CONFIG_PM_RUNTIME. The two new helper functions
smothly addresses this.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / runtime: Fetch runtime PM callbacks using a macro

While fetching the proper runtime PM callback, we walk the hierarchy of
device's power domains, subsystems and drivers.

This is common for rpm_suspend(), rpm_idle() and rpm_resume(). Let's
clean up the code by using a macro that handles this.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / Runtime: Respect autosuspend when idle triggers suspend

For devices which don't have a .runtime_idle() callback or if it
returns 0, rpm_idle() will end up in triggering a call to
rpm_suspend(), thus trying to carry out a runtime suspend directly
from runtime_idle().

In the above situation we want to respect devices which has enabled
autosuspend, we therfore append the flag sent to rpm_suspend with
RPM_AUTO.

Do note that drivers still needs to update the device last busy mark,
to control the delay for this circumstance.

Updated runtime PM documentation accordingly.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Acked-by: Kevin Hilman <khilman@linaro.org>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / Runtime: Rework the "runtime idle" helper routine

The "runtime idle" helper routine, rpm_idle(), currently ignores
return values from .runtime_idle() callbacks executed by it.
However, it turns out that many subsystems use
pm_generic_runtime_idle() which checks the return value of the
driver's callback and executes pm_runtime_suspend() for the device
unless that value is not 0. If that logic is moved to rpm_idle()
instead, pm_generic_runtime_idle() can be dropped and its users
will not need any .runtime_idle() callbacks any more.

Moreover, the PCI, SCSI, and SATA subsystems' .runtime_idle()
routines, pci_pm_runtime_idle(), scsi_runtime_idle(), and
ata_port_runtime_idle(), respectively, as well as a few drivers'
ones may be simplified if rpm_idle() calls rpm_suspend() after 0 has
been returned by the .runtime_idle() callback executed by it.

To reduce overall code bloat, make the changes described above.

Tested-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Tested-by: Kevin Hilman <khilman@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Kevin Hilman <khilman@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Acked-by: Alan Stern <stern@rowland.harvard.edu>

PM / Runtime: Asyncronous idle|suspend parent devices at removal

For irq safe devices return the runtime reference for the parent
by using the asyncronous runtime PM API. Thus we don't have to
wait for it to become idle|suspended. Instead we can move on and
handle the next device in queue.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

pm / runtime: force memory allocation with no I/O during Runtime PM callbcack

Apply the introduced memalloc_noio_save() and memalloc_noio_restore() to
force memory allocation with no I/O during runtime_resume/runtime_suspend
callback on device with the flag of 'memalloc_noio' set.

Signed-off-by: Ming Lei <ming.lei@canonical.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: David Decotigny <david.decotigny@google.com>
Cc: Tom Herbert <therbert@google.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Oliver Neukum <oneukum@suse.de>
Cc: Jiri Kosina <jiri.kosina@suse.com>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: "Rafael J. Wysocki" <rjw@sisk.pl>
Cc: Greg KH <greg@kroah.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

pm / runtime: introduce pm_runtime_set_memalloc_noio()

Introduce the flag memalloc_noio in 'struct dev_pm_info' to help PM core
to teach mm not allocating memory with GFP_KERNEL flag for avoiding
probable deadlock.

As explained in the comment, any GFP_KERNEL allocation inside
runtime_resume() or runtime_suspend() on any one of device in the path
from one block or network device to the root device in the device tree
may cause deadlock, the introduced pm_runtime_set_memalloc_noio() sets
or clears the flag on device in the path recursively.

Signed-off-by: Ming Lei <ming.lei@canonical.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Oliver Neukum <oneukum@suse.de>
Cc: Jiri Kosina <jiri.kosina@suse.com>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: "Rafael J. Wysocki" <rjw@sisk.pl>
Cc: Greg KH <greg@kroah.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: David Decotigny <david.decotigny@google.com>
Cc: Tom Herbert <therbert@google.com>
Cc: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

PM / Runtime: let rpm_resume() succeed if RPM_ACTIVE, even when disabled, v2

There are several drivers where the return value of
pm_runtime_get_sync() is used to decide whether or not it is safe to
access hardware and that don't provide .suspend() callbacks for system
suspend (but may use late/noirq callbacks.) If such a driver happens
to call pm_runtime_get_sync() during system suspend, after the core
has disabled runtime PM, it will get the error code and will decide
that the hardware should not be accessed, although this may be a wrong
conclusion, depending on the state of the device when runtime PM was
disabled.

Drivers might work around this problem by using a test like:

ret = pm_runtime_get_sync(dev);
if (!ret || (ret == -EACCES && driver_private_data(dev)->suspended)) {
/* access hardware */
}

where driver_private_data(dev)->suspended is a flag set by the
driver's .suspend() method (that would have to be added for this
purpose). However, that potentially would need to be done by multiple
drivers which means quite a lot of duplicated code and bloat.

To avoid that we can use the observation that the core sets
dev->power.is_suspended before disabling runtime PM and use that
instead of the driver's private flag. Still, potentially many drivers
would need to repeat that same check in quite a few places, so it's
better to let the core do it.

Then we can be a bit smarter and check whether or not runtime PM was
disabled by the core only (disable_depth == 1) or by someone else in
addition to the core (disable_depth > 1). In the former case
rpm_resume() can return 1 if the runtime PM status is RPM_ACTIVE,
because it means the device was active when the core disabled runtime
PM. In the latter case it should still return -EACCES, because it
isn't clear why runtime PM has been disabled.

Tested on AM3730/Beagle-xM where a wakeup IRQ firing during the late
suspend phase triggers runtime PM activity in the I2C driver since the
wakeup IRQ is on an I2C-connected PMIC.

[rjw: Modified whitespace to follow the file's convention.]

Signed-off-by: Kevin Hilman <khilman@ti.com>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM: Do not use the syscore flag for runtime PM

The syscore device PM flag used to mark the devices (belonging to
PM domains) that should never be turned off, except for the system
core (syscore) suspend/hibernation and resume stages, need not be
accessed by the runtime PM core functions, because all of the devices
it is set for need to be marked as "irq safe" anyway and are
protected from being turned off by runtime PM by ensuring that their
usage counters are always set.

For this reason, make the syscore flag system-wide PM-specific
and simplify the code used for manipulating it, because it need not
acquire the device's power.lock any more.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Domains: Move syscore flag from subsys data to struct device

The syscore device PM flag is used to mark the devices (belonging to
a PM domain) that should never be turned off, except for the system
core (syscore) suspend/hibernation and resume stages. That flag is
stored in the device's struct pm_subsys_data object whose address is
available from struct device. However, in some situations it may be
convenient to set that flag before the device is added to a PM
domain, so it is better to move it directly to the "power" member of
struct device. Then, it can be checked by the routines in
drivers/base/power/runtime.c and drivers/base/power/main.c, which is
more straightforward.

This also reduces the number of dev_gpd_data() invocations in the
generic PM domains framework, so the overhead related to the syscore
flag is slightly smaller.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Acked-by: Magnus Damm <damm@opensource.se>

PM / Runtime: Check device PM QoS setting before "no callbacks" check

If __dev_pm_qos_read_value(dev) returns a negative value,
rpm_suspend() should return -EPERM for dev even if its
power.no_callbacks flag is set. For this to happen, the device's
power.no_callbacks flag has to be checked after the PM QoS check,
so move the PM QoS check to rpm_check_suspend_allowed() (this will
make it cover idle notifications as well as runtime suspend too).

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Cc: stable@vger.kernel.org

PM / Runtime: Clear power.deferred_resume on success in rpm_suspend()

The power.deferred_resume can only be set if the runtime PM status
of device is RPM_SUSPENDING and it should be cleared after its
status has been changed, regardless of whether or not the runtime
suspend has been successful. However, it only is cleared on
suspend failure, while it may remain set on successful suspend and
is happily leaked to rpm_resume() executed in that case.

That shouldn't happen, so if power.deferred_resume is set in
rpm_suspend() after the status has been changed to RPM_SUSPENDED,
clear it before calling rpm_resume(). Then, it doesn't need to be
cleared before changing the status to RPM_SUSPENDING any more,
because it's always cleared after the status has been changed to
either RPM_SUSPENDED (on success) or RPM_ACTIVE (on failure).

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Cc: stable@vger.kernel.org

PM / Runtime: Fix rpm_resume() return value for power.no_callbacks set

For devices whose power.no_callbacks flag is set, rpm_resume()
should return 1 if the device's parent is already active, so that
the callers of pm_runtime_get() don't think that they have to wait
for the device to resume (asynchronously) in that case (the core
won't queue up an asynchronous resume in that case, so there's
nothing to wait for anyway).

Modify the code accordingly (and make sure that an idle notification
will be queued up on success, even if 1 is to be returned).

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Cc: stable@vger.kernel.org

PM / Runtime: Remove device fields related to suspend time, v2

After the previous changes in default_stop_ok() and
default_power_down_ok() for PM domains, there are two fields in
struct dev_pm_info that aren't necessary any more, suspend_time
and max_time_suspended_ns.

Remove those fields along with all of the code that accesses them,
which simplifies the runtime PM framework quite a bit.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: don't forget to wake up waitqueue on failure

This patch (as1535) fixes a bug in the runtime PM core. When a
runtime suspend attempt completes, whether successfully or not, the
device's power.wait_queue is supposed to be signalled. But this
doesn't happen in the failure pathway of rpm_suspend() when another
autosuspend attempt is rescheduled. As a result, a task can get stuck
indefinitely on the wait queue (I have seen this happen in testing).

The patch fixes the problem by moving the wake_up_all() call up near
the start of the failure code.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
CC: <stable@vger.kernel.org>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM: Run the driver callback directly if the subsystem one is not there

Make the PM core execute driver PM callbacks directly if the
corresponding subsystem callbacks are not present.

There are three reasons for doing that. First, it reflects the
behavior of drivers/base/dd.c:really_probe() that runs the driver's
.probe() callback directly if the bus type's one is not defined, so
this change will remove one arbitrary difference between the PM core
and the remaining parts of the driver core. Second, it will allow
some subsystems, whose PM callbacks don't do anything except for
executing driver callbacks, to be simplified quite a bit by removing
those "forward-only" callbacks. Finally, it will allow us to remove
one level of indirection in the system suspend and resume code paths
where it is not necessary, which is going to lead to less debug noise
with initcall_debug passed in the kernel command line (messages won't
be printed for driverless devices whose subsystems don't provide
PM callbacks among other things).

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Use device PM QoS constraints (v2)

Make the runtime PM core use device PM QoS constraints to check if
it is allowed to suspend a given device, so that an error code is
returned if the device's own PM QoS constraint is negative or one of
its children has already been suspended for too long. If this is
not the case, the maximum estimated time the device is allowed to be
suspended, computed as the minimum of the device's PM QoS constraint
and the PM QoS constraints of its children (reduced by the difference
between the current time and their suspend times) is stored in a new
device's PM field power.max_time_suspended_ns that can be used by
the device's subsystem or PM domain to decide whether or not to put
the device into lower-power (and presumably higher-latency) states
later (if the constraint is 0, which means "no constraint", the
power.max_time_suspended_ns is set to -1).

Additionally, the time of execution of the subsystem-level
.runtime_suspend() callback for the device is recorded in the new
power.suspend_time field for later use by the device's subsystem or
PM domain along with power.max_time_suspended_ns (it also is used
by the core code when the device's parent is suspended).

Introduce a new helper function,
pm_runtime_update_max_time_suspended(), allowing subsystems and PM
domains (or device drivers) to update the power.max_time_suspended_ns
field, for example after changing the power state of a suspended
device.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Automatically retry failed autosuspends

Originally, the runtime PM core would send an idle notification
whenever a suspend attempt failed. The idle callback routine could
then schedule a delayed suspend for some time later.

However this behavior was changed by commit
f71648d73c1650b8b4aceb3856bebbde6daa3b86 (PM / Runtime: Remove idle
notification after failing suspend). No notifications were sent, and
there was no clear mechanism to retry failed suspends.

This caused problems for the usbhid driver, because it fails
autosuspend attempts as long as a key is being held down. Therefore
this patch (as1492) adds a mechanism for retrying failed
autosuspends. If the callback routine updates the last_busy field so
that the next autosuspend expiration time is in the future, the
autosuspend will automatically be rescheduled.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Tested-by: Henrik Rydberg <rydberg@euromail.se>
Cc: <stable@kernel.org>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Fix runtime accounting calculation error

With delta type being int, its value is made zero
for all values of now > 0x80000000.
Hence fixing it.

Signed-off-by: venu byravarasu <vbyravarasu@nvidia.com>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

drivers/base: Add export.h for EXPORT_SYMBOL/THIS_MODULE as required.

Most of these files were implicitly getting EXPORT_SYMBOL via
device.h which was including module.h, but that path will be broken
soon.

[ with input from Stephen Rothwell <sfr@canb.auug.org.au> ]

Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>

PM / Runtime: Handle .runtime_suspend() failure correctly

If .runtime_suspend() returns -EAGAIN or -EBUSY, the device should
still be in ACTIVE state, so it is not necessary to send an idle
notification to its parent. If .runtime_suspend() returns other
fatal failure, it doesn't make sense to send idle notification to
its parent.

Skip parent idle notification when failure is returned from
.runtime_suspend() and update comments in rpm_suspend() to reflect
that change.

[rjw: Modified the subject and changelog slightly.]

Signed-off-by: Ming Lei <ming.lei@canonical.com>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Fix kerneldoc comment for rpm_suspend()

This patch fix kerneldoc comments for rpm_suspend():

- 'Cancel a pending idle notification' should be put before, also
should be changed to 'Cancel a pending idle notification,
autosuspend or suspend'.

- idle notification for the device after succeeding suspend has
been removed, so update the comment accordingly.

[rjw: Modified the subject and changelog slightly.]

Signed-off-by: Ming Lei <ming.lei@canonical.com>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Replace dev_dbg() with trace_rpm_*()

This patch replaces dev_dbg with trace_rpm_* inside
the three important functions:

rpm_idle
rpm_suspend
rpm_resume

Trace points have the below advantages compared with dev_dbg:

- trace points include much runtime information(such as
running cpu, current task, ...)

- most of linux distributions may disable "verbose debug"
driver debug compile switch, so it is very difficult to
report/debug runtime pm related problems from distribution
users without this kind of debug information.

- for upstream kernel users, enableing the debug switch will
produce many useless "rpm_resume" output, and it is very noise.

- dev_dbg inside rpm_suspend/rpm_resume may have some effects
on runtime pm behaviour of console devicer

Signed-off-by: Ming Lei <ming.lei@canonical.com>
Acked-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Don't run callbacks under lock for power.irq_safe set

The rpm_suspend() and rpm_resume() routines execute subsystem or PM
domain callbacks under power.lock if power.irq_safe is set for the
given device. This is inconsistent with that rpm_idle() does after
commit 02b2677 (PM / Runtime: Allow _put_sync() from
interrupts-disabled context) and is problematic for subsystems and PM
domains wanting to use power.lock for synchronization in their
runtime PM callbacks.

This change requires the code checking if the device's runtime PM
status is RPM_SUSPENDING or RPM_RESUMING to be modified too, to take
the power.irq_safe set case into account (that code wasn't reachable
before with power.irq_safe set, because it's executed with the
device's power.lock held).

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Reviewed-by: Ming Lei <tom.leiming@gmail.com>
Reviewed-by: Kevin Hilman <khilman@ti.com>

PM / Runtime: Add might_sleep() to runtime PM functions

Some of the entry points to pm runtime are not safe to
call in atomic context unless pm_runtime_irq_safe() has
been called. Inspecting the code, it is not immediately
obvious that the functions sleep at all, as they run
inside a spin_lock_irqsave, but under some conditions
they can drop the lock and turn on irqs.

If a driver incorrectly calls the pm_runtime apis, it can
cause sleeping and irq processing when it expects to stay
in atomic context.

Add might_sleep_if to the majority of the __pm_runtime_* entry points
to enforce correct usage.

Add pm_runtime_put_sync_autosuspend to the list of
functions that can be called in atomic context.

Signed-off-by: Colin Cross <ccross@android.com>
Reviewed-by: Kevin Hilman <khilman@ti.com>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Allow _put_sync() from interrupts-disabled context

Currently the use of pm_runtime_put_sync() is not safe from
interrupts-disabled context because rpm_idle() will release the
spinlock and enable interrupts for the idle callbacks. This enables
interrupts during a time where interrupts were expected to be
disabled, and can have strange side effects on drivers that expected
interrupts to be disabled.

This is not a bug since the documentation clearly states that only
_put_sync_suspend() is safe in IRQ-safe mode.

However, pm_runtime_put_sync() could be made safe when in IRQ-safe
mode by releasing the spinlock but not re-enabling interrupts, which
is what this patch aims to do.

Problem was found when using some buggy drivers that set
pm_runtime_irq_safe() and used _put_sync() in interrupts-disabled
context.

Reported-by: Colin Cross <ccross@google.com>
Tested-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Consistent utilization of deferred_resume

dev->power.deferred_resume is used as a bool typically, so change
one assignment to false from 0, like other places.

Signed-off-by: ShuoX Liu <shuox.liu@intel.com>

PM / Runtime: Replace "run-time" with "runtime" in documentation

The runtime PM documentation and kerneldoc comments sometimes spell
"runtime" with a dash (i.e. "run-time"). Replace all of those
instances with "runtime" to make the naming consistent.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Return special error code if runtime PM is disabled

Some callers of pm_runtime_get_sync() and other runtime PM helper
functions, scsi_autopm_get_host() and scsi_autopm_get_device() in
particular, need to distinguish error codes returned when runtime PM
is disabled (i.e. power.disable_depth is nonzero for the given
device) from error codes returned in other situations. For this
reason, make the runtime PM helper functions return -EACCES when
power.disable_depth is nonzero and ensure that this error code
won't be returned by them in any other circumstances. Modify
scsi_autopm_get_host() and scsi_autopm_get_device() to check the
error code returned by pm_runtime_get_sync() and ignore -EACCES.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Domains: Rename struct dev_power_domain to struct dev_pm_domain

The naming convention used by commit 7538e3db6e015e890825fbd9f86599b
(PM: Add support for device power domains), which introduced the
struct dev_power_domain type for representing device power domains,
evidently confuses some developers who tend to think that objects
of this type must correspond to "power domains" as defined by
hardware, which is not the case. Namely, at the kernel level, a
struct dev_power_domain object can represent arbitrary set of devices
that are mutually dependent power management-wise and need not belong
to one hardware power domain. To avoid that confusion, rename struct
dev_power_domain to struct dev_pm_domain and rename the related
pointers in struct device and struct pm_clk_notifier_block from
pwr_domain to pm_domain.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Acked-by: Kevin Hilman <khilman@ti.com>

PM: Make power domain callbacks take precedence over subsystem ones

Change the PM core's behavior related to power domains in such a way
that, if a power domain is defined for a given device, its callbacks
will be executed instead of and not in addition to the device
subsystem's PM callbacks.

The idea behind the initial implementation of power domains handling
by the PM core was that power domain callbacks would be executed in
addition to subsystem callbacks, so that it would be possible to
extend the subsystem callbacks by using power domains. It turns out,
however, that this wouldn't be really convenient in some important
situations.

For example, there are systems in which power can only be removed
from entire power domains. On those systems it is not desirable to
execute device drivers' PM callbacks until it is known that power is
going to be removed from the devices in question, which means that
they should be executed by power domain callbacks rather then by
subsystem (e.g. bus type) PM callbacks, because subsystems generally
have no information about what devices belong to which power domain.
Thus, for instance, if the bus type in question is the platform bus
type, its PM callbacks generally should not be called in addition to
power domain callbacks, because they run device drivers' callbacks
unconditionally if defined.

While in principle the default subsystem PM callbacks, or a subset of
them, may be replaced with different functions, it doesn't seem
correct to do so, because that would change the subsystem's behavior
with respect to all devices in the system, regardless of whether or
not they belong to any power domains. Thus, the only remaining
option is to make power domain callbacks take precedence over
subsystem callbacks.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Acked-by: Grant Likely <grant.likely@secretlab.ca>
Acked-by: Kevin Hilman <khilman@ti.com>

Fix common misspellings

Fixes generated by 'codespell' and manually reviewed.

Signed-off-by: Lucas De Marchi <lucas.demarchi@profusion.mobi>

PM: Make system-wide PM and runtime PM treat subsystems consistently

The code handling system-wide power transitions (eg. suspend-to-RAM)
can in theory execute callbacks provided by the device's bus type,
device type and class in each phase of the power transition. In
turn, the runtime PM core code only calls one of those callbacks at
a time, preferring bus type callbacks to device type or class
callbacks and device type callbacks to class callbacks.

It seems reasonable to make them both behave in the same way in that
respect. Moreover, even though a device may belong to two subsystems
(eg. bus type and device class) simultaneously, in practice power
management callbacks for system-wide power transitions are always
provided by only one of them (ie. if the bus type callbacks are
defined, the device class ones are not and vice versa). Thus it is
possible to modify the code handling system-wide power transitions
so that it follows the core runtime PM code (ie. treats the
subsystem callbacks as mutually exclusive).

On the other hand, the core runtime PM code will choose to execute,
for example, a runtime suspend callback provided by the device type
even if the bus type's struct dev_pm_ops object exists, but the
runtime_suspend pointer in it happens to be NULL. This is confusing,
because it may lead to the execution of callbacks from different
subsystems during different operations (eg. the bus type suspend
callback may be executed during runtime suspend of the device, while
the device type callback will be executed during system suspend).

Make all of the power management code treat subsystem callbacks in
a consistent way, such that:
(1) If the device's type is defined (eg. dev->type is not NULL)
and its pm pointer is not NULL, the callbacks from dev->type->pm
will be used.
(2) If dev->type is NULL or dev->type->pm is NULL, but the device's
class is defined (eg. dev->class is not NULL) and its pm pointer
is not NULL, the callbacks from dev->class->pm will be used.
(3) If dev->type is NULL or dev->type->pm is NULL and dev->class is
NULL or dev->class->pm is NULL, the callbacks from dev->bus->pm
will be used provided that both dev->bus and dev->bus->pm are
not NULL.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Acked-by: Kevin Hilman <khilman@ti.com>
Reasoning-sounds-sane-to: Grant Likely <grant.likely@secretlab.ca>
Acked-by: Greg Kroah-Hartman <gregkh@suse.de>

PM: Add support for device power domains

The platform bus type is often used to handle Systems-on-a-Chip (SoC)
where all devices are represented by objects of type struct
platform_device. In those cases the same "platform" device driver
may be used with multiple different system configurations, but the
actions needed to put the devices it handles into a low-power state
and back into the full-power state may depend on the design of the
given SoC. The driver, however, cannot possibly include all the
information necessary for the power management of its device on all
the systems it is used with. Moreover, the device hierarchy in its
current form also is not suitable for representing this kind of
information.

The patch below attempts to address this problem by introducing
objects of type struct dev_power_domain that can be used for
representing power domains within a SoC. Every struct
dev_power_domain object provides a sets of device power
management callbacks that can be used to perform what's needed for
device power management in addition to the operations carried out by
the device's driver and subsystem.

Namely, if a struct dev_power_domain object is pointed to by the
pwr_domain field in a struct device, the callbacks provided by its
ops member will be executed in addition to the corresponding
callbacks provided by the device's subsystem and driver during all
power transitions.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Tested-and-acked-by: Kevin Hilman <khilman@ti.com>

PM / Runtime: Don't enable interrupts while running in_interrupt

This patch (as1445) fixes a bug in the runtime PM core left over from
the addition of the no_callbacks flag. If this flag is set then it is
possible for rpm_suspend() to be called in_interrupt, so when
releasing spinlocks it's important not to re-enable interrupts.

To avoid an unnecessary save-and-restore of the interrupt flag, the
patch also inlines a pm_request_idle() call.

This fixes Bugzilla #27482.

(The offending code was added in 2.6.37, so it's not necessary to apply
this to any earlier stable kernels.)

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Reported-by: tim blechmann <tim@klingt.org>
CC: <stable@kernel.org>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Add synchronous runtime interface for interrupt handlers (v3)

This patch (as1431c) makes the synchronous runtime-PM interface
suitable for use in interrupt handlers. Subsystems can call the new
pm_runtime_irq_safe() function to tell the PM core that a device's
runtime_suspend and runtime_resume callbacks should be invoked with
interrupts disabled and the spinlock held. This permits the
pm_runtime_get_sync() and the new pm_runtime_put_sync_suspend()
routines to be called from within interrupt handlers.

When a device is declared irq-safe in this way, the PM core increments
the parent's usage count, so the parent will never be runtime
suspended. This prevents difficult situations in which an irq-safe
device can't resume because it is forced to wait for its non-irq-safe
parent.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Fix typo in status comparison causing warning

GCC version 4.5.1 gives the following warning:

drivers/base/power/runtime.c: In function ‘rpm_check_suspend_allowed’:
drivers/base/power/runtime.c:146:25: warning: comparison between ‘enum dpm_state’ and ‘enum rpm_status’

which seems to be a typo in that dev->power.runtime_status
should be compared instead of dev->power.status.

Signed-off-by: Kevin Winchester <kjwinchester@gmail.com>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: fix recursive locking warning of lockdep from rpm_resume()

For device with no_callbacks flag set, its power lock and its parent's
power lock may be held nestedly in rpm_resume, so we should take
spin_lock_nested(lock, SINGLE_DEPTH_NESTING) to acquire parent power lock
to avoid lockdep warning.

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Remove idle notification after failing suspend

If runtime suspend of a device fails returning -EAGAIN or -EBUSY,
which means that it's safe to try to suspend it again, the PM core
runs the runtime idle helper function for it. Unfortunately this may
lead to problems, for example for PCI devices whose drivers don't
implement the ->runtime_idle() callback, because in that case the
PCI bus type's ->runtime_idle() always calls pm_runtime_suspend()
for the given device. Then, if there's an automatic idle
notification after the driver's ->runtime_suspend() returning -EAGAIN
or -EBUSY, it will make the suspend happen again possibly causing a
busy loop to appear. To avoid that, remove the idle notification
after failing runtime suspend of a device altogether and let the
callers of pm_runtime_suspend() repeat the operation if need be.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Acked-by: Alan Stern <stern@rowland.harvard.edu>

PM / Runtime: Reduce code duplication in core helper functions

Reduce code duplication in rpm_idle(), rpm_suspend() and rpm_resume()
by using local pointers to store callback addresses and moving some
duplicated code into a separate function.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Reviewed-by: Alan Stern <stern@rowland.harvard.edu>

PM / Runtime: Implement autosuspend support

This patch (as1427) implements the "autosuspend" facility for runtime
PM. A few new fields are added to the dev_pm_info structure and
several new PM helper functions are defined, for telling the PM core
whether or not a device uses autosuspend, for setting the autosuspend
delay, and for marking periods of device activity.

Drivers that do not want to use autosuspend can continue using the
same helper functions as before; their behavior will not change. In
addition, drivers supporting autosuspend can also call the old helper
functions to get the old behavior.

The details are all explained in Documentation/power/runtime_pm.txt
and Documentation/ABI/testing/sysfs-devices-power.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Add no_callbacks flag

Some devices, such as USB interfaces, cannot be power-managed
independently of their parents, i.e., they cannot be put in low power
while the parent remains at full power. This patch (as1425) creates a
new "no_callbacks" flag, which tells the PM core not to invoke the
runtime-PM callback routines for the such devices but instead to
assume that the callbacks always succeed. In addition, the
non-debugging runtime-PM sysfs attributes for the devices are removed,
since they are pretty much meaningless.

The advantage of this scheme comes not so much from avoiding the
callbacks themselves, but rather from the fact that without the need
for a process context in which to run the callbacks, more work can be
done in interrupt context.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Combine runtime PM entry points

This patch (as1424) combines the various public entry points for the
runtime PM routines into three simple functions: one for idle, one for
suspend, and one for resume. A new bitflag specifies whether or not
to increment or decrement the usage_count field.

The new entry points are named __pm_runtime_idle,
__pm_runtime_suspend, and __pm_runtime_resume, to reflect that they
are trampolines. Simultaneously, the corresponding internal routines
are renamed to rpm_idle, rpm_suspend, and rpm_resume.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Merge synchronous and async runtime routines

This patch (as1423) merges the asynchronous routines
__pm_request_idle(), __pm_request_suspend(), and __pm_request_resume()
with their synchronous counterparts. The RPM_ASYNC bitflag argument
serves to indicate what sort of operation to perform.

In the course of performing this merger, it became apparent that the
various functions don't all behave consistenly with regard to error
reporting and cancellation of outstanding requests. A new routine,
rpm_check_suspend_allowed(), was written to centralize much of the
testing, and the other functions were revised to follow a simple
algorithm:

If the operation is disallowed because of the device's
settings or current state, return an error.

Cancel pending or scheduled requests of lower priority.

Schedule, queue, or perform the desired operation.

A few special cases and exceptions are noted in comments.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Replace boolean arguments with bitflags

The "from_wq" argument in __pm_runtime_suspend() and
__pm_runtime_resume() supposedly indicates whether or not the function
was called by the PM workqueue thread, but in fact it isn't always
used this way. It really indicates whether or not the function should
return early if the requested operation is already in progress.

Along with this badly-named boolean argument, later patches in this
series will add several other boolean arguments to these functions and
others. Therefore this patch (as1422) begins the conversion process
by replacing from_wq with a bitflag argument. The same bitflags are
also used in __pm_runtime_get() and __pm_runtime_put(), where they
indicate whether or not the operation should be asynchronous.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Move code in drivers/base/power/runtime.c

This patch (as1421) moves the PM runtime accounting subroutines up to
the beginning of runtime.c, taking them out of the middle of the
functions that do the actual work. No operational changes.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Wakeup: Introduce wakeup source objects and event statistics (v3)

Introduce struct wakeup_source for representing system wakeup sources
within the kernel and for collecting statistics related to them.
Make the recently introduced helper functions pm_wakeup_event(),
pm_stay_awake() and pm_relax() use struct wakeup_source objects
internally, so that wakeup statistics associated with wakeup devices
can be collected and reported in a consistent way (the definition of
pm_relax() is changed, which is harmless, because this function is
not called directly by anyone yet). Introduce new wakeup-related
sysfs device attributes in /sys/devices/.../power for reporting the
device wakeup statistics.

Change the global wakeup events counters event_count and
events_in_progress into atomic variables, so that it is not necessary
to acquire a global spinlock in pm_wakeup_event(), pm_stay_awake()
and pm_relax(), which should allow us to avoid lock contention in
these functions on SMP systems with many wakeup devices.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Acked-by: Greg Kroah-Hartman <gregkh@suse.de>

PM / Runtime: Add runtime PM statistics (v3)

In order for PowerTOP to be able to report how well the new runtime PM is
working for the various drivers, the kernel needs to export some basic
statistics in sysfs.

This patch adds two sysfs files in the runtime PM domain that expose the
total time a device has been active, and the time a device has been
suspended.

With this PowerTOP can compute the activity percentage

Active %age = 100 * (delta active) / (delta active + delta suspended)

and present the information to the user.

I've written the PowerTOP code (slated for version 1.12) already, and the
output looks like this:

Runtime Device Power Management statistics
Active Device name
10.0% 06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8101E/RTL8102E PCI Express Fast Ethernet controller

[version 2: fix stat update bugs noticed by Alan Stern]
[version 3: rebase to -next and move the sysfs declaration]

Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM: Allow runtime_suspend methods to call pm_schedule_suspend()

This patch (as1361) changes the runtime PM interface slightly; it
allows suspend requests to be scheduled while the runtime_suspend
method is running. If the method succeeds then the scheduled request
is cancelled, whereas if the method fails then an idle notification is
sent only if no request was scheduled.

Being able to schedule suspend requests from within a runtime_suspend
method is useful for drivers that need to test for idleness and
suspend the device all while holding a single spinlock, or for drivers
that want to check for idleness by polling.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Add sysfs switch for disabling device run-time PM

Add new device sysfs attribute, power/control, allowing the user
space to block the run-time power management of the devices. If this
attribute is set to "on", the driver of the device won't be able to power
manage it at run time (without breaking the rules) and the device will
always be in the full power state (except when the entire system goes
into a sleep state).

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Acked-by: Alan Stern <stern@rowland.harvard.edu>

PM / Runtime: Use device type and device class callbacks

The power management of some devices is handled through device types
and device classes rather than through bus types. Since these
devices may also benefit from using the run-time power management
core, extend it so that the device type and device class run-time PM
callbacks can be taken into consideration by it if the bus type
callback is not defined.

Update the run-time PM core documentation to reflect this change.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM: allow for usage_count > 0 in pm_runtime_get()

This patch (as1308c) fixes __pm_runtime_get(). Currently the routine
will resume a device if the prior usage count was 0. But this isn't
right; thanks to pm_runtime_get_noresume() the usage count can be
positive even while the device is suspended.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Remove unnecessary braces in __pm_runtime_set_status()

Some braces in __pm_runtime_set_status() are not necessary, so
remove them.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Ensure timer_expires is nonzero in pm_schedule_suspend()

The runtime PM core code assumes that dev->power.timer_expires is
nonzero when the timer is scheduled, but it may become zero
incidentally in pm_schedule_suspend(). Prevent this from happening
by bumping dev->power.timer_expires up to 1 if it's 0 before calling
mod_timer().

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Reported-by: Alan Stern <stern@rowland.harvard.edu>

PM / Runtime: Use deferred_resume flag in pm_request_resume

This patch (as1307) adds a small optimization to
__pm_request_resume(). If the device is currently being suspended,
there's no need to queue a work routine to resume it. Setting the
deferred_resume flag will suffice. (There's also a minor improvement
to the function's code layout: An unnecessary "else" is removed.)

Also, the patch clarifies the usage of the deferred_resume flag. It
is meaningful only while a suspend is in progress, so it should be
cleared just before a suspend starts, not just after one ends.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM / Runtime: Fix lockdep warning in __pm_runtime_set_status()

Lockdep complains about taking the parent lock in
__pm_runtime_set_status(), so mark it as nested.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Reported-by: Alan Stern <stern@rowland.harvard.edu>
Cc: stable@kernel.org

PM: fix irq enable/disable in runtime PM code

This patch (as1305) fixes a bug in the irq-enable settings and removes
some related overhead in the runtime PM code.

In __pm_runtime_resume(), within the scope of the original
spin_lock_irq(), we know that irqs are disabled. There's no
reason to go through a pair of enable/disable cycles when
acquiring and releasing the parent's lock.

In __pm_runtime_set_status(), irqs are already disabled when
the parent's lock is acquired, and they must remain disabled
when it is released.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM: Remove some debug messages producing too much noise

pm_runtime_idle() is somewhat noisy. Remove debug prints.

Signed-off-by: Pavel Machek <pavel@ucw.cz>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>

PM: Introduce core framework for run-time PM of I/O devices (rev. 17)

Introduce a core framework for run-time power management of I/O
devices. Add device run-time PM fields to 'struct dev_pm_info'
and device run-time PM callbacks to 'struct dev_pm_ops'. Introduce
a run-time PM workqueue and define some device run-time PM helper
functions at the core level. Document all these things.

Special thanks to Alan Stern for his help with the design and
multiple detailed reviews of the pereceding versions of this patch
and to Magnus Damm for testing feedback.

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Acked-by: Magnus Damm <damm@igel.co.jp>

PM: remove deprecated dpm_runtime_* routines

This patch (as933) removes the deprecated dpm_runtime_suspend() and
dpm_runtime_resume() routines from the PM core. The only user of
those routines is the PCMCIA ds driver; local replacements are added.

Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
CC: Dominik Brodowski <linux@dominikbrodowski.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

Power Management: use mutexes instead of semaphores

The Power Management code uses semaphores as mutexes. Use the mutex API
instead of the (binary) semaphores.

Signed-off-by: Matthias Kaehlcke <matthias.kaehlcke@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

[PATCH] pcmcia: merge suspend into device model

Merge the suspend and resume methods for 16-bit PCMCIA cards into the
device model -- for both runtime power management and suspend to ram/disk.

Bugfix in ds.c by Richard Purdie
Signed-Off-By: Richard Purdie <rpurdie@rpsys.net>

Signed-off-by: Dominik Brodowski <linux@dominikbrodowski.net>

[PATCH] drivers/base/power/runtime.c: #if 0 dpm_set_power_state()

This patch #if 0's an unused global function.

Signed-off-by: Adrian Bunk <bunk@stusta.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

[PATCH] root hub changes (lesser half)

This patch collects various small updates related to root hubs, to shrink
later patches which build on them.

- For root hub suspend/resume support:
* Make the existing usb_hcd_resume_root_hub() routine respect pmcore
locking, exporting and using the dpm_runtime_resume() method.
* Add a new usb_hcd_suspend_root_hub() to pair with that routine.
(Essential to make OHCI autosuspend behave again...)
* HC_SUSPENDED by itself only refers to the root hub's downstream ports.
So let HCDs see root hub URBs unless the parent device is suspended.

- Remove an assertion we no longer need (and now, also don't want).

- Generic suspend/resume updates to work better with swsusp.
* Ignore the FREEZE vs SUSPEND distinction for hardware; trying to
use it breaks the swsusp snapshots it's supposed to help (sigh).
* On resume, mark devices as resumed right away, but then
do nothing else if the device is marked NOTATTACHED.

These changes shouldn't be very noticable by themselves.

Signed-off-by: David Brownell <dbrownell@users.sourceforge.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

drivers/base/power/runtime.c | 1
drivers/usb/core/hcd.c | 64 ++++++++++++++++++++++++++++++++++++++-----
drivers/usb/core/hcd.h | 1
drivers/usb/core/hub.c | 45 ++++++++++++++++++++++++------
drivers/usb/core/usb.c | 20 +++++++++----
drivers/usb/core/usb.h | 1
6 files changed, 111 insertions(+), 21 deletions(-)

[PATCH] swsusp: switch pm_message_t to struct

This adds type-checking to pm_message_t, so that people can't confuse it
with int or u32. It also allows us to fix "disk yoyo" during suspend (disk
spinning down/up/down).

[We've tried that before; since that cpufreq problems were fixed and I've
tried make allyes config and fixed resulting damage.]

Signed-off-by: Pavel Machek <pavel@suse.cz>
Signed-off-by: Alexander Nyberg <alexn@telia.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

Linux-2.6.12-rc2

Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.

Let it rip!