OPP: Extend dev_pm_opp_data with turbo support

Let's extend the dev_pm_opp_data with a turbo variable, to allow users to
specify if it's a boost frequency for a dynamically added OPP.

Signed-off-by: Sibi Sankar <quic_sibis@quicinc.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Rename 'rate_clk_single'

The field's name isn't clear enough. Rename it.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Pass rounded rate to _set_opp()

The OPP core finds the eventual frequency to set with the help of
clk_round_rate() and the same was earlier getting passed to _set_opp()
and that's what would get configured.

The commit 1efae8d2e777 ("OPP: Make dev_pm_opp_set_opp() independent of
frequency") mistakenly changed that. Fix it.

Fixes: 1efae8d2e777 ("OPP: Make dev_pm_opp_set_opp() independent of frequency")
Cc: v5.18+ <stable@vger.kernel.org> # v6.0+
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Relocate dev_pm_opp_sync_regulators()

Move this to a more relevant place in the file. No functional changes.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Fix _set_required_opps when opp is NULL

_set_required_opps can be called with opp NULL in _disable_opp_table().

commit e37440e7e2c2 ("OPP: Call dev_pm_opp_set_opp() for required OPPs")
requires the opp pointer to be non-NULL to function.

[ 81.253439] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000048
[ 81.438407] Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT)
[ 81.445296] Workqueue: pm pm_runtime_work
[ 81.449446] pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 81.456609] pc : _set_required_opps+0x178/0x28c
[ 81.461288] lr : _set_required_opps+0x178/0x28c
[ 81.465962] sp : ffff80008078bb00
[ 81.469375] x29: ffff80008078bb00 x28: ffffd1cd71bfe308 x27: 0000000000000000
[ 81.476730] x26: ffffd1cd70ebc578 x25: ffffd1cd70a08710 x24: 00000000ffffffff
[ 81.484083] x23: 00000000ffffffff x22: 0000000000000000 x21: ffff56ff892b3c48
[ 81.491435] x20: ffff56f1071c10 x19: 0000000000000000 x18: ffffffffffffffff
[ 81.498788] x17: 2030207865646e69 x16: 2030303131207370 x15: 706f5f6465726975
[ 81.506141] x14: 7165725f7465735f x13: ffff5700f5c00000 x12: 00000000000008ac
[ 81.513495] x11: 00000000000002e4 x10: ffff5700f6700000 x9 : ffff5700f5c00000
[ 81.520848] x8 : 00000000fffdffff x7 : ffff5700f6700000 x6 : 80000000fffe0000
[ 81.528200] x5 : ffff5700fef40d08 x4 : 0000000000000000 x3 : 0000000000000000
[ 81.535551] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff56ff81298f80
[ 81.542904] Call trace:
[ 81.545437] _set_required_opps+0x178/0x28c
[ 81.549754] _set_opp+0x3fc/0x5c0
[ 81.553181] dev_pm_opp_set_rate+0x90/0x26c
[ 81.557498] core_power_v4+0x44/0x15c [venus_core]
[ 81.562509] venus_runtime_suspend+0x40/0xd0 [venus_core]
[ 81.568135] pm_generic_runtime_suspend+0x2c/0x44
[ 81.572983] __rpm_callback+0x48/0x1d8
[ 81.576852] rpm_callback+0x6c/0x78
[ 81.580453] rpm_suspend+0x10c/0x570
[ 81.584143] pm_runtime_work+0xc4/0xc8
[ 81.588011] process_one_work+0x138/0x244
[ 81.592153] worker_thread+0x320/0x438
[ 81.596021] kthread+0x110/0x114
[ 81.599355] ret_from_fork+0x10/0x20
[ 81.603052] Code: f10000ff fa5410e0 54fffbe1 97f05ae8 (f94026c5)
[ 81.609317] ---[ end trace 0000000000000000 ]---

Fix it.

Fixes: e37440e7e2c2 ("OPP: Call dev_pm_opp_set_opp() for required OPPs")
Signed-off-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
[ Viresh: Implemented the fix differently ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Tested-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>

OPP: The level field is always of unsigned int type

By mistake, dev_pm_opp_find_level_floor() used the level parameter as
unsigned long instead of unsigned int. Fix it.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Check for invalid OPP in dev_pm_opp_find_level_ceil()

_find_key_ceil() may return an error and that must be checked before
passing the same to dev_pm_opp_put().

Fixes: 41907aa4ae37 ("OPP: Level zero is valid")
Reported-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Don't set OPP recursively for a parent genpd

Like other frameworks (clk, regulator, etc.) genpd core too takes care
of propagation to performance state to parent genpds. The OPP core
shouldn't attempt the same, or it may result in undefined behavior.

Add checks at various places to take care of the same.

Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Tested-by: Stephan Gerhold <stephan.gerhold@kernkonzept.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Call dev_pm_opp_set_opp() for required OPPs

Configuring the required OPP was never properly implemented, we just
took an exception for genpds and configured them directly, while leaving
out all other required OPP types.

Now that a standard call to dev_pm_opp_set_opp() takes care of
configuring the opp->level too, the special handling for genpds can be
avoided by simply calling dev_pm_opp_set_opp() for the required OPPs,
which shall eventually configure the corresponding level for genpds.

This also makes it possible for us to configure other type of required
OPPs (no concrete users yet though), via the same path. This is how
other frameworks take care of parent nodes, like clock, regulators, etc,
where we recursively call the same helper.

In order to call dev_pm_opp_set_opp() for the virtual genpd devices,
they must share the OPP table of the genpd. Call _add_opp_dev() for them
to get that done.

This commit also extends the struct dev_pm_opp_config to pass required
devices, for non-genpd cases, which can be used to call
dev_pm_opp_set_opp() for the non-genpd required devices.

Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Tested-by: Stephan Gerhold <stephan.gerhold@kernkonzept.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Use _set_opp_level() for single genpd case

There are two genpd (as required-opp) cases that we need to handle,
devices with a single genpd and ones with multiple genpds.

The multiple genpds case is clear, where the OPP core calls
dev_pm_domain_attach_by_name() for them and uses the virtual devices
returned by this helper to call dev_pm_domain_set_performance_state()
later to change the performance state.

The single genpd case however requires special handling as we need to
use the same `dev` structure (instead of a virtual one provided by genpd
core) for setting the performance state via
dev_pm_domain_set_performance_state().

As we move towards more generic code to take care of the required OPPs,
where we will recursively call dev_pm_opp_set_opp() for all the required
OPPs, the above special case becomes a problem.

It doesn't make sense for a device's DT entry to have both "opp-level"
and single "required-opps" entry pointing to a genpd's OPP, as that
would make the OPP core call dev_pm_domain_set_performance_state() for
two different values for the same device structure. And so we can reuse
the 'opp->level" field in such a case and call _set_opp_level() for the
device.

Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Tested-by: Stephan Gerhold <stephan.gerhold@kernkonzept.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Level zero is valid

The level zero can be used by some OPPs to drop performance state vote
for the device. It is perfectly fine to allow the same.

_set_opp_level() considers it as an invalid value currently and returns
early.

In order to support this properly, initialize the level field with
U32_MAX, which denotes unused level field.

Reported-by: Stephan Gerhold <stephan.gerhold@kernkonzept.com>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Tested-by: Stephan Gerhold <stephan.gerhold@kernkonzept.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: No need to defer probe from _opp_attach_genpd()

When the new interface for attaching genpd's via the OPP core was added,
it was possible for required_opp_count to be zero, but not anymore.

Remove the unused check.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

OPP: Remove genpd_virt_dev_lock

All the config operations for OPP tables share common code paths now and
none of the other ones have such protection in place. Either all should
have it or none.

The understanding here is that user won't clear the OPP configs while
still using them and so such a case won't happen. We can always come
back and use a wider lock for all resource types if required.

Also fix the error on failing to allocate memory.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

OPP: Reorder code in _opp_set_required_opps_genpd()

Reorder code in _opp_set_required_opps_genpd() to reduce duplicate code.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

OPP: Remove doc style comments for internal routines

These were previously exposed outside of the OPP core and needed doc
style commenting. They aren't public now and don't need the same.

This fixes warnings generated for builds with `W=1`:

core.c:2105: warning: Function parameter or member 'opp_table' not described in '_opp_set_supported_hw'
core.c:2105: warning: Excess function parameter 'dev' description in '_opp_set_supported_hw'
core.c:2148: warning: Function parameter or member 'opp_table' not described in '_opp_set_prop_name'
core.c:2148: warning: Excess function parameter 'dev' description in '_opp_set_prop_name'
core.c:2189: warning: Function parameter or member 'opp_table' not described in '_opp_set_regulators'
core.c:2189: warning: Excess function parameter 'count' description in '_opp_set_regulators'
core.c:2293: warning: Function parameter or member 'opp_table' not described in '_opp_set_clknames'
core.c:2293: warning: Function parameter or member 'config_clks' not described in '_opp_set_clknames'
core.c:2391: warning: Function parameter or member 'opp_table' not described in '_opp_set_config_regulators_helper'
core.c:2455: warning: Function parameter or member 'opp_table' not described in '_opp_attach_genpd'
core.c:2682: warning: Function parameter or member 'token' not described in 'dev_pm_opp_clear_config'
core.c:2682: warning: Excess function parameter 'opp_table' description in 'dev_pm_opp_clear_config'

Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202309091558.x3JJrxFI-lkp@intel.com/
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Add dev_pm_opp_find_level_floor()

Add dev_pm_opp_find_level_floor(), as is done for frequency and
bandwidth.

Signed-off-by: Krishna chaitanya chundru <quic_krichai@quicinc.com>
[ Viresh: Updated commit log and rearranged code ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Extend support for the opp-level beyond required-opps

At this point the level (performance state) for an OPP is currently limited
to be requested for a device that is attached to a PM domain. Moreover,
the device needs to have the so called required-opps assigned to it, which
are based upon OPP tables being described in DT.

To extend the support beyond required-opps and DT, let's enable the level
to be set for all OPPs. More precisely, if the requested OPP has a valid
level let's try to request it through the device's optional PM domain, via
calling dev_pm_domain_set_performance_state().

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
[ Viresh: Handle NULL opp in _set_opp_level() ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Switch to use dev_pm_domain_set_performance_state()

To support performance scaling for any kinds of PM domains, let's move away
from using the genpd specific API, dev_pm_genpd_set_performance_state(), to
the common dev_pm_domain_set_performance_state().

No intended functional impact at this point.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Extend dev_pm_opp_data with a level

Let's extend the dev_pm_opp_data with a level variable, to allow users to
specify a corresponding level (performance state) for a dynamically added
OPP.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Add dev_pm_opp_add_dynamic() to allow more flexibility

The dev_pm_opp_add() API is limited to add dynamic OPPs with a frequency
and a voltage level. To enable more flexibility, let's add a new API,
dev_pm_opp_add_dynamic() that's takes a struct dev_pm_opp_data* instead of
a list of in-parameters.

Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Fix passing 0 to PTR_ERR in _opp_attach_genpd()

If dev_pm_domain_attach_by_name() returns NULL, then 0 will be passed to
PTR_ERR() as reported by the smatch warning below:

drivers/opp/core.c:2456 _opp_attach_genpd() warn: passing zero to 'PTR_ERR'

Fix it by checking for the non-NULL virt_dev pointer before passing it to
PTR_ERR. Otherwise return -ENODEV.

Fixes: 4ea9496cbc95 ("opp: Fix error check in dev_pm_opp_attach_genpd()")
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Fix potential null ptr dereference in dev_pm_opp_get_required_pstate()

"opp" pointer is dereferenced before the IS_ERR_OR_NULL() check. Fix it by
removing the dereference to cache opp_table and dereference it directly
where opp_table is used.

This fixes the following smatch warning:

drivers/opp/core.c:232 dev_pm_opp_get_required_pstate() warn: variable
dereferenced before IS_ERR check 'opp' (see line 230)

Fixes: 84cb7ff35fcf ("OPP: pstate is only valid for genpd OPP tables")
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Reuse dev_pm_opp_get_freq_indexed()

Reuse dev_pm_opp_get_freq_indexed() from dev_pm_opp_get_freq().

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

OPP: Update _read_freq() to return the correct frequency

Now that we support finding indexed frequencies, lets update
_read_freq() to return the right one.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

OPP: Add dev_pm_opp_find_freq_exact_indexed()

The indexed version of the API is added for other floor and ceil, add
the same for exact as well for completeness.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Introduce dev_pm_opp_get_freq_indexed() API

In the case of devices with multiple clocks, drivers need to specify the
frequency index for the OPP framework to get the specific frequency within
the required OPP. So let's introduce the dev_pm_opp_get_freq_indexed() API
accepting the frequency index as an argument.

Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
[ Viresh: Fixed potential access to NULL opp pointer ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Introduce dev_pm_opp_find_freq_{ceil/floor}_indexed() APIs

In the case of devices with multiple clocks, drivers need to specify the
clock index for the OPP framework to find the OPP corresponding to the
floor/ceil of the supplied frequency. So let's introduce the two new APIs
accepting the clock index as an argument.

These APIs use the exising _find_key_ceil() helper by supplying the clock
index to it.

Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
[ Viresh: Rearranged definitions in pm_opp.h ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Simplify the over-designed pstate <-> level dance

While adding support for "performance states" in the OPP and genpd core,
it was decided to set the `pstate` field via genpd's
pm_genpd_opp_to_performance_state() helper, to allow platforms to set
`pstate` even if they don't have a corresponding `level` field in the DT
OPP tables (More details are present in commit 6e41766a6a50 ("PM /
Domain: Implement of_genpd_opp_to_performance_state()")).

Revisiting that five years later clearly suggests that it was
over-designed as all current users are eventually using the `level`
value only.

The previous commit already added necessary checks to make sure pstate
is only used for genpd tables. Lets now simplify this a little, and use
`level` directly and remove `pstate` field altogether.

Suggested-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

OPP: pstate is only valid for genpd OPP tables

It is not very clear right now that the `pstate` field is only valid for
genpd OPP tables and not consumer tables. And there is no checking for
the same at various places.

Add checks in place to verify that and make it clear to the reader.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Bjorn Andersson <quic_bjorande@quicinc.com>
Tested-by: Bjorn Andersson <quic_bjorande@quicinc.com>

OPP: don't drop performance constraint on OPP table removal

This code was added (long back) by commit 009acd196fc8 ("PM / OPP:
Support updating performance state of device's power domain") and at
that time the `opp->pstate` field was used to store the performance
state required by a device's OPP.

Over time that changed and the `->pstate` field is now used only for
genpd devices and consumer devices access that via the required-opps
instead.

Because of all these changes, _opp_table_kref_release() now drops the
constraint only when the genpd's OPP table gets freed and not the
device's. Which is definitely not what we wanted. And dropping the
constraint doesn't have much meaning as the genpd itself is going away.

Moreover, if we want to drop constraints here, then just dropping the
performance constraint alone isn't sufficient as there are other
resource constraints like clk, regulator, etc. too, which must be
handled.

Probably the right thing to do here is to leave this decision to the
consumers, which can call `dev_pm_opp_set_rate(dev, 0)` or similar APIs
to drop all constraints properly. Which many of the consumers already
do.

Remove the special code, which is broken anyway.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

OPP: Staticize `lazy_opp_tables` in of.c

`lazy_opp_tables` is only used in of.c, move it there and mark it
`static`.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Fix use-after-free in lazy_opp_tables after probe deferral

When dev_pm_opp_of_find_icc_paths() in _allocate_opp_table() returns
-EPROBE_DEFER, the opp_table is freed again, to wait until all the
interconnect paths are available.

However, if the OPP table is using required-opps then it may already
have been added to the global lazy_opp_tables list. The error path
does not remove the opp_table from the list again.

This can cause crashes later when the provider of the required-opps
is added, since we will iterate over OPP tables that have already been
freed. E.g.:

Unable to handle kernel NULL pointer dereference when read
CPU: 0 PID: 7 Comm: kworker/0:0 Not tainted 6.4.0-rc3
PC is at _of_add_opp_table_v2 (include/linux/of.h:949
drivers/opp/of.c:98 drivers/opp/of.c:344 drivers/opp/of.c:404
drivers/opp/of.c:1032) -> lazy_link_required_opp_table()

Fix this by calling _of_clear_opp_table() to remove the opp_table from
the list and clear other allocated resources. While at it, also add the
missing mutex_destroy() calls in the error path.

Cc: stable@vger.kernel.org
Suggested-by: Viresh Kumar <viresh.kumar@linaro.org>
Fixes: 7eba0c7641b0 ("opp: Allow lazy-linking of required-opps")
Signed-off-by: Stephan Gerhold <stephan.gerhold@kernkonzept.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: rate-limit debug messages when no change in OPP is required

Otherwise, when enabling a debug build and dynamic debug in the kernel, it
quickly floods the kernel ring buffer and makes debugging of other
subsystems almost impossible, unless manually disabled.

Signed-off-by: Adrián Larumbe <adrian.larumbe@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Move required opps configuration to specialized callback

The required-opps configuration is closely tied to genpd and performance
states at the moment and it is not very obvious that required-opps can
live without genpds. Though we don't support configuring required-opps
for non-genpd cases currently.

This commit aims at separating these parts, where configuring genpds
would be a special case of configuring the required-opps.

Add a specialized callback, set_required_opps(), to the opp table and
set it to different callbacks accordingly.

This shouldn't result in any functional changes for now.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

OPP: Handle all genpd cases together in _set_required_opps()

There is no real need of keeping separate code for single genpd case, it
can be made to work with a simple change.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

OPP: Fix an un-initialized variable usage

smatch complains that 'ret' may be returned un-initialized.

Explicitly return 0 if we reach the end of the function (should
'opp_table->clk_count' be 0).

Fixes: 8174a3a613af ("OPP: Provide a simple implementation to configure multiple clocks")
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Don't drop opp->np reference while it is still in use

The struct dev_pm_opp contains a reference of the DT node, opp->np,
throughout its lifetime. We should increase the refcount for the same
from _opp_add_static_v2(), and drop it while removing the OPP finally.

Signed-off-by: Liang He <windhl@126.com>
[ Viresh: Updated subject / commit log, create _of_clear_opp() and drop
reference from it]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Allow config_clks helper for single clk case

There is a corner case with Tegra30, where we want to skip clk
configuration via dev_pm_opp_set_opp(), but still want the OPP core to
read the "opp-hz" property so we can find the right OPP via freq finding
helpers.

This is the easiest of the ways to make it work, without any special
hacks in the OPP core. Allow config_clks to be passed for single clk
case.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Provide a simple implementation to configure multiple clocks

This provides a simple implementation to configure multiple clocks for a
device.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Assert clk_count == 1 for single clk helpers

Many helpers can be safely called only for devices that have a single
clk associated with them. Assert the same for those routines.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Add key specific assert() method to key finding helpers

The helpers for the clock key, at least, would need to assert that the
helpers are called only for single clock case. Prepare for that by
adding an argument to the key finding helpers.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Compare bandwidths for all paths in _opp_compare_key()

Replicate the same behavior as "rates" here and compare all values
instead of relying on the first entry alone.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Allow multiple clocks for a device

This patch adds support to allow multiple clocks for a device.

The design is pretty much similar to how this is done for regulators,
and platforms can supply their own version of the config_clks() callback
if they have multiple clocks for their device. The core manages the
calls via opp_table->config_clks() eventually.

We have kept both "clk" and "clks" fields in the OPP table structure and
the reason is provided as a comment in _opp_set_clknames(). The same
isn't done for "rates" though and we use rates[0] at most of the places
now.

Co-developed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Tested-by: Jon Hunter <jonathanh@nvidia.com>
Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Make dev_pm_opp_set_opp() independent of frequency

dev_pm_opp_set_opp() can be called for any device, it may or may not
have a frequency value associated with it.

If a frequency value isn't available, we pass 0 to _set_opp(). Make it
optional instead by making _set_opp() accept a pointer instead, as the
frequency value is anyway available in the OPP. This makes
dev_pm_opp_set_opp() and _set_opp() completely independent of any
special key value.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Reuse _opp_compare_key() in _opp_add_static_v2()

Reuse _opp_compare_key() in _opp_add_static_v2() instead of just
comparing frequency while finding suspend frequency. Also add a comment
over _opp_compare_key() explaining its return values.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Remove rate_not_available parameter to _opp_add()

commit 32715be4fe95 ("opp: Fix adding OPP entries in a wrong order if
rate is unavailable") removed the only user of this field, get rid of
rest of it now.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Use consistent names for OPP table instances

The OPP table is called "opp_table" at most of the places and "table" at
few. Make all of them follow the same naming convention, "opp_table".

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Use generic key finding helpers for bandwidth key

Use the recently added generic key findings helpers to find bandwidth
key values.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Use generic key finding helpers for level key

Use the recently added generic key findings helpers to find level key
values.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Add generic key finding helpers and use them for freq APIs

There are three type of helpers, to find exact, ceil, and floor values,
replicated for multiple key types, freq, level, bw. And all of these
helpers share a lot of boilerplate code.

Add generic key finding helpers to reduce code redundancy.

Also update the freq finder helpers to use the new infrastructure.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Remove dev_pm_opp_find_freq_ceil_by_volt()

This was added few years back, but the code that was supposed to use it
never got merged. Remove the unused helper.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Remove custom OPP helper support

The only user of the custom helper is migrated to use
dev_pm_opp_set_config_regulators() interface. Remove the now unused
custom OPP helper support.

This cleans up _set_opp() and leaves a single code path to be used by
all users.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Add dev_pm_opp_get_supplies()

We already have an API for getting voltage information for a single
regulator, dev_pm_opp_get_voltage(), but there is nothing available for
multiple regulator case.

This patch adds a new API, dev_pm_opp_get_supplies(), to get all
information related to the supplies for an OPP.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Make _generic_set_opp_regulator() a config_regulators() interface

In order to reuse the same code path, i.e. clk_set_rate() from
_set_opp(), migrate _generic_set_opp_regulator() to implement a
config_regulators() interface.

It is renamed to _opp_config_regulator_single() and is set as the
preferred config_regulators() interface whenever we have a single
regulator available.

Note that this also drops code responsible for restoring the
voltage/freq in case of errors. We aren't handling that properly
currently, restoring only some of the resources while leaving others out
(like bandwidth and required OPPs). It is better to drop all of it
instead of partial restoration.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Add support for config_regulators() helper

Extend the dev_pm_opp_set_config() interface to allow adding
config_regulators() helpers. This helper will be called to set the
voltages of the regulators from the regular path in _set_opp(), while we
are trying to change the OPP.

This will eventually replace the custom set_opp() helper.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Migrate set-prop-name helper API to use set-config helpers

Now that we have a central API to handle all OPP table configurations,
migrate the set-prop-name family of helpers to use the new
infrastructure.

The return type and parameter to the APIs change a bit due to this,
update the current users as well in the same commit in order to avoid
breaking builds.

Acked-by: Samuel Holland <samuel@sholland.org> # sun50i
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Migrate attach-genpd API to use set-config helpers

Now that we have a central API to handle all OPP table configurations,
migrate the attach-genpd family of helpers to use the new
infrastructure.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Migrate set-opp-helper API to use set-config helpers

Now that we have a central API to handle all OPP table configurations,
migrate the set-opp-helper family of helpers to use the new
infrastructure.

The return type and parameter to the APIs change a bit due to this,
update the current users as well in the same commit in order to avoid
breaking builds.

Remove devm_pm_opp_register_set_opp_helper() as it has no users
currently.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Migrate set-clk-name API to use set-config helpers

Now that we have a central API to handle all OPP table configurations,
migrate the set-clk-name family of helpers to use the new
infrastructure.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Migrate set-supported-hw API to use set-config helpers

Now that we have a central API to handle all OPP table configurations,
migrate the set-supported-hw family of helpers to use the new
infrastructure.

The return type and parameter to the APIs change a bit due to this,
update the current users as well in the same commit in order to avoid
breaking builds.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Migrate set-regulators API to use set-config helpers

Now that we have a central API to handle all OPP table configurations,
migrate the set-regulators family of helpers to use the new
infrastructure.

The return type and parameter to the APIs change a bit due to this,
update the current users as well in the same commit in order to avoid
breaking builds.

Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Add dev_pm_opp_set_config() and friends

The OPP core already have few configuration specific APIs and it is
getting complex or messy for both the OPP core and its users.

Lets introduce a new set of API which will be used for all kind of
different configurations, and shall eventually be used by all the
existing ones.

The new API, returns a unique token instead of a pointer to the OPP
table, which allows the OPP core to drop the resources selectively later
on.

Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Make dev_pm_opp_set_regulators() accept NULL terminated list

Make dev_pm_opp_set_regulators() accept a NULL terminated list of names
instead of making the callers keep the two parameters in sync, which
creates an opportunity for bugs to get in.

Suggested-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Steven Price <steven.price@arm.com> # panfrost
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Track if clock name is configured by platform

Track if the clock name is configured by the platform or not. This is a
preparatory change and will be used by later commits. This also makes
the behavior of the clkname API similar to other ones, which allow
repeated calls to the same API for each CPU.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Fix typo in comment

Replace rate with state.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Fix some kernel-doc comments

Make @freq to @bw in dev_pm_opp_find_bw_ceil() and
dev_pm_opp_find_bw_floor() kernel-doc comment to
remove warnings found by running scripts/kernel-doc,
which is caused by using 'make W=1'.

drivers/opp/core.c:753: warning: Function parameter or member 'bw' not
described in 'dev_pm_opp_find_bw_ceil'
drivers/opp/core.c:753: warning: Excess function parameter 'freq'
description in 'dev_pm_opp_find_bw_ceil'
drivers/opp/core.c:812: warning: Function parameter or member 'bw' not
described in 'dev_pm_opp_find_bw_floor'
drivers/opp/core.c:812: warning: Excess function parameter 'freq'
description in 'dev_pm_opp_find_bw_floor'

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Signed-off-by: Yang Li <yang.lee@linux.alibaba.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Fix error check in dev_pm_opp_attach_genpd()

dev_pm_domain_attach_by_name() may return NULL in some cases,
so IS_ERR() doesn't meet the requirements. Thus fix it.

Fixes: 6319aee10e53 ("opp: Attach genpds to devices from within OPP core")
Signed-off-by: Tang Bin <tangbin@cmss.chinamobile.com>
[ Viresh: Replace ENODATA with ENODEV ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Reorder definition of ceil/floor helpers

Reorder the helpers to keep all freq specific ones, followed by level
and bw.

No functional change.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Add apis to retrieve opps with interconnect bandwidth

Add dev_pm_opp_find_bw_ceil and dev_pm_opp_find_bw_floor to retrieve opps
based on interconnect associated with the opp and bandwidth. The index
variable is the index of the interconnect as specified in the opp table
in Devicetree.

Co-developed-by: Thara Gopinath <thara.gopinath@linaro.org>
Signed-off-by: Thara Gopinath <thara.gopinath@linaro.org>
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: replace usage of found with dedicated list iterator variable

To move the list iterator variable into the list_for_each_entry_*()
macro in the future it should be avoided to use the list iterator
variable after the loop body.

To *never* use the list iterator variable after the loop it was
concluded to use a separate iterator variable instead of a
found boolean [1].

This removes the need to use a found variable and simply checking if
the variable was set, can determine if the break/goto was hit.

Link: https://lore.kernel.org/all/CAHk-=wgRr_D8CB-D9Kg-c=EHreAsk5SqXPwr9Y7k9sA6cWXJ6w@mail.gmail.com/
Signed-off-by: Jakob Koschel <jakobkoschel@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

PM: opp: simplify with dev_err_probe()

Common pattern of handling deferred probe can be simplified with
dev_err_probe(). Less code and the error value gets printed.

Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Add "opp-microwatt" supporting code

Add new property to the OPP: power value. The OPP entry in the DT can have
"opp-microwatt". Add the needed code to handle this new property in the
existing infrastructure.

Signed-off-by: Lukasz Luba <lukasz.luba@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Change type of dev_pm_opp_attach_genpd(names) argument

Elements of the 'names' array are not changed by the code, constify them
for consistency.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: core: Check for pending links before reading required_opp pointers

Commit 4fa82a87ba55 ("opp: Allow required-opps to be used for non genpd
use cases") dereferences the pointers in required_opp_tables but these
might be set to an ERR_PTR if the list still has lazy links pending,
resulting in segfaults. Prior to this patch IS_ERR was also checked on
required_opp_tables[i] before reading ->is_genpd inside
_opp_table_alloc_required_tables, which is at the same time the
predicate to add this table to the lazy list. This segfault is solved
by reordering the checks to bail on lazy pending tables before reading
->is_genpd.

Fixes: 4fa82a87ba55 ("opp: Allow required-opps to be used for non genpd use cases")
Reviewed-by: AngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
Signed-off-by: Marijn Suijten <marijn.suijten@somainline.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Drop empty-table checks from _put functions

The current_opp is released only when whole OPP table is released,
otherwise it's only marked as removed by dev_pm_opp_remove_table().
Functions like dev_pm_opp_put_clkname() and dev_pm_opp_put_supported_hw()
are checking whether OPP table is empty and it's not if current_opp is
set since it holds the refcount of OPP, this produces a noisy warning
from these functions about busy OPP table. Remove the checks to fix it.

Cc: stable@vger.kernel.org
Fixes: 81c4d8a3c414 ("opp: Keep track of currently programmed OPP")
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Allow required-opps to be used for non genpd use cases

Don't limit required_opp_table to genpd only. One possible use case is
cpufreq based devfreq governor, which can use required-opps property to
derive devfreq from cpufreq.

Though the OPP core still doesn't support non-genpd required-opps in
_set_required_opps().

Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Hsin-Yi Wang <hsinyi@chromium.org>
[ Viresh: Update _set_required_opps() to check for genpd ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Change return type of devm_pm_opp_attach_genpd()

Make devm_pm_opp_attach_genpd() to return error code instead of
opp_table pointer in order to have return type consistent with the
other resource-managed OPP helpers.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Change return type of devm_pm_opp_register_set_opp_helper()

Make devm_pm_opp_register_set_opp_helper() to return error code instead
of opp_table pointer in order to have return type consistent with the
other resource-managed OPP helpers.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Add devres wrapper for dev_pm_opp_set_supported_hw

Add devres wrapper for dev_pm_opp_set_supported_hw() to simplify drivers
code.

Signed-off-by: Yangtao Li <tiny.windzz@gmail.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Add devres wrapper for dev_pm_opp_set_regulators

Add devres wrapper for dev_pm_opp_set_regulators() to simplify drivers
code.

Signed-off-by: Yangtao Li <tiny.windzz@gmail.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Add devres wrapper for dev_pm_opp_set_clkname

Add devres wrapper for dev_pm_opp_set_clkname() to simplify drivers code.

Signed-off-by: Yangtao Li <tiny.windzz@gmail.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Don't drop extra references to OPPs accidentally

We are required to call dev_pm_opp_put() from outside of the
opp_table->lock as debugfs removal needs to happen lock-less to avoid
circular dependency issues.

commit cf1fac943c63 ("opp: Reduce the size of critical section in
_opp_kref_release()") tried to fix that introducing a new routine
_opp_get_next() which keeps returning OPPs that can be freed by the
callers and this routine shall be called without holding the
opp_table->lock.

Though the commit overlooked the fact that the OPPs can be referenced by
other users as well and this routine will end up dropping references
which were taken by other users and hence freeing the OPPs prematurely.

In effect, other users of the OPPs will end up having invalid pointers
at hand. We didn't see any crash reports earlier as the exact situation
never happened, though it is certainly possible.

We need a way to mark which OPPs are no longer referenced by the OPP
core, so we don't drop extra references to them accidentally.

This commit adds another OPP flag, "removed", which is used to track
this. And now we should never end up dropping extra references to the
OPPs.

Cc: v5.11+ <stable@vger.kernel.org> # v5.11+
Fixes: cf1fac943c63 ("opp: Reduce the size of critical section in _opp_kref_release()")
Signed-off-by: Beata Michalska <beata.michalska@arm.com>
[ Viresh: Almost rewrote entire patch, added new "removed" field,
rewrote commit log and added the correct Fixes tag. ]
Co-developed-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Don't skip freq update for different frequency

We skip the OPP update if the current and target OPPs are same. This is
fine for the devices that don't support frequency but may cause issues
for the ones that need to program frequency.

An OPP entry doesn't really signify a single operating frequency but
rather the highest frequency at which the other properties of the OPP
entry apply. And we may reach here with different frequency values,
while all of them would point to the same OPP entry in the OPP table.

We just need to update the clock frequency in that case, though in order
to not add special exit points we reuse the code flow from a normal
path.

While at it, rearrange the conditionals in the 'if' statement to check
'enabled' flag at the end.

Fixes: 81c4d8a3c414 ("opp: Keep track of currently programmed OPP")
Signed-off-by: Jonathan Marek <jonathan@marek.ca>
[ Viresh: Improved commit log and subject, rename current_freq as
current_rate, document it, remove local variable and rearrange
code. ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Add function to look up required OPP's for a given OPP

Add a function that allows looking up required OPPs given a source OPP
table, destination OPP table and the source OPP.

Signed-off-by: Saravana Kannan <saravanak@google.com>
Signed-off-by: Hsin-Yi Wang <hsinyi@chromium.org>
[ Viresh: Rearranged code, fixed return errors ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Don't ignore clk_get() errors other than -ENOENT

Not all devices that need to use OPP core need to have clocks, a missing
clock is fine in which case -ENOENT shall be returned by clk_get().

Anything else is an error and must be handled properly.

Reported-by: Dmitry Osipenko <digetx@gmail.com>
Tested-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Update bandwidth requirements based on scaling up/down

The bandwidth must be scaled at a different point in the code flow based
on if we are scaling up or down the frequency, otherwise this may cause
undesired effects as the device will try to use more of the memory
bandwidth which may be shared across several devices. Much like how
regulators and required-opps are programmed.

Reported-by: Dmitry Osipenko <digetx@gmail.com>
Reported-by: Akhil P Oommen <akhilpo@codeaurora.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Dmitry Osipenko <digetx@gmail.com>

opp: Allow lazy-linking of required-opps

The OPP core currently requires the required opp tables to be available
before the dependent OPP table is added, as it needs to create links
from the dependent OPP table to the required ones. This may not be
convenient for all the platforms though, as this requires strict
ordering for probing the drivers.

This patch allows lazy-linking of the required-opps. The OPP tables for
which the required-opp-tables aren't available at the time of their
initialization, are added to a special list of OPP tables:
lazy_opp_tables. Later on, whenever a new OPP table is registered with
the OPP core, we check if it is required by an OPP table in the pending
list; if yes, then we complete the linking then and there.

An OPP table is marked unusable until the time all its required-opp
tables are available. And if lazy-linking fails for an OPP table, the
OPP core disables all of its OPPs to make sure no one can use them.

Tested-by: Hsin-Yi Wang <hsinyi@chromium.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Remove dev_pm_opp_set_bw()

All the users have migrated to dev_pm_opp_set_opp() now, get rid of the
duplicate API, dev_pm_opp_set_bw(), which only performs a part of the new API.

While at it, remove the unnecessary parameter to _set_opp_bw().

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Dmitry Osipenko <digetx@gmail.com>

opp: Implement dev_pm_opp_set_opp()

The new helper dev_pm_opp_set_opp() can be used for configuring the
devices for a particular OPP and can be used by different type of
devices, even the ones which don't change frequency (like power
domains).

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Dmitry Osipenko <digetx@gmail.com>

opp: Update parameters of _set_opp_custom()

Drop the unnecessary parameters and follow the pattern from
_generic_set_opp_regulator().

While at it, also remove the local variable old_freq.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Dmitry Osipenko <digetx@gmail.com>

opp: Allow _generic_set_opp_clk_only() to work for non-freq devices

In order to avoid conditional statements at the caller site, this patch
updates _generic_set_opp_clk_only() to work for devices that don't
change frequency (like power domains, etc.). Return 0 if the clk pointer
passed to this routine is not valid.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Dmitry Osipenko <digetx@gmail.com>

opp: Allow _generic_set_opp_regulator() to work for non-freq devices

The _generic_set_opp_regulator() helper will be used for devices which
don't change frequency (like power domains, etc.) later on, prepare for
that by not relying on frequency for making decisions here.

While at it, update its parameters to pass only what is necessary.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Dmitry Osipenko <digetx@gmail.com>

opp: Allow _set_opp() to work for non-freq devices

The _set_opp() helper will be used for devices which don't change frequency
(like power domains, etc.) later on, prepare for that by not relying on
frequency for making decisions here.

While at it, also update the debug print to contain all relevant
information.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Dmitry Osipenko <digetx@gmail.com>

opp: Split _set_opp() out of dev_pm_opp_set_rate()

The _set_opp() helper will be used for devices which don't change their
frequency (like power domains, etc.) later on, prepare for that by
breaking the generic part out of dev_pm_opp_set_rate().

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Dmitry Osipenko <digetx@gmail.com>

opp: Keep track of currently programmed OPP

The dev_pm_opp_set_rate() helper needs to know the currently programmed
OPP to make few decisions and currently we try to find it on every
invocation of this routine.

Lets start keeping track of the current_opp programmed for the devices
of the opp table, that will be quite useful going forward.

If we fail to find the current OPP, we pick the first one available in
the list, as the list is in ascending order of frequencies, level, or
bandwidth and that's the best guess we can make anyway.

Note that we used to do the frequency comparison a bit early in
dev_pm_opp_set_rate() previously, and now instead we check the target
opp, which shall be more accurate anyway.

We need to make sure that current_opp's memory doesn't get freed while
it is being used and so we keep a reference of it until the time it is
used.

Now that current_opp will always be set, we can drop some unnecessary
checks as well.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Dmitry Osipenko <digetx@gmail.com>

opp: No need to check clk for errors

Clock is not optional for users who call into dev_pm_opp_set_rate().
Remove the unnecessary checks.

While at it also drop the local variable for clk and use opp_table->clk
instead.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Dmitry Osipenko <digetx@gmail.com>

opp: Rename _opp_set_rate_zero()

This routine has nothing to do with frequency, it just disables all the
resources previously enabled. Rename it to match its purpose.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Dmitry Osipenko <digetx@gmail.com>

opp: Make _set_opp_custom() work without regulators

Check whether OPP table has regulators in _set_opp_custom() and set up
dev_pm_set_opp_data accordingly. Now _set_opp_custom() works properly,
i.e. it doesn't crash if OPP table doesn't have assigned regulators.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
[ Viresh: Rearrange the routine a bit ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Prepare for ->set_opp() helper to work without regulators

Until now the ->set_opp() helper (i.e. special implementation for
setting the OPPs for platforms) was implemented only to take care of
multiple regulators case, but going forward we would need that for other
use cases as well.

This patch prepares for that by allocating the regulator specific part
from dev_pm_opp_set_regulators() and the opp helper part from
dev_pm_opp_register_set_opp_helper().

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Dmitry Osipenko <digetx@gmail.com>

opp: Handle missing OPP table in dev_pm_opp_xlate_performance_state()

NVIDIA Tegra SoCs have a power domains topology such that child domains
only clamp a power rail, while parent domain controls shared performance
state of the multiple child domains. In this case child's domain doesn't
need to have OPP table. Hence we want to allow children power domains to
pass performance state to the parent domain if child's domain doesn't have
OPP table.

The dev_pm_opp_xlate_performance_state() gets src_table=NULL if a child
power domain doesn't have OPP table and in this case we should pass the
performance state to the parent domain.

Tested-by: Peter Geis <pgwipeout@gmail.com>
Tested-by: Nicolas Chauvet <kwizart@gmail.com>
Tested-by: Matt Merhar <mattmerhar@protonmail.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Add devm_pm_opp_attach_genpd

Add resource-managed version of dev_pm_opp_attach_genpd().

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
[ Viresh: Manually apply the patch and relocate the routines ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Add devm_pm_opp_register_set_opp_helper

Add resource-managed version of dev_pm_opp_register_set_opp_helper().

Tested-by: Peter Geis <pgwipeout@gmail.com>
Tested-by: Nicolas Chauvet <kwizart@gmail.com>
Tested-by: Matt Merhar <mattmerhar@protonmail.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
[ Viresh: Manually apply the patch and relocate the routines ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Defer acquiring the clk until OPPs are added

We acquire the clk at the time the OPP table is allocated, though it
works fine, it is not the best place to do so. One of the main reason
being we may need to acquire it again from dev_pm_opp_set_clkname() if
the platform wants another clock to be acquired instead.

There is also requirement from some of the platforms where they do not
want the OPP core to manage the clock at all.

This patch hence defers acquiring the clk until the time we are certain
about which clk we need to acquire and if we really need to acquire one.
With this commit, the clk will get acquired either from
dev_pm_opp_set_clkname() or while we initialize the OPPs within the
table.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Dmitry Osipenko <digetx@gmail.com>

opp: Add dev_pm_opp_sync_regulators()

Extend OPP API with dev_pm_opp_sync_regulators() function, which syncs
voltage state of regulators.

Tested-by: Peter Geis <pgwipeout@gmail.com>
Tested-by: Nicolas Chauvet <kwizart@gmail.com>
Tested-by: Matt Merhar <mattmerhar@protonmail.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
[ Viresh: Added unlikely() ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Add dev_pm_opp_get_required_pstate()

Add dev_pm_opp_get_required_pstate() which allows OPP users to retrieve
required performance state of a given OPP.

Tested-by: Peter Geis <pgwipeout@gmail.com>
Tested-by: Nicolas Chauvet <kwizart@gmail.com>
Tested-by: Matt Merhar <mattmerhar@protonmail.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Add dev_pm_opp_find_level_ceil()

Add a ceil version of the dev_pm_opp_find_level(). It's handy to have if
levels don't start from 0 in OPP table and zero usually means a minimal
level.

Tested-by: Peter Geis <pgwipeout@gmail.com>
Tested-by: Nicolas Chauvet <kwizart@gmail.com>
Tested-by: Matt Merhar <mattmerhar@protonmail.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Staticize _add_opp_table()

_add_opp_table() isn't used outside of core.c, mark it static.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Filter out OPPs based on availability of a required-OPP

A required OPP may not be available, and thus, all OPPs which are using
this required OPP should be unavailable too.

Tested-by: Peter Geis <pgwipeout@gmail.com>
Tested-by: Nicolas Chauvet <kwizart@gmail.com>
Tested-by: Matt Merhar <mattmerhar@protonmail.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Fix adding OPP entries in a wrong order if rate is unavailable

Fix adding OPP entries in a wrong (opposite) order if OPP rate is
unavailable. The OPP comparison was erroneously skipped, thus OPPs
were left unsorted.

Tested-by: Peter Geis <pgwipeout@gmail.com>
Tested-by: Nicolas Chauvet <kwizart@gmail.com>
Tested-by: Matt Merhar <mattmerhar@protonmail.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Call the missing clk_put() on error

Fix the clock reference counting by calling the missing clk_put() in the
error path.

Cc: v5.10 <stable@vger.kernel.org> # v5.10
Fixes: dd461cd9183f ("opp: Allow dev_pm_opp_get_opp_table() to return -EPROBE_DEFER")
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: fix memory leak in _allocate_opp_table

In function _allocate_opp_table, opp_dev is allocated and referenced
by opp_table via _add_opp_dev. But in the case that the subsequent calls
return -EPROBE_DEFER, it will jump to err label and opp_table will be
freed. Then opp_dev becomes an unreferenced object to cause memory leak.
So let's call _remove_opp_dev to do the cleanup.

This fixes the following kmemleak report:

unreferenced object 0xffff000801524a00 (size 128):
comm "swapper/0", pid 1, jiffies 4294892465 (age 84.616s)
hex dump (first 32 bytes):
40 00 56 01 08 00 ff ff 40 00 56 01 08 00 ff ff @.V.....@.V.....
b8 52 77 7f 08 00 ff ff 00 3c 4c 00 08 00 ff ff .Rw......<L.....
backtrace:
[<00000000b1289fb1>] kmemleak_alloc+0x30/0x40
[<0000000056da48f0>] kmem_cache_alloc+0x3d4/0x588
[<00000000a84b3b0e>] _add_opp_dev+0x2c/0x88
[<0000000062a380cd>] _add_opp_table_indexed+0x124/0x268
[<000000008b4c8f1f>] dev_pm_opp_of_add_table+0x20/0x1d8
[<00000000e5316798>] dev_pm_opp_of_cpumask_add_table+0x48/0xf0
[<00000000db0a8ec2>] dt_cpufreq_probe+0x20c/0x448
[<0000000030a3a26c>] platform_probe+0x68/0xd8
[<00000000c618e78d>] really_probe+0xd0/0x3a0
[<00000000642e856f>] driver_probe_device+0x58/0xb8
[<00000000f10f5307>] device_driver_attach+0x74/0x80
[<0000000004f254b8>] __driver_attach+0x58/0xe0
[<0000000009d5d19e>] bus_for_each_dev+0x70/0xc8
[<0000000000d22e1c>] driver_attach+0x24/0x30
[<0000000001d4e952>] bus_add_driver+0x14c/0x1f0
[<0000000089928aaa>] driver_register+0x64/0x120

Cc: v5.10 <stable@vger.kernel.org> # v5.10
Fixes: dd461cd9183f ("opp: Allow dev_pm_opp_get_opp_table() to return -EPROBE_DEFER")
Signed-off-by: Quanyang Wang <quanyang.wang@windriver.com>
[ Viresh: Added the stable tag ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Allow dev_pm_opp_put_*() APIs to accept NULL opp_table

This allows the callers to drop the unnecessary checks.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Don't create an OPP table from dev_pm_opp_get_opp_table()

It has been found that some users (like cpufreq-dt and others on LKML)
have abused the helper dev_pm_opp_get_opp_table() to create the OPP
table instead of just finding it, which is the wrong thing to do. This
routine was meant for OPP core's internal working and exposed the whole
functionality by mistake.

Change the scope of dev_pm_opp_get_opp_table() to only finding the
table. The internal helpers _opp_get_opp_table*() are thus renamed to
_add_opp_table*(), dev_pm_opp_get_opp_table_indexed() is removed (as we
don't need the index field for finding the OPP table) and so the only
user, genpd, is updated.

Note that the prototype of _add_opp_table() was already left in opp.h by
mistake when it was removed earlier and so we weren't required to add it
now.

Acked-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Reduce the size of critical section in _opp_kref_release()

There is a lot of stuff here which can be done outside of the
opp_table->lock, do that. This helps avoiding a circular dependency
lockdeps around debugfs.

Reported-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Don't return opp_dev from _find_opp_dev()

The caller of _find_opp_dev() only needs to know if the opp_dev is there
in the list or not.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Allocate the OPP table outside of opp_table_lock

There is no critical section which needs protection with locks while
allocating an OPP table, except while adding it to the opp_tables list.
And taking the opp_table_lock for the entire duration causes circular
dependency issues like the one mentioned below. This patch takes
another approach to reduce the size of the critical section to avoid
such issues, the details of that are present within the patch.

======================================================
WARNING: possible circular locking dependency detected
5.4.72 #14 Not tainted
------------------------------------------------------
chrome/1865 is trying to acquire lock:
ffffffdd34921750 (opp_table_lock){+.+.}, at: _find_opp_table+0x34/0x74

but task is already holding lock:
ffffff81f0fc71a8 (reservation_ww_class_mutex){+.+.}, at: submit_lock_objects+0x70/0x1ec

fscrypt: AES-256-CTS-CBC using implementation "cts-cbc-aes-ce"
which lock already depends on the new lock.

the existing dependency chain (in reverse order) is:
-> #4 (reservation_ww_class_mutex){+.+.}:
__mutex_lock_common+0xec/0xc0c
ww_mutex_lock_interruptible+0x5c/0xc4
msm_gem_fault+0x2c/0x124
__do_fault+0x40/0x16c
handle_mm_fault+0x7cc/0xd98
do_page_fault+0x230/0x3b4
do_translation_fault+0x5c/0x78
do_mem_abort+0x4c/0xb4
el0_da+0x1c/0x20
-> #3 (&mm->mmap_sem){++++}:
__might_fault+0x70/0x98
compat_filldir+0xf8/0x48c
dcache_readdir+0x70/0x1dc
iterate_dir+0xd4/0x180
__arm64_compat_sys_getdents+0xa0/0x19c
el0_svc_common+0xa8/0x178
el0_svc_compat_handler+0x2c/0x40
el0_svc_compat+0x8/0x10
-> #2 (&sb->s_type->i_mutex_key#3){++++}:
down_write+0x54/0x16c
start_creating+0x68/0x128
debugfs_create_dir+0x28/0x114
opp_debug_register+0x8c/0xc0
_add_opp_dev_unlocked+0x5c/0x70
_add_opp_dev+0x38/0x58
_opp_get_opp_table+0xdc/0x1ac
dev_pm_opp_get_opp_table_indexed+0x24/0x30
dev_pm_opp_of_add_table_indexed+0x48/0x84
of_genpd_add_provider_onecell+0xc0/0x1b8
rpmhpd_probe+0x240/0x268
platform_drv_probe+0x90/0xb0
really_probe+0x134/0x2ec
driver_probe_device+0x64/0xfc
__device_attach_driver+0x8c/0xa4
bus_for_each_drv+0x90/0xd8
__device_attach+0xc0/0x148
device_initial_probe+0x20/0x2c
bus_probe_device+0x34/0x94
device_add+0x1fc/0x3b0
of_device_add+0x3c/0x4c
of_platform_device_create_pdata+0xb8/0xfc
of_platform_bus_create+0x1e4/0x368
of_platform_populate+0x70/0xbc
devm_of_platform_populate+0x58/0xa0
rpmh_rsc_probe+0x36c/0x3cc
platform_drv_probe+0x90/0xb0
really_probe+0x134/0x2ec
driver_probe_device+0x64/0xfc
__device_attach_driver+0x8c/0xa4
bus_for_each_drv+0x90/0xd8
__device_attach+0xc0/0x148
device_initial_probe+0x20/0x2c
bus_probe_device+0x34/0x94
device_add+0x1fc/0x3b0
of_device_add+0x3c/0x4c
of_platform_device_create_pdata+0xb8/0xfc
of_platform_bus_create+0x1e4/0x368
of_platform_bus_create+0x230/0x368
of_platform_populate+0x70/0xbc
of_platform_default_populate_init+0xa8/0xc0
do_one_initcall+0x1c8/0x3fc
do_initcall_level+0xb4/0x10c
do_basic_setup+0x30/0x48
kernel_init_freeable+0x124/0x1a4
kernel_init+0x14/0x104
ret_from_fork+0x10/0x18
-> #1 (&opp_table->lock){+.+.}:
__mutex_lock_common+0xec/0xc0c
mutex_lock_nested+0x40/0x50
_add_opp_dev+0x2c/0x58
_opp_get_opp_table+0xdc/0x1ac
dev_pm_opp_get_opp_table_indexed+0x24/0x30
dev_pm_opp_of_add_table_indexed+0x48/0x84
of_genpd_add_provider_onecell+0xc0/0x1b8
rpmhpd_probe+0x240/0x268
platform_drv_probe+0x90/0xb0
really_probe+0x134/0x2ec
driver_probe_device+0x64/0xfc
__device_attach_driver+0x8c/0xa4
bus_for_each_drv+0x90/0xd8
__device_attach+0xc0/0x148
device_initial_probe+0x20/0x2c
bus_probe_device+0x34/0x94
device_add+0x1fc/0x3b0
of_device_add+0x3c/0x4c
of_platform_device_create_pdata+0xb8/0xfc
of_platform_bus_create+0x1e4/0x368
of_platform_populate+0x70/0xbc
devm_of_platform_populate+0x58/0xa0
rpmh_rsc_probe+0x36c/0x3cc
platform_drv_probe+0x90/0xb0
really_probe+0x134/0x2ec
driver_probe_device+0x64/0xfc
__device_attach_driver+0x8c/0xa4
bus_for_each_drv+0x90/0xd8
__device_attach+0xc0/0x148
device_initial_probe+0x20/0x2c
bus_probe_device+0x34/0x94
device_add+0x1fc/0x3b0
of_device_add+0x3c/0x4c
of_platform_device_create_pdata+0xb8/0xfc
of_platform_bus_create+0x1e4/0x368
of_platform_populate+0x70/0xbc
devm_of_platform_populate+0x58/0xa0
rpmh_rsc_probe+0x36c/0x3cc
platform_drv_probe+0x90/0xb0
really_probe+0x134/0x2ec
driver_probe_device+0x64/0xfc
__device_attach_driver+0x8c/0xa4
bus_for_each_drv+0x90/0xd8
__device_attach+0xc0/0x148
device_initial_probe+0x20/0x2c
bus_probe_device+0x34/0x94
device_add+0x1fc/0x3b0
of_device_add+0x3c/0x4c
of_platform_device_create_pdata+0xb8/0xfc
of_platform_bus_create+0x1e4/0x368
of_platform_bus_create+0x230/0x368
of_platform_populate+0x70/0xbc
of_platform_default_populate_init+0xa8/0xc0
do_one_initcall+0x1c8/0x3fc
do_initcall_level+0xb4/0x10c
do_basic_setup+0x30/0x48
kernel_init_freeable+0x124/0x1a4
kernel_init+0x14/0x104
ret_from_fork+0x10/0x18
-> #0 (opp_table_lock){+.+.}:
__lock_acquire+0xee4/0x2450
lock_acquire+0x1cc/0x210
__mutex_lock_common+0xec/0xc0c
mutex_lock_nested+0x40/0x50
_find_opp_table+0x34/0x74
dev_pm_opp_find_freq_exact+0x2c/0xdc
a6xx_gmu_resume+0xc8/0xecc
a6xx_pm_resume+0x148/0x200
adreno_resume+0x28/0x34
pm_generic_runtime_resume+0x34/0x48
__rpm_callback+0x70/0x10c
rpm_callback+0x34/0x8c
rpm_resume+0x414/0x550
__pm_runtime_resume+0x7c/0xa0
msm_gpu_submit+0x60/0x1c0
msm_ioctl_gem_submit+0xadc/0xb60
drm_ioctl_kernel+0x9c/0x118
drm_ioctl+0x27c/0x408
drm_compat_ioctl+0xcc/0xdc
__se_compat_sys_ioctl+0x100/0x206c
__arm64_compat_sys_ioctl+0x20/0x2c
el0_svc_common+0xa8/0x178
el0_svc_compat_handler+0x2c/0x40
el0_svc_compat+0x8/0x10
other info that might help us debug this:
Chain exists of: opp_table_lock --> &mm->mmap_sem --> reservation_ww_class_mutex
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(reservation_ww_class_mutex);
lock(&mm->mmap_sem);
lock(reservation_ww_class_mutex);
lock(opp_table_lock);

*** DEADLOCK ***

3 locks held by chrome/1865:
#0: ffffff81edecc0d8 (&dev->struct_mutex){+.+.}, at: msm_ioctl_gem_submit+0x264/0xb60
#1: ffffff81d0000870 (reservation_ww_class_acquire){+.+.}, at: msm_ioctl_gem_submit+0x8e8/0xb60
#2: ffffff81f0fc71a8 (reservation_ww_class_mutex){+.+.}, at: submit_lock_objects+0x70/0x1ec
stack backtrace:
CPU: 0 PID: 1865 Comm: chrome Not tainted 5.4.72 #14
Hardware name: Google Lazor (rev1+) with LTE (DT)
Call trace:
dump_backtrace+0x0/0x158
show_stack+0x20/0x2c
dump_stack+0xc8/0x160
print_circular_bug+0x2c4/0x2c8
check_noncircular+0x1a8/0x1b0
__lock_acquire+0xee4/0x2450
lock_acquire+0x1cc/0x210
__mutex_lock_common+0xec/0xc0c
mutex_lock_nested+0x40/0x50
_find_opp_table+0x34/0x74
dev_pm_opp_find_freq_exact+0x2c/0xdc
a6xx_gmu_resume+0xc8/0xecc
a6xx_pm_resume+0x148/0x200
adreno_resume+0x28/0x34
pm_generic_runtime_resume+0x34/0x48
__rpm_callback+0x70/0x10c
rpm_callback+0x34/0x8c
rpm_resume+0x414/0x550
__pm_runtime_resume+0x7c/0xa0
msm_gpu_submit+0x60/0x1c0
msm_ioctl_gem_submit+0xadc/0xb60
drm_ioctl_kernel+0x9c/0x118
drm_ioctl+0x27c/0x408
drm_compat_ioctl+0xcc/0xdc
__se_compat_sys_ioctl+0x100/0x206c
__arm64_compat_sys_ioctl+0x20/0x2c
el0_svc_common+0xa8/0x178
el0_svc_compat_handler+0x2c/0x40
el0_svc_compat+0x8/0x10

Reported-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Always add entries in dev_list with opp_table->lock held

The readers of dev_list expect the updates to it to take place from
within the opp_table->lock and this is missing in the case where the
dev_list is updated for already managed OPPs.

Fix that by calling _add_opp_dev() from there and remove the now unused
_add_opp_dev_unlocked() callback. While at it, also reduce the length of
the critical section in _add_opp_dev().

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Reduce the size of critical section in _opp_table_kref_release()

There is a lot of stuff here which can be done outside of the big
opp_table_lock, do that. This helps avoiding few circular dependency
lockdeps around debugfs and interconnects.

Reported-by: Rob Clark <robdclark@gmail.com>
Reported-by: Dmitry Osipenko <digetx@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Fix early exit from dev_pm_opp_register_set_opp_helper()

We returned earlier by mistake even when there were no failures. Fix it.

Fixes: dd461cd9183f ("opp: Allow dev_pm_opp_get_opp_table() to return -EPROBE_DEFER")
Reported-by: Naresh Kamboju <naresh.kamboju@linaro.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Naresh Kamboju <naresh.kamboju@linaro.com>

opp: Allow opp-level to be set to 0

The DT bindings don't put such a constraint, nor should the kernel. It
is perfectly fine for opp-level to be set to 0, if we need to put the
performance state votes for a domain for a particular OPP.

Reported-by: Stephan Gerhold <stephan@gerhold.net>
Tested-by: Stephan Gerhold <stephan@gerhold.net>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Prevent memory leak in dev_pm_opp_attach_genpd()

If dev_pm_opp_attach_genpd() is called multiple times (once for each CPU
sharing the table), then it would result in unwanted behavior like
memory leak, attaching the domain multiple times, etc.

Handle that by checking and returning earlier if the domains are already
attached. Now that dev_pm_opp_detach_genpd() can get called multiple
times as well, we need to protect that too.

Note that the virtual device pointers aren't returned in this case, as
they may become unavailable to some callers during the middle of the
operation.

Reported-by: Stephan Gerhold <stephan@gerhold.net>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Set required OPPs in reverse order when scaling down

The OPP core already has well-defined semantics to ensure required
OPPs/regulators are set before/after the frequency change, depending
on if we scale up or down.

Similar requirements might exist for the order of required OPPs
when multiple power domains need to be scaled for a frequency change.

For example, on Qualcomm platforms using CPR (Core Power Reduction),
we need to scale the VDDMX and CPR power domain. When scaling up,
MX should be scaled up before CPR. When scaling down, CPR should be
scaled down before MX.

In general, if there are multiple "required-opps" in the device tree
I would expect that the order is either irrelevant, or there is some
dependency between the power domains. In that case, the power domains
should be scaled down in reverse order.

This commit updates _set_required_opps() to set required OPPs in
reverse order when scaling down.

Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
[ Viresh: Fix rebase conflict and minor rearrangement of the code ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Reduce code duplication in _set_required_opps()

Move call to dev_pm_genpd_set_performance_state() to a separate
function so we can avoid duplicating the code for the single and
multiple genpd case.

Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
[ Viresh: Validate virtual device before use ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Drop unnecessary check from dev_pm_opp_attach_genpd()

Since commit c0ab9e0812da ("opp: Allocate genpd_virt_devs from
dev_pm_opp_attach_genpd()"), the allocation of the virtual devices is
moved to dev_pm_opp_attach_genpd() and this check isn't required anymore
as it will always fail. Drop it.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Don't drop reference for an OPP table that was never parsed

dev_pm_opp_remove_table() should drop a reference to the OPP table only
if the DT OPP table was parsed earlier with a call to
dev_pm_opp_of_add_table() earlier. Else it may end up dropping the
reference to the OPP table, which was added as a result of other calls
like dev_pm_opp_set_clkname(). And would hence result in undesirable
behavior later on when caller would try to free the resource again.

Fixes: 03758d60265c ("opp: Replace list_kref with a local counter")
Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org>
Reported-by: Anders Roxell <anders.roxell@linaro.org>
Tested-by: Naresh Kamboju <naresh.kamboju@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Allow dev_pm_opp_get_opp_table() to return -EPROBE_DEFER

The OPP core manages various resources, e.g. clocks or interconnect paths.
These resources are looked up when the OPP table is allocated once
dev_pm_opp_get_opp_table() is called the first time (either directly
or indirectly through one of the many helper functions).

At this point, the resources may not be available yet, i.e. looking them
up will result in -EPROBE_DEFER. Unfortunately, dev_pm_opp_get_opp_table()
is currently unable to propagate this error code since it only returns
the allocated OPP table or NULL.

This means that all consumers of the OPP core are required to make sure
that all necessary resources are available. Usually this happens by
requesting them, checking the result and releasing them immediately after.

For example, we have added "dev_pm_opp_of_find_icc_paths(dev, NULL)" to
several drivers now just to make sure the interconnect providers are
ready before the OPP table is allocated. If this call is missing,
the OPP core will only warn about this and then attempt to continue
without interconnect. This will eventually fail horribly, e.g.:

cpu cpu0: _allocate_opp_table: Error finding interconnect paths: -517
... later ...
of: _read_bw: Mismatch between opp-peak-kBps and paths (1 0)
cpu cpu0: _opp_add_static_v2: opp key field not found
cpu cpu0: _of_add_opp_table_v2: Failed to add OPP, -22

This example happens when trying to use interconnects for a CPU OPP
table together with qcom-cpufreq-nvmem.c. qcom-cpufreq-nvmem calls
dev_pm_opp_set_supported_hw(), which ends up allocating the OPP table
early. To fix the problem with the current approach we would need to add
yet another call to dev_pm_opp_of_find_icc_paths(dev, NULL).
But actually qcom-cpufreq-nvmem.c has nothing to do with interconnects...

This commit attempts to make this more robust by allowing
dev_pm_opp_get_opp_table() to return an error pointer. Fixing all
the usages is trivial because the function is usually used indirectly
through another helper (e.g. dev_pm_opp_set_supported_hw() above).
These other helpers already return an error pointer.

The example above then works correctly because set_supported_hw() will
return -EPROBE_DEFER, and qcom-cpufreq-nvmem.c already propagates that
error. It should also be possible to remove the remaining usages of
"dev_pm_opp_of_find_icc_paths(dev, NULL)" from other drivers as well.

Note that this commit currently only handles -EPROBE_DEFER for the
clock/interconnects within _allocate_opp_table(). Other errors are just
ignored as before. Eventually those should be propagated as well.

Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
Acked-by: Krzysztof Kozlowski <krzk@kernel.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
[ Viresh: skip checking return value of dev_pm_opp_get_opp_table() for
EPROBE_DEFER in domain.c, fix NULL return value and reorder
code a bit in core.c, and update exynos-asv.c ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Remove _dev_pm_opp_find_and_remove_table() wrapper

Remove the unnecessary wrapper and merge
_dev_pm_opp_find_and_remove_table() with dev_pm_opp_remove_table().

Tested-by: Rajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Split out _opp_set_rate_zero()

Create separate routine _opp_set_rate_zero() to handle !target_freq
case.

Tested-by: Rajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Reuse the enabled flag in !target_freq path

The OPP core needs to track if the resources of devices are
enabled/configured or not, as it disables the resources when target_freq
is set to 0.

Handle that with the new enabled flag and remove otherwise complex
conditional statements.

Tested-by: Rajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Rename regulator_enabled and use it as status of all resources

Expand the scope of the regulator_enabled flag and use it to track
status of all the resources. This will be used for other stuff in the
next patch.

Tested-by: Rajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Enable resources again if they were disabled earlier

dev_pm_opp_set_rate() can now be called with freq = 0 in order
to either drop performance or bandwidth votes or to disable
regulators on platforms which support them.

In such cases, a subsequent call to dev_pm_opp_set_rate() with
the same frequency ends up returning early because 'old_freq == freq'

Instead make it fall through and put back the dropped performance
and bandwidth votes and/or enable back the regulators.

Cc: v5.3+ <stable@vger.kernel.org> # v5.3+
Fixes: cd7ea582866f ("opp: Make dev_pm_opp_set_rate() handle freq = 0 to drop performance votes")
Reported-by: Sajida Bhanu <sbhanu@codeaurora.org>
Reviewed-by: Sibi Sankar <sibis@codeaurora.org>
Reported-by: Matthias Kaehlcke <mka@chromium.org>
Tested-by: Matthias Kaehlcke <mka@chromium.org>
Reviewed-by: Stephen Boyd <sboyd@kernel.org>
Signed-off-by: Rajendra Nayak <rnayak@codeaurora.org>
[ Viresh: Don't skip clk_set_rate() and massaged changelog ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Put opp table in dev_pm_opp_set_rate() if _set_opp_bw() fails

We get the opp_table pointer at the top of the function and so we should
put the pointer at the end of the function like all other exit paths
from this function do.

Cc: v5.8+ <stable@vger.kernel.org> # v5.8+
Fixes: b00e667a6d8b ("opp: Remove bandwidth votes when target_freq is zero")
Reviewed-by: Rajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: Stephen Boyd <swboyd@chromium.org>
[ Viresh: Split the patch into two ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Put opp table in dev_pm_opp_set_rate() for empty tables

We get the opp_table pointer at the top of the function and so we should
put the pointer at the end of the function like all other exit paths
from this function do.

Cc: v5.7+ <stable@vger.kernel.org> # v5.7+
Fixes: aca48b61f963 ("opp: Manage empty OPP tables with clk handle")
Reviewed-by: Rajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: Stephen Boyd <swboyd@chromium.org>
[ Viresh: Split the patch into two ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Add and export helper to set bandwidth

Add and export 'dev_pm_opp_set_bw' to set the bandwidth
levels associated with an OPP.

Signed-off-by: Sibi Sankar <sibis@codeaurora.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Allow disabled OPPs in dev_pm_opp_get_freq()

Allow dev_pm_opp_get_freq() to work for disabled OPPs.

Signed-off-by: Andrew-sh.Cheng <andrew-sh.cheng@mediatek.com>
[ Viresh: Massaged commit log ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: core: Add missing export for dev_pm_opp_adjust_voltage

Export dev_pm_opp_adjust_voltage() as it may be used by modules later
on.

Signed-off-by: Valdis Kletnieks <valdis.kletnieks@vt.edu>
[ Viresh: Rewrote commit log ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Remove bandwidth votes when target_freq is zero

We already drop several votes when target_freq is set to zero, drop
bandwidth votes as well.

Reported-by: Sibi Sankar <sibis@codeaurora.org>
Reviewed-by: Georgi Djakov <georgi.djakov@linaro.org>
Tested-by: Georgi Djakov <georgi.djakov@linaro.org>
Reviewed-by: Sibi Sankar <sibis@codeaurora.org>
Tested-by: Sibi Sankar <sibis@codeaurora.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: core: add regulators enable and disable

Add enable regulators to dev_pm_opp_set_regulators() and disable
regulators to dev_pm_opp_put_regulators(). Even if bootloader
leaves regulators enabled, they should be enabled in kernel in
order to increase the reference count.

Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Acked-by: Clément Péron <peron.clem@gmail.com>
Tested-by: Clément Péron <peron.clem@gmail.com>
Signed-off-by: Kamil Konieczny <k.konieczny@samsung.com>
[ Viresh: Enable the regulator only after it is programmed and add a
flag to track its status. ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Reorder the code for !target_freq case

Reorder the code a bit to make it more readable. Add additional comment
as well.

Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Acked-by: Clément Péron <peron.clem@gmail.com>
Tested-by: Clément Péron <peron.clem@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Update the bandwidth on OPP frequency changes

If the OPP bandwidth values are populated, we want to switch also the
interconnect bandwidth in addition to frequency and voltage.

Signed-off-by: Georgi Djakov <georgi.djakov@linaro.org>
Reviewed-by: Matthias Kaehlcke <mka@chromium.org>
Reviewed-by: Sibi Sankar <sibis@codeaurora.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Add support for parsing interconnect bandwidth

The OPP bindings now support bandwidth values, so add support to parse it
from device tree and store it into the new dev_pm_opp_icc_bw struct, which
is part of the dev_pm_opp.

Signed-off-by: Georgi Djakov <georgi.djakov@linaro.org>
Reviewed-by: Matthias Kaehlcke <mka@chromium.org>
[ Viresh: Create _read_bw() and use it, renamed _of_find_icc_paths() to
dev_pm_opp_of_find_icc_paths(), exported it and made opp_table
argument optional. Also drop the depends on from Kconfig. ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Add helpers for reading the binding properties

The opp-hz DT property is not mandatory and we may use another property
as a key in the OPP table. Add helper functions to simplify the reading
and comparing the keys.

Signed-off-by: Saravana Kannan <saravanak@google.com>
Signed-off-by: Georgi Djakov <georgi.djakov@linaro.org>
Reviewed-by: Matthias Kaehlcke <mka@chromium.org>
Reviewed-by: Sibi Sankar <sibis@codeaurora.org>
[ Viresh: Removed an unnecessary comment ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Manage empty OPP tables with clk handle

With OPP core now supporting DVFS for IO devices, we have instances of
IO devices (same IP block) which require an OPP on some platforms/SoCs
while just needing to scale the clock on some others.

In order to avoid conditional code in every driver which supports such
devices (to check for availability of OPPs and then deciding to do
either dev_pm_opp_set_rate() or clk_set_rate()) add support to manage
empty OPP tables with a clk handle.

This makes dev_pm_opp_set_rate() equivalent of a clk_set_rate() for
devices with just a clk and no OPPs specified, and makes
dev_pm_opp_set_rate(0) bail out without throwing an error.

Signed-off-by: Rajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Replace list_kref with a local counter

A kref or refcount isn't the right tool to be used here for counting
number of devices that are sharing the static OPPs created for the OPP
table. For example, we are reinitializing the kref again, after it
reaches a value of 0 and frees the resources, if the static OPPs get
added for the same OPP table structure (as the OPP table structure was
never freed). That is messy and very unclear.

This patch makes parsed_static_opps an unsigned integer and uses it to
count the number of users of the static OPPs. The increment and
decrement to parsed_static_opps is done under opp_table->lock now to
make sure no races are possible if the OPP table is getting added and
removed in parallel (which doesn't happen in practice, but can in
theory).

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

PM / OPP: Support adjusting OPP voltages at runtime

On some SoCs the Adaptive Voltage Scaling (AVS) technique is
employed to optimize the operating voltage of a device. At a
given frequency, the hardware monitors dynamic factors and either
makes a suggestion for how much to adjust a voltage for the
current frequency, or it automatically adjusts the voltage
without software intervention. Add an API to the OPP library for
the former case, so that AVS type devices can update the voltages
for an OPP when the hardware determines the voltage should
change. The assumption is that drivers like CPUfreq or devfreq
will register for the OPP notifiers and adjust the voltage
according to suggestions that AVS makes.

This patch is derived from [1] submitted by Stephen.
[1] https://lore.kernel.org/patchwork/patch/599279/

Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
[Roger Lu: Changed to rcu less implementation]
Signed-off-by: Roger Lu <roger.lu@mediatek.com>
[s.nawrocki@samsung.com: added handling of OPP min/max voltage]
Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

PM / OPP: Support adjusting OPP voltages at runtime

On some SoCs the Adaptive Voltage Scaling (AVS) technique is
employed to optimize the operating voltage of a device. At a
given frequency, the hardware monitors dynamic factors and either
makes a suggestion for how much to adjust a voltage for the
current frequency, or it automatically adjusts the voltage
without software intervention. Add an API to the OPP library for
the former case, so that AVS type devices can update the voltages
for an OPP when the hardware determines the voltage should
change. The assumption is that drivers like CPUfreq or devfreq
will register for the OPP notifiers and adjust the voltage
according to suggestions that AVS makes.

This patch is derived from [1] submitted by Stephen.
[1] https://lore.kernel.org/patchwork/patch/599279/

Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: Roger Lu <roger.lu@mediatek.com>
[s.nawrocki@samsung.com: added handling of OPP min/max voltage]
Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: core: Revert "add regulators enable and disable"

All the drivers, which use the OPP framework control regulators, which
are already enabled. Typically those regulators are also system critical,
due to providing power to CPU core or system buses. It turned out that
there are cases, where calling regulator_enable() on such boot-enabled
regulator has side-effects and might change its initial voltage due to
performing initial voltage balancing without all restrictions from the
consumers. Until this issue becomes finally solved in regulator core,
avoid calling regulator_enable()/disable() from the OPP framework.

This reverts commit 7f93ff73f7c8c8bfa6be33bcc16470b0b44682aa.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: Mark Brown <broonie@kernel.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: core: add regulators enable and disable

Add enable regulators to dev_pm_opp_set_regulators() and disable
regulators to dev_pm_opp_put_regulators(). Even if bootloader
leaves regulators enabled, they should be enabled in kernel in
order to increase the reference count.

Signed-off-by: Kamil Konieczny <k.konieczny@partner.samsung.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Don't decrement uninitialized list_kref

The list_kref was added for static OPPs and to track their users. The
kref is initialized while the static OPPs are added, but removed
unconditionally even if the static OPPs were never added. This causes
refcount mismatch warnings currently.

Fix that by always initializing the kref when the OPP table is first
initialized. The refcount is later incremented only for the second user
onwards.

Fixes: d0e8ae6c26da ("OPP: Create separate kref for static OPPs list")
Reported-by: Rajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Add dev_pm_opp_find_level_exact()

Since the performance states in the OPP table are unique, implement a
dev_pm_opp_find_level_exact() in order to be able to fetch a specific OPP.

Signed-off-by: Niklas Cassel <niklas.cassel@linaro.org>
[ Viresh: Updated commit log ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Return genpd virtual devices from dev_pm_opp_attach_genpd()

The cpufreq drivers don't need to do runtime PM operations on the
virtual devices returned by dev_pm_domain_attach_by_name() and so the
virtual devices weren't shared with the callers of
dev_pm_opp_attach_genpd() earlier.

But the IO device drivers would want to do that. This patch updates the
prototype of dev_pm_opp_attach_genpd() to accept another argument to
return the pointer to the array of genpd virtual devices.

Reported-by: Rajendra Nayak <rnayak@codeaurora.org>
Tested-by: Rajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Not all power-domains are scalable

A device may have multiple power-domains and not all of them may be
scalable (i.e. support performance states). But
dev_pm_opp_attach_genpd() doesn't take that into account currently.

Fix that by not verifying the names argument with "power-domain-names"
DT property and finding the index into the required-opps array. The
names argument will anyway get verified later on when we call
dev_pm_domain_attach_by_name().

Fixes: 6319aee10e53 ("opp: Attach genpds to devices from within OPP core")
Reported-by: Rajendra Nayak <rnayak@codeaurora.org>
Tested-by: Rajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Don't use IS_ERR on invalid supplies

_set_opp_custom() receives a set of OPP supplies as its arguments and
the caller of it passes NULL when the supplies are not valid. But
_set_opp_custom(), by mistake, checks for error by performing
IS_ERR(old_supply) on it which will always evaluate to false.

The problem was spotted during of testing of upcoming update for the
NVIDIA Tegra CPUFreq driver.

Cc: stable <stable@vger.kernel.org>
Fixes: 7e535993fa4f ("OPP: Separate out custom OPP handler specific code")
Reported-by: Marc Dietrich <marvin24@gmx.de>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
[ Viresh: Massaged changelog ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500

Based on 2 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation #

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-only

has been chosen to replace the boilerplate/reference in 4122 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Enrico Weigelt <info@metux.net>
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Allison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

opp: Make dev_pm_opp_set_rate() handle freq = 0 to drop performance votes

For devices with performance state, we use dev_pm_opp_set_rate() to set
the appropriate clk rate and the performance state.

We do need a way to remove the performance state vote when we idle the
device and turn the clocks off. Use dev_pm_opp_set_rate() with freq = 0
to achieve this.

Signed-off-by: Rajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: Stephen Boyd <swboyd@chromium.org>
[ Viresh: Updated _set_required_opps() to handle the !opp case ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Don't overwrite rounded clk rate

The OPP table normally contains 'fmax' values corresponding to the
voltage or performance levels of each OPP, but we don't necessarily want
all the devices to run at fmax all the time. Running at fmax makes sense
for devices like CPU/GPU, which have a finite amount of work to do and
since a specific amount of energy is consumed at an OPP, its better to
run at the highest possible frequency for that voltage value.

On the other hand, we have IO devices which need to run at specific
frequencies only for their proper functioning, instead of maximum
possible frequency.

The OPP core currently roundup to the next possible OPP for a frequency
and select the fmax value. To support the IO devices by the OPP core,
lets do the roundup to fetch the voltage or performance state values,
but not use the OPP frequency value. Rather use the value returned by
clk_round_rate().

The current user, cpufreq, of dev_pm_opp_set_rate() already does the
rounding to the next OPP before calling this routine and it won't
have any side affects because of this change.

Signed-off-by: Stephen Boyd <swboyd@chromium.org>
Signed-off-by: Rajendra Nayak <rnayak@codeaurora.org>
[ Viresh: Massaged changelog, added comment and use temp_opp variable
instead ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Allocate genpd_virt_devs from dev_pm_opp_attach_genpd()

Currently the space for the array of virtual devices is allocated along
with the OPP table, but that isn't going to work well from now onwards.
For single power domain case, a driver can either use the original
device structure for setting the performance state (if genpd attached
with dev_pm_domain_attach()) or use the virtual device structure (if
genpd attached with dev_pm_domain_attach_by_name(), which returns the
virtual device) and so we can't know in advance if we are going to need
genpd_virt_devs array or not.

Lets delay the allocation a bit and do it along with
dev_pm_opp_attach_genpd() rather. The deallocation is done from
dev_pm_opp_detach_genpd().

Tested-by: Niklas Cassel <niklas.cassel@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

opp: Attach genpds to devices from within OPP core

The OPP core requires the virtual device pointers to set performance
state on behalf of the device, for the multiple power domain case. The
genpd API (dev_pm_domain_attach_by_name()) has evolved now to support
even the single power domain case and that lets us add common code for
handling both the cases more efficiently.

The virtual device structure returned by dev_pm_domain_attach_by_name()
isn't normally used by the cpufreq drivers as they don't manage power
on/off of the domains and so is only useful for the OPP core.

This patch moves all the complexity into the OPP core to make the end
drivers simple. The earlier APIs dev_pm_opp_{set|put}_genpd_virt_dev()
are reworked into dev_pm_opp_{attach|detach}_genpd(). The new helper
dev_pm_opp_attach_genpd() accepts a NULL terminated array of strings
which contains names of all the genpd's to attach. It then attaches all
the domains and saves the pointers to the virtual devices. The other
helper undo the work done by this helper.

Tested-by: Niklas Cassel <niklas.cassel@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Introduce dev_pm_opp_find_freq_ceil_by_volt()

This patch introduces a new helper routine in the OPP core, which
returns the OPP with the highest frequency which has voltage less than
or equal to the target voltage passed to the helper.

Signed-off-by: Andrew-sh.Cheng <andrew-sh.cheng@mediatek.com>
[ Viresh: Massaged the commit log and renamed the helper with some
cleanups. ]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

PM / OPP: Update performance state when freq == old_freq

At boot up, CPUFreq core performs a sanity check to see if the system is
running at a frequency defined in the frequency table of the CPU. If so,
we try to find a valid frequency (lowest frequency greater than the
currently programmed frequency) from the table and set it. When the call
reaches dev_pm_opp_set_rate(), it calls _find_freq_ceil(opp_table,
&old_freq) to find the previously configured OPP and this call also
updates the old_freq. This eventually sets the old_freq == freq (new
target requested by cpufreq core) and we skip updating the performance
state in this case.

Fix this by also updating the performance state when the old_freq ==
freq.

Fixes: ca1b5d77b1c6 ("OPP: Configure all required OPPs")
Cc: v5.0 <stable@vger.kernel.org> # v5.0
Reported-by: Niklas Cassel <niklas.cassel@linaro.org>
Tested-by: Jorge Ramirez-Ortiz <jorge.ramirez-ortiz@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

opp: no need to check return value of debugfs_create functions

When calling debugfs functions, there is no need to ever check the
return value. The function can work or not, but the code logic should
never do something different based on this.

Cc: Viresh Kumar <vireshk@kernel.org>
Cc: Nishanth Menon <nm@ti.com>
Cc: Stephen Boyd <sboyd@kernel.org>
Cc: linux-pm@vger.kernel.org
Reviewed-by: Stephen Boyd <sboyd@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

PM / OPP: Remove unused parameter of _generic_set_opp_clk_only()

The previous frequency value isn't getting used in the routine
_generic_set_opp_clk_only(), drop it.

Reviewed-by: Stephen Boyd <sboyd@kernel.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Add support for parsing the 'opp-level' property

Now that the OPP bindings are updated to include an optional
'opp-level' property, add support to parse it from device tree
and store it as part of dev_pm_opp structure.
Also add and export an helper 'dev_pm_opp_get_level()' that can be
used to get the level value read from device tree when present.

Reviewed-by: Stephen Boyd <swboyd@chromium.org>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Signed-off-by: Andy Gross <andy.gross@linaro.org>

cpufreq: scpi/scmi: Fix freeing of dynamic OPPs

Since the commit 2a4eb7358aba "OPP: Don't remove dynamic OPPs from
_dev_pm_opp_remove_table()", dynamically created OPP aren't
automatically removed anymore by dev_pm_opp_cpumask_remove_table(). This
affects the scpi and scmi cpufreq drivers which no longer free OPPs on
failures or on invocations of the policy->exit() callback.

Create a generic OPP helper dev_pm_opp_remove_all_dynamic() which can be
called from these drivers instead of dev_pm_opp_cpumask_remove_table().

In dev_pm_opp_remove_all_dynamic(), we need to make sure that the
opp_list isn't getting accessed simultaneously from other parts of the
OPP core while the helper is freeing dynamic OPPs, i.e. we can't drop
the opp_table->lock while traversing through the OPP list. And to
accomplish that, this patch also creates _opp_kref_release_unlocked()
which can be called from this new helper with the opp_table lock already
held.

Cc: 4.20 <stable@vger.kernel.org> # v4.20
Reported-by: Valentin Schneider <valentin.schneider@arm.com>
Fixes: 2a4eb7358aba "OPP: Don't remove dynamic OPPs from _dev_pm_opp_remove_table()"
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Valentin Schneider <valentin.schneider@arm.com>
Reviewed-by: Sudeep Holla <sudeep.holla@arm.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

OPP: Add dev_pm_opp_xlate_performance_state() helper

dev_pm_genpd_set_performance_state() needs to handle performance state
propagation going forward. Currently this routine only gets the required
performance state of the device's genpd as an argument, but it doesn't
know how to translate that to master genpd(s) of the device's genpd.

Introduce a new helper dev_pm_opp_xlate_performance_state() which will
be used to translate from performance state of a device (or genpd
sub-domain) to another device (or master genpd).

Normally the src_table (of genpd sub-domain) will have the
"required_opps" property set to point to one of the OPPs in the
dst_table (of master genpd), but in some cases the genpd and its master
have one to one mapping of performance states and so none of them have
the "required-opps" property set. Return the performance state of the
src_table as it is in such cases.

Tested-by: Rajendra Nayak <rnayak@codeaurora.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Fix missing debugfs supply directory for OPPs

There is one case where we may end up with no "supply" directory for the
OPPs in debugfs. That happens when the OPP core isn't managing the
regulators for the device and the device's OPP do have microvolt
property. It happens because the opp_table->regulator_count remains set
to 0 and the debugfs routines don't add any supply directory in such a
case.

This commit fixes that by setting opp_table->regulator_count to 1 in
that particular case. But to make everything work nicely and not break
other parts of the core, regulator_count is defined as "int" now instead
of "unsigned int" and it can have different special values now. It is
set to -1 initially to mark it "uninitialized" and later only we set it
to 0 or positive values after checking how many supplies are there.

This also helps in finding the bugs where only few of the OPPs have the
"opp-microvolt" property set and not all.

Fixes: 1fae788ed640 ("PM / OPP: Don't create debugfs "supply-0" directory unnecessarily")
Reported-by: Quentin Perret <quentin.perret@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Use opp_table->regulators to verify no regulator case

The value of opp_table->regulator_count is not very consistent right now
and it may end up being 0 while we do have a "opp-microvolt" property in
the OPP table. It was kept that way as we used to check if any
regulators are set with the OPP core for a device or not using value of
regulator_count.

Lets use opp_table->regulators for that purpose as the meaning of
regulator_count is going to change in the later patches.

Reported-by: Quentin Perret <quentin.perret@arm.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Configure all required OPPs

Now that all the infrastructure is in place to support multiple required
OPPs, lets switch over to using it.

A new internal routine _set_required_opps() takes care of updating
performance state for all the required OPPs. With this the performance
state updates are supported even when the end device needs to configure
regulators as well, that wasn't the case earlier.

The pstates were earlier stored in the end device's OPP structures, that
also changes now as those values are stored in the genpd's OPP
structures. And so we switch over to using
pm_genpd_opp_to_performance_state() instead of
of_genpd_opp_to_performance_state() to get performance state for the
genpd OPPs.

The routine _generic_set_opp_domain() is not required anymore and is
removed.

On errors we don't try to recover by reverting to old settings as things
are really complex now and the calls here should never really fail
unless there is a bug. There is no point increasing the complexity, for
code which will never be executed.

Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Add dev_pm_opp_{set|put}_genpd_virt_dev() helper

Multiple generic power domains for a consumer device are supported with
the help of virtual devices, which are created for each consumer device
- genpd pair. These are the device structures which are attached to the
power domain and are required by the OPP core to set the performance
state of the genpd.

The helpers added by this commit are required to be called once for each
of these virtual devices. These are required only if multiple domains
are available for a device, otherwise the actual device structure will
be used instead by the OPP core.

The new helpers also support the complex cases where the consumer device
wouldn't always require all the domains. For example, a camera may
require only one power domain during normal operations but two during
high resolution operations. The consumer driver can call
dev_pm_opp_put_genpd_virt_dev(high_resolution_genpd_virt_dev) if it is
currently operating in the normal mode and doesn't have any performance
requirements from the genpd which manages high resolution power
requirements. The consumer driver can later call
dev_pm_opp_set_genpd_virt_dev(high_resolution_genpd_virt_dev) once it
switches back to the high resolution mode.

The new helpers differ from other OPP set/put helpers as the new ones
can be called with OPPs initialized for the table as we may need to call
them on the fly because of the complex case explained above. For this
reason it is possible that the genpd virt_dev structure may be used in
parallel while the new helpers are running and a new mutex is added to
protect against that. We didn't use the existing opp_table->lock mutex
as that is widely used in the OPP core and we will need this lock in the
dev_pm_opp_set_rate() helper while changing OPP and we need to make sure
there is not much contention while doing that as that's the hotpath.

Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Populate OPPs from "required-opps" property

An earlier commit populated the OPP tables from the "required-opps"
property, this commit populates the individual OPPs. This is repeated
for each OPP in the OPP table and these populated OPPs will be used by
later commits.

Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Populate required opp tables from "required-opps" property

The current implementation works only for the case where a single
phandle is present in the "required-opps" property, while DT allows
multiple phandles to be present there.

This patch adds new infrastructure to parse all the phandles present in
"required-opps" property and save pointers of the required OPP's OPP
tables. These will be used by later commits.

Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Separate out custom OPP handler specific code

Create a separate routine to take care of custom set_opp() handler
specific stuff.

Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Return error on error from dev_pm_opp_get_opp_count()

Return error number instead of 0 on failures.

Fixes: a1e8c13600bf ("PM / OPP: "opp-hz" is optional for power domains")
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Prevent creating multiple OPP tables for devices sharing OPP nodes

When two or more devices are sharing their clock and voltage rails, they
share the same OPP table. But there are some corner cases where the OPP
core incorrectly creates separate OPP tables for them.

For example, CPU 0 and 1 share clock/voltage rails. The platform
specific code calls dev_pm_opp_set_regulators() for CPU0 and the OPP
core creates an OPP table for it (the individual OPPs aren't initialized
as of now). The same is repeated for CPU1 then. Because
_opp_get_opp_table() doesn't compare DT node pointers currently, it
fails to find the link between CPU0 and CPU1 and so creates a new OPP
table.

Fix this by calling _managed_opp() from _opp_get_opp_table().
_managed_opp() gain an additional argument (index) to get the right node
pointer. This resulted in simplifying code in _of_add_opp_table_v2() as
well.

Tested-by: Niklas Cassel <niklas.cassel@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Use a single mechanism to free the OPP table

Currently there are two separate ways to free the OPP table based on how
it is created in the first place.

We call _dev_pm_opp_remove_table() to free the static and/or dynamic
OPP, OPP list devices, etc. This is done for the case where the OPP
table is added while initializing the OPPs, like via the path
dev_pm_opp_of_add_table().

We also call dev_pm_opp_put_opp_table() in some cases which eventually
frees the OPP table structure once the reference count reaches 0. This
is used by the first case as well as other cases like
dev_pm_opp_set_regulators() where the OPPs aren't necessarily
initialized at this point.

This whole thing is a bit unclear and messy and obstruct any further
cleanup/fixup of OPP core.

This patch tries to streamline this by keeping a single path for OPP
table destruction, i.e. dev_pm_opp_put_opp_table().

All the cleanup happens in _opp_table_kref_release() now after the
reference count reaches 0. _dev_pm_opp_remove_table() is removed as it
isn't required anymore.

We don't drop the reference to the OPP table after creating it from
_of_add_opp_table_v{1|2}() anymore and the same is dropped only when we
try to remove them.

Tested-by: Niklas Cassel <niklas.cassel@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Don't remove dynamic OPPs from _dev_pm_opp_remove_table()

Only one platform was depending on this feature and it is already
updated now. Stop removing dynamic OPPs from _dev_pm_opp_remove_table().
This simplifies lot of paths and removes unnecessary parameters.

Tested-by: Niklas Cassel <niklas.cassel@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Create separate kref for static OPPs list

The static OPPs don't always get freed with the OPP table, it can happen
before that as well. For example, if the OPP table is first created
using helpers like dev_pm_opp_set_supported_hw() and the OPPs are
created at a later point. Now when the OPPs are removed, the OPP table
stays until the time dev_pm_opp_put_supported_hw() is called.

Later patches will streamline the freeing of OPP table and that requires
the static OPPs to get freed with help of a separate kernel reference.
This patch prepares for that by creating a separate kref for static OPPs
list.

Tested-by: Niklas Cassel <niklas.cassel@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Don't take OPP table's kref for static OPPs

The reference count is only required to be incremented for every call
that may lead to adding the OPP table. For static OPPs the same should
be done from the parent routine which adds all static OPPs together and
so only one refcount for all static OPPs.

Update code to reflect that.

The refcount is incremented every time a dynamic OPP is created (as that
can lead to creating the OPP table) and the same is dropped when the OPP
is removed.

Tested-by: Niklas Cassel <niklas.cassel@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Pass index to _of_init_opp_table()

This is a preparatory patch required for the next commit which will
start using OPP table's node pointer in _of_init_opp_table(), which
requires the index in order to read the OPP table's phandle.

This commit adds the index argument in the call chains in order to get
it delivered to _of_init_opp_table().

Tested-by: Niklas Cassel <niklas.cassel@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

OPP: Protect dev_list with opp_table lock

The dev_list needs to be protected with a lock, else we may have
simultaneous access (addition/removal) to it and that would be racy.
Extend scope of the opp_table lock to protect dev_list as well.

Tested-by: Niklas Cassel <niklas.cassel@linaro.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

PM / OPP: Update voltage in case freq == old_freq

This commit fixes a rare but possible case when the clk rate is updated
without update of the regulator voltage.

At boot up, CPUfreq checks if the system is running at the right freq. This
is a sanity check in case a bootloader set clk rate that is outside of freq
table present with cpufreq core. In such cases system can be unstable so
better to change it to a freq that is preset in freq-table.

The CPUfreq takes next freq that is >= policy->cur and this is our
target_freq that needs to be set now.

dev_pm_opp_set_rate(dev, target_freq) checks the target_freq and the
old_freq (a current rate). If these are equal it returns early. If not,
it searches for OPP (old_opp) that fits best to old_freq (not listed in
the table) and updates old_freq (!).

Here, we can end up with old_freq = old_opp.rate = target_freq, which
is not handled in _generic_set_opp_regulator(). It's supposed to update
voltage only when freq > old_freq || freq > old_freq.

if (freq > old_freq) {
ret = _set_opp_voltage(dev, reg, new_supply);
[...]
if (freq < old_freq) {
ret = _set_opp_voltage(dev, reg, new_supply);
if (ret)

It results in, no voltage update while clk rate is updated.

Example:
freq-table = {
1000MHz 1.15V
666MHZ 1.10V
333MHz 1.05V
}
boot-up-freq = 800MHz # not listed in freq-table
freq = target_freq = 1GHz
old_freq = 800Mhz
old_opp = _find_freq_ceil(opp_table, &old_freq); #(old_freq is modified!)
old_freq = 1GHz

Fixes: 6a0712f6f199 ("PM / OPP: Add dev_pm_opp_set_rate()")
Cc: 4.6+ <stable@vger.kernel.org> # v4.6+
Signed-off-by: Waldemar Rymarkiewicz <waldemar.rymarkiewicz@gmail.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

PM / OPP: Fix shared OPP table support in dev_pm_opp_register_set_opp_helper()

It should be fine to call dev_pm_opp_register_set_opp_helper() for all
possible CPUs, even if some of them share the OPP table as the caller
may not be aware of sharing policy.

Lets increment the reference count of the OPP table and return its
pointer. The caller need to call dev_pm_opp_register_put_opp_helper()
the same number of times later on to drop all the references.

To avoid adding another counter to count how many times
dev_pm_opp_register_set_opp_helper() is called for the same OPP table,
dev_pm_opp_register_put_opp_helper() frees the resources on the very
first call made to it, assuming that the caller would be calling it
sequentially for all the CPUs. We can revisit that if that assumption is
broken in the future.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

PM / OPP: Fix shared OPP table support in dev_pm_opp_set_regulators()

It should be fine to call dev_pm_opp_set_regulators() for all possible
CPUs, even if some of them share the OPP table as the caller may not be
aware of sharing policy.

Lets increment the reference count of the OPP table and return its
pointer. The caller need to call dev_pm_opp_put_regulators() the same
number of times later on to drop all the references.

To avoid adding another counter to count how many times
dev_pm_opp_set_regulators() is called for the same OPP table,
dev_pm_opp_put_regulators() frees the resources on the very first call
made to it, assuming that the caller would be calling it sequentially
for all the CPUs. We can revisit that if that assumption is broken in
the future.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

PM / OPP: Fix shared OPP table support in dev_pm_opp_set_prop_name()

It should be fine to call dev_pm_opp_set_prop_name() for all possible
CPUs, even if some of them share the OPP table as the caller may not be
aware of sharing policy.

Lets increment the reference count of the OPP table and return its
pointer. The caller need to call dev_pm_opp_put_prop_name() the same
number of times later on to drop all the references.

To avoid adding another counter to count how many times
dev_pm_opp_set_prop_name() is called for the same OPP table,
dev_pm_opp_put_prop_name() frees the resources on the very first call
made to it, assuming that the caller would be calling it sequentially
for all the CPUs. We can revisit that if that assumption is broken in
the future.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

PM / OPP: Fix shared OPP table support in dev_pm_opp_set_supported_hw()

It should be fine to call dev_pm_opp_set_supported_hw() for all possible
CPUs, even if some of them share the OPP table as the caller may not be
aware of sharing policy.

Lets increment the reference count of the OPP table and return its
pointer. The caller need to call dev_pm_opp_put_supported_hw() the same
number of times later on to drop all the references.

To avoid adding another counter to count how many times
dev_pm_opp_set_supported_hw() is called for the same OPP table,
dev_pm_opp_put_supported_hw() frees the resources on the very first call
made to it, assuming that the caller would be calling it sequentially
for all the CPUs. We can revisit that if that assumption is broken in
the future.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

PM / OPP: Remove dev_pm_opp_{un}register_get_pstate_helper()

These helpers aren't used anymore, remove them.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

PM / OPP: Get performance state using genpd helper

The genpd core provides an API now to retrieve the performance state
from DT, use that instead of the ->get_pstate() callback.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

PM / OPP: Implement of_dev_pm_opp_find_required_opp()

A device's DT node or its OPP nodes can contain a phandle to other
device's OPP node, in the "required-opps" property.

This patch implements a routine to find that required OPP from the node
that contains the "required-opps" property.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

PM / OPP: "opp-hz" is optional for power domains

"opp-hz" property is optional for power domains now and we shouldn't
error out if it is missing for power domains.

This patch creates two new routines, _get_opp_count() and
_opp_is_duplicate(), by separating existing code from their parent
functions. Also skip duplicate OPP check for power domain OPPs as they
may not have any the "opp-hz" field, but a platform specific performance
state binding to uniquely identify OPP nodes.

By default the debugfs OPP nodes are named using the "rate" value, but
that isn't possible for the power domain OPP nodes and hence they use
the index of the OPP node in the OPP node list instead.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org>

PM / OPP: Add dev_pm_opp_{un}register_get_pstate_helper()

This adds the dev_pm_opp_{un}register_get_pstate_helper() helper
routines which will be used to set the get_pstate() callback for a
device. This callback will be later called internally by the OPP core to
get performance state corresponding to an OPP.

This is required temporarily until the time we have proper DT bindings
to include the performance state information.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / OPP: Support updating performance state of device's power domain

The genpd framework now provides an API to request device's power
domain to update its performance state. Use that interface from the
OPP core for devices whose power domains support performance states.

Note that this commit doesn't add any mechanism by which performance
states are made available to the OPP core. That would be done by a
later commit.

Note that the current implementation is restricted to the case where
the device doesn't have separate regulators for itself. We shouldn't
over engineer the code before we have real use case for them. We can
always come back and add more code to support such cases later on.

Tested-by: Rajendra Nayak <rnayak@codeaurora.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / OPP: Rename dev_pm_opp_register_put_opp_helper()

The routine is named incorrectly since the first attempt as there is
nothing like a put_opp() helper. We wanted to unregister the set_opp()
helper here and so it should rather be named as
dev_pm_opp_unregister_set_opp_helper().

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / OPP: Move error message to debug level

On some i.MX6 platforms which do not have speed grading
check, opp table will not be created in platform code,
so cpufreq driver prints the following error message:

cpu cpu0: dev_pm_opp_get_opp_count: OPP table not found (-19)

However, this is not really an error in this case because the
imx6q-cpufreq driver first calls dev_pm_opp_get_opp_count()
and if it fails, it means that platform code does not provide
OPP and then dev_pm_opp_of_add_table() will be called.

In order to avoid such confusing error message, move it to
debug level.

It is up to the caller of dev_pm_opp_get_opp_count() to check its
return value and decide if it will print an error or not.

Signed-off-by: Fabio Estevam <fabio.estevam@nxp.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

PM / OPP: Move the OPP directory out of power/

The drivers/base/power/ directory is special and contains code related
to power management core like system suspend/resume, hibernation, etc.
It was fine to keep the OPP code inside it when we had just one file for
it, but it is growing now and already has a directory for itself.

Lets move it directly under drivers/ directory, just like cpufreq and
cpuidle.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>