cpuidle, OMAP4: Push RCU-idle into omap4_enter_lowpower()

OMAP4 uses full SoC suspend modes as idle states, as such it needs the
whole power-domain and clock-domain code from the idle path.

All that code is not suitable to run with RCU disabled, as such push
RCU-idle deeper still.

Signed-off-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Tony Lindgren <tony@atomide.com>
Tested-by: Ulf Hansson <ulf.hansson@linaro.org>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://lkml.kernel.org/r/Yqcv6crSNKuSWoTu@atomide.com
Link: https://lore.kernel.org/r/20230112195541.660272394@infradead.org

cpuidle: Fix ct_idle_*() usage

The whole disable-RCU, enable-IRQS dance is very intricate since
changing IRQ state is traced, which depends on RCU.

Add two helpers for the cpuidle case that mirror the entry code:

ct_cpuidle_enter()
ct_cpuidle_exit()

And fix all the cases where the enter/exit dance was buggy.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Tony Lindgren <tony@atomide.com>
Tested-by: Ulf Hansson <ulf.hansson@linaro.org>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://lore.kernel.org/r/20230112195540.130014793@infradead.org

cpuidle, OMAP4: Push RCU-idle into driver

Doing RCU-idle outside the driver, only to then temporarily enable it
again, some *four* times, before going idle is suboptimal.

Notably three times explicitly using RCU_NONIDLE() and once implicitly
through cpu_pm_*().

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Tony Lindgren <tony@atomide.com>
Tested-by: Ulf Hansson <ulf.hansson@linaro.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Reviewed-by: Tony Lindgren <tony@atomide.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Link: https://lore.kernel.org/r/20230112195540.007918454@infradead.org

ARM: OMAP2+: Fix suspcious RCU usage splats for omap_enter_idle_coupled

We get suspcious RCU usage splats with cpuidle in several places in
omap_enter_idle_coupled() with the kernel debug options enabled:

RCU used illegally from extended quiescent state!
...
(_raw_spin_lock_irqsave)
(omap_enter_idle_coupled+0x17c/0x2d8)
(omap_enter_idle_coupled)
(cpuidle_enter_state)
(cpuidle_enter_state_coupled)
(cpuidle_enter)

Let's use RCU_NONIDLE to suppress these splats. Things got changed around
with commit 1098582a0f6c ("sched,idle,rcu: Push rcu_idle deeper into the
idle path") that started triggering these warnings.

For the tick_broadcast related calls, ideally we'd just switch over to
using CPUIDLE_FLAG_TIMER_STOP for omap_enter_idle_coupled() to have the
generic cpuidle code handle the tick_broadcast related calls for us and
then just drop the tick_broadcast calls here.

But we're currently missing the call in the common cpuidle code for
tick_broadcast_enable() that CPU1 hotplug needs as described in earlier
commit 50d6b3cf9403 ("ARM: OMAP2+: fix lack of timer interrupts on CPU1
after hotplug").

Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Russell King <rmk+kernel@armlinux.org.uk>
Acked-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Tony Lindgren <tony@atomide.com>

ARM: OMAP2+: Manage MPU state properly for omap_enter_idle_coupled()

Based on more testing, commit 8ca5ee624b4c ("ARM: OMAP2+: Restore MPU
power domain if cpu_cluster_pm_enter() fails") is a poor fix for handling
cpu_cluster_pm_enter() returned errors.

We should not override the cpuidle states with a hardcoded PWRDM_POWER_ON
value. Instead, we should use a configured idle state that does not cause
the context to be lost. Otherwise we end up configuring a potentially
improper state for the MPUSS. We also want to update the returned state
index for the selected state.

Let's just select the highest power idle state C1 to ensure no context
loss is allowed on cpu_cluster_pm_enter() errors. With these changes we
can now unconditionally call omap4_enter_lowpower() for WFI like we did
earlier before commit 55be2f50336f ("ARM: OMAP2+: Handle errors for
cpu_pm"). And we can return the selected state index.

Fixes: 8f04aea048d5 ("ARM: OMAP2+: Restore MPU power domain if cpu_cluster_pm_enter() fails")
Fixes: 55be2f50336f ("ARM: OMAP2+: Handle errors for cpu_pm")
Signed-off-by: Tony Lindgren <tony@atomide.com>

ARM: OMAP2+: Restore MPU power domain if cpu_cluster_pm_enter() fails

If cpu_cluster_pm_enter() fails, we need to set MPU power domain back
to enabled to prevent the next WFI from potentially triggering an
undesired MPU power domain state change.

We already do this for omap_enter_idle_smp() but are missing it for
omap_enter_idle_coupled().

Fixes: 55be2f50336f ("ARM: OMAP2+: Handle errors for cpu_pm")
Signed-off-by: Tony Lindgren <tony@atomide.com>

ARM: OMAP2+: Handle errors for cpu_pm

We need to check for errors when calling cpu_pm_enter() and
cpu_cluster_pm_enter(). And we need to bail out on errors as
otherwise we can enter a deeper idle state when not desired.

I'm not aware of the lack of error handling causing issues yet,
but we need this at least for blocking deeper idle states when
a GPIO instance has pending interrupts.

Cc: Dave Gerlach <d-gerlach@ti.com>
Cc: Grygorii Strashko <grygorii.strashko@ti.com>
Cc: Keerthy <j-keerthy@ti.com>
Cc: Ladislav Michl <ladis@linux-mips.org>
Cc: Russell King <rmk+kernel@armlinux.org.uk>
Cc: Tero Kristo <t-kristo@ti.com>
Signed-off-by: Tony Lindgren <tony@atomide.com>
Link: https://lore.kernel.org/r/20200304225433.37336-2-tony@atomide.com
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500

Based on 2 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation #

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-only

has been chosen to replace the boilerplate/reference in 4122 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Enrico Weigelt <info@metux.net>
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Allison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

ARM: OMAP2+: fix lack of timer interrupts on CPU1 after hotplug

If we have a kernel configured for periodic timer interrupts, and we
have cpuidle enabled, then we end up with CPU1 losing timer interupts
after a hotplug.

This can manifest itself in RCU stall warnings, or userspace becoming
unresponsive.

The problem is that the kernel initially wants to use the TWD timer
for interrupts, but the TWD loses context when we enter the C3 cpuidle
state. Nothing reprograms the TWD after idle.

We have solved this in the past by switching to broadcast timer ticks,
and cpuidle44xx switches to that mode at boot time. However, there is
nothing to switch from periodic mode local timers after a hotplug
operation.

We call tick_broadcast_enter() in omap_enter_idle_coupled(), which one
would expect would take care of the issue, but internally this only
deals with one-shot local timers - tick_broadcast_enable() on the other
hand only deals with periodic local timers. So, we need to call both.

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
[tony@atomide.com: just standardized the subject line]
Signed-off-by: Tony Lindgren <tony@atomide.com>

ARM: OMAP5: Add basic cpuidle MPU CSWR support

Add OMAP5 CPUIDLE support.

This patch adds MPUSS low power states in cpuidle.

C1 - CPU0 WFI + CPU1 WFI + MPU ON
C2 - CPU0 RET + CPU1 RET + MPU CSWR

Modified from TI kernel tree commit 605967fd2205 ("ARM: DRA7: PM:
cpuidle MPU CSWR support") except enable cpuidle for omap5 instead
of dra7.

According to Nishanth Menon <nm@ti.com>, cpuidle on dra7 is not
supported properly in the hardware so we don't want to enable it.
However, for omap5 this adds some nice power savings. Note that
the TI 3.8 based tree has other cpuidle states that we may be able
to enable later on.

On omap5-uevm, the power consumption eventually settles down to about
920mW with ehci-omap and ohci-omap3 unloaded compared to about 1.7W
without these patches. Note that it seems to take few minutes after
booting for the idle power to go down to 920mW from 1.3W, no idea so
far what might be causing that.

Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
[ j-keerthy@ti.com rework on 3.14]
Signed-off-by: Keerthy <j-keerthy@ti.com>
[nm@ti.com: updates based on profiling]
[tony@atomide.com: dropped CPUIDLE_FLAG_TIME_VALID no longer used,
changed for omap5 only as requested by Nishanth, updated comments]
Signed-off-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Tony Lindgren <tony@atomide.com>

ARM: OMAP2+: clockdomain: add usecounting support to autoidle APIs

The previous implementation was racy in many locations, where the current
status of the clockdomain was read out, some operations were executed,
and the previous status info was used afterwards to decide next state
for the clockdomain. Instead, fix the implementation of the allow_idle /
deny_idle APIs to properly have usecounting support. This allows clean
handling internally within the clockdomain core, and simplifies the
usage also within hwmod.

Signed-off-by: Tero Kristo <t-kristo@ti.com>
Signed-off-by: Tony Lindgren <tony@atomide.com>

ARM: OMAP: Use explicit broadcast oneshot control function

Replace the clockevents_notify() call with an explicit function call.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Tony Lindgren <tony@atomide.com>
Link: http://lkml.kernel.org/r/3123047.uVjevtxDV7@vostro.rjw.lan
Signed-off-by: Ingo Molnar <mingo@kernel.org>

ARM: OMAP: Use explicit broadcast control function

Replace the clockevents_notify() call with an explicit function call.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Tony Lindgren <tony@atomide.com>
Link: http://lkml.kernel.org/r/2124877.3nbWGILHCV@vostro.rjw.lan
Signed-off-by: Ingo Molnar <mingo@kernel.org>

ARM: cpuidle: Remove duplicate header inclusion

The cpu_do_idle() function is always used by the cpuidle drivers.

That led to have each driver including cpuidle.h and proc-fns.h, they are
always paired. That makes a lot of duplicate headers inclusion. Instead of
including both in each .c file, move the proc-fns.h header inclusion in the
cpuidle.h header file directly, so we can save some line of code.

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Acked-by: Kevin Hilman <khilman@linaro.org>
Acked-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

cpuidle: Invert CPUIDLE_FLAG_TIME_VALID logic

The only place where the time is invalid is when the ACPI_CSTATE_FFH entry
method is not set. Otherwise for all the drivers, the time can be correctly
measured.

Instead of duplicating the CPUIDLE_FLAG_TIME_VALID flag in all the drivers
for all the states, just invert the logic by replacing it by the flag
CPUIDLE_FLAG_TIME_INVALID, hence we can set this flag only for the acpi idle
driver, remove the former flag from all the drivers and invert the logic with
this flag in the different governor.

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

ARM: OMAP4: Fix the boot regression with CPU_IDLE enabled

On OMAP4 panda board, there have been several bug reports about boot
hang and lock-ups with CPU_IDLE enabled. The root cause of the issue
is missing interrupts while in idle state. Commit cb7094e8 {cpuidle / omap4 :
use CPUIDLE_FLAG_TIMER_STOP flag} moved the broadcast notifiers to common
code for right reasons but on OMAP4 which suffers from a nasty ROM code
bug with GIC, commit ff999b8a {ARM: OMAP4460: Workaround for ROM bug ..},
we loose interrupts which leads to issues like lock-up, hangs etc.

Patch reverts commit cb7094 {cpuidle / omap4 : use CPUIDLE_FLAG_TIMER_STOP
flag} and 54769d6 {cpuidle: OMAP4: remove timer broadcast initialization} to
avoid the issue. With this change, OMAP4 panda boards, the mentioned
issues are getting fixed. We no longer loose interrupts which was the cause
of the regression.

Fixes: cb7094e8 (cpuidle / omap4 : use CPUIDLE_FLAG_TIMER_STOP flag)
Fixes: ff999b8a (cpuidle: OMAP4: remove timer broadcast initialization)
Cc: stable@vger.kernel.org # v3.9+
Cc: Roger Quadros <rogerq@ti.com>
Cc: Kevin Hilman <khilman@linaro.org>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Reported-tested-by: Roger Quadros <rogerq@ti.com>
Reported-tested-by: Kevin Hilman <khilman@linaro.org>
Tested-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Tony Lindgren <tony@atomide.com>

ARM: OMAP: Kill warning in CPUIDLE code with !CONFIG_SMP

for non SMP build, NR_CPUS is 1 and hence the code complains with below
warnings.

arch/arm/mach-omap2/cpuidle44xx.c:207:8: warning: array subscript is above array bounds [-Warray-bounds]
arch/arm/mach-omap2/cpuidle44xx.c:212:11: warning: array subscript is above array bounds [-Warray-bounds]

Kill it by making array size fixed.

Acked-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Mugunthan V N <mugunthanvnm@ti.com>
Signed-off-by: Tony Lindgren <tony@atomide.com>

ARM: OMAP4460: cpuidle: Extend PM_OMAP4_ROM_SMP_BOOT_ERRATUM_GICD on cpuidle

The same workaround as ff999b8a0983ee15668394ed49e38d3568fc6859
"ARM: OMAP4460: Workaround for ROM bug because of CA9 r2pX GIC ..."
need to be applied not only when system is booting, but when MPUSS hits
OSWR state through CPUIdle too. Without this WA the same issue is
reproduced now on boards PandaES and Tablet/Blaze with SOM OMAP4460
when CONFIG_CPU_IDLE is enabled.
After MPUSS has enterred OSWR and waken up:
- GIC distributor became disabled forever
- scheduling is not performed any more

Cc: Kevin Hilman <khilman@linaro.org>
Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Reported-by: Taras Kondratiuk <taras.kondratiuk@linaro.org>
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
Acked-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Kevin Hilman <khilman@linaro.org>
Signed-off-by: Olof Johansson <olof@lixom.net>

ARM: OMAP4: cpuidle: fix: call cpu_cluster_pm_exit conditionally

We call cpu_cluster_pm_enter for dev->cpu == 0 only, but
cpu_cluster_pm_exit called without that check.

Because of that unhandled page fault may happen:

[ 3.803405] Unable to handle kernel paging request at virtual address 00002500
[ 3.810974] pgd = c0004000
[ 3.813812] [00002500] *pgd=00000000
[ 3.817596] Internal error: Oops: 5 [#1] SMP ARM
[ 3.822418] Modules linked in:
[ 3.825653] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 3.11.0-rc6+ #21
[ 3.832397] task: ed86ef40 ti: ed896000 task.ti: ed896000
[ 3.838073] PC is at irq_notifier+0x234/0x25c
[ 3.842651] LR is at irq_notifier+0x218/0x25c
[ 3.847229] pc : [<c0029ed8>] lr : [<c0029ebc>] psr: 80000193
[ 3.847229] sp : ed897ee8 ip : 00000005 fp : 00000001
[ 3.859283] r10: c0b395f0 r9 : c0b30594 r8 : c0b8c2ac
[ 3.864776] r7 : ffffffff r6 : 00000000 r5 : 00000005 r4 : 00000000
[ 3.871643] r3 : 00002500 r2 : 00000000 r1 : 00000005 r0 : 44302244
[ 3.878479] Flags: Nzcv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel
[ 3.886260] Control: 10c5387d Table: 8000404a DAC: 00000015
[ 3.892272] Process swapper/1 (pid: 0, stack limit = 0xed896240)
[ 3.898590] Stack: (0xed897ee8 to 0xed898000)
[ 3.903167] 7ee0: c0979c3a 00000001 ed897ef8 ed896000 c0014f7c 00000000
[ 3.911743] 7f00: 00000005 00000000 ffffffff c0b8c2ac c0b395f0 c077c04c c0c94b48 c0b3953c
[ 3.920318] 7f20: c0bcd928 00000002 c0b39524 c00cfad8 00000000 ffffffff 00000000 c00cfb10
[ 3.928924] 7f40: c14e62c0 c002c1c8 c002c0ac c14e62c0 00000002 e251c37d 00000000 c0b39548
[ 3.937499] 7f60: c0b395f0 c05a1bc4 e251c37d 00000000 00000005 c05a3870 edc90380 edc90380
[ 3.946105] 7f80: edc90394 c14e62c0 c0b39548 00000002 c0784064 c05a3c78 c0b395e0 c14e62c0
[ 3.954681] 7fa0: 00000002 c0b39548 c0bc9db8 00000000 00000001 c05a1dc0 ed896000 00000015
[ 3.963287] 7fc0: c0bc9db8 ed896000 8000406a c0b30594 c0784064 c000e504 00000746 c007a528
[ 3.971862] 7fe0: 00000001 0000001d 600001d3 c0bcc004 00000000 800086c4 ee0aa6a7 d2aabaa9
[ 3.980499] [<c0029ed8>] (irq_notifier+0x234/0x25c) from [<c077c04c>] (notifier_call_chain+0x38/0x68)
[ 3.990173] [<c077c04c>] (notifier_call_chain+0x38/0x68) from [<c00cfad8>] (cpu_pm_notify+0x20/0x38)
[ 3.999786] [<c00cfad8>] (cpu_pm_notify+0x20/0x38) from [<c00cfb10>] (cpu_cluster_pm_exit+0x20/0x50)
[ 4.009399] [<c00cfb10>] (cpu_cluster_pm_exit+0x20/0x50) from [<c002c1c8>] (omap_enter_idle_coupled+0x11c/0x14c)
[ 4.020111] [<c002c1c8>] (omap_enter_idle_coupled+0x11c/0x14c) from [<c05a1bc4>] (cpuidle_enter_state+0x40/0xec)
[ 4.030822] [<c05a1bc4>] (cpuidle_enter_state+0x40/0xec) from [<c05a3c78>] (cpuidle_enter_state_coupled+0x1f4/0x240)
[ 4.041870] [<c05a3c78>] (cpuidle_enter_state_coupled+0x1f4/0x240) from [<c05a1dc0>] (cpuidle_idle_call+0x150/0x228)
[ 4.052947] [<c05a1dc0>] (cpuidle_idle_call+0x150/0x228) from [<c000e504>] (arch_cpu_idle+0x8/0x38)
[ 4.062499] [<c000e504>] (arch_cpu_idle+0x8/0x38) from [<c007a528>] (cpu_startup_entry+0x178/0x1e4)
[ 4.071990] [<c007a528>] (cpu_startup_entry+0x178/0x1e4) from [<800086c4>] (0x800086c4)
[ 4.080383] Code: e5922288 03a03b0a 13a03c25 e0823003 (e5932000)
[ 4.086791] ---[ end trace d83954a84a6fa69e ]---

It is supposed that sar_base is initialized in irq_save_context, which
is called on CPU_CLUSTER_PM_ENTER notification. If this notification
has been missed and CPU_CLUSTER_PM_EXIT is received sar_base is NULL.

Fix it by calling CPU_CLUSTER_PM_{ENTER,EXIT} under the same condition.

Signed-off-by: Vladimir Murzin <murzin.v@gmail.com>
Acked-by: Kevin Hilman <khilman@linaro.org>
Signed-off-by: Tony Lindgren <tony@atomide.com>

ARM: OMAP4: cpuidle: use init/exit common routine

Remove the duplicated code and use the cpuidle common code for initialization.

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Reviewed-by: Kevin Hilman <khilman@linaro.org>
Tested-by: Kevin Hilman <khilman@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

cpuidle: remove en_core_tk_irqen flag

The en_core_tk_irqen flag is set in all the cpuidle driver which
means it is not necessary to specify this flag.

Remove the flag and the code related to it.

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Acked-by: Kevin Hilman <khilman@linaro.org> # for mach-omap2/*
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

ARM: OMAP4+: CPUidle: Consolidate idle driver for OMAP5 support

The OMAP5 idle driver can re-use most of OMAP4 CPUidle driver
implementation. Also the next derivative SOCs are going to re-use
the MPUSS so, same driver with minor updates can be re-used.

Prepare the code so that its easier to add CPUidle support for
OMAP5 devices.

Acked-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Kevin Hilman <khilman@linaro.org>

ARM: OMAP4+: CPUidle: Deprecate use of omap4_mpuss_read_prev_context_state()

Current OMAP4 CPUIdle driver is using omap4_mpuss_read_prev_context_state()
to check whether the MPU cluster lost context or not before calling
cpu_cluster_pm_exit(). This was initially done an optimization for
corner cases, where if the cluster low power entry fails for some
reason, the cluster context restore gets skipped. However, since
reading the previous context is expensive (involving slow accesses to
the PRCM), it's better to avoid it and simply check the target cluster
state instead.

Moving forward, OMAP CPUidle drivers needs to be moved to drivers/idle/*
once the PRM/CM code gets moved to drivers. This patch also reduces one
dependency with platform code for idle driver movement.

Acked-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
[khilman@linaro.org: minor changelog edits]
Signed-off-by: Kevin Hilman <khilman@linaro.org>

ARM: OMAP4: CPUidle: Make C-state description field more precise

It is useful to know the CPU power state along with MPUSS power state
in a supported C-state. Since the data is available via sysfs, one can
avoid scrolling the source code for precise construction of C-state.

Reported-by: Nishanth Menon <nm@ti.com>
Acked-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Kevin Hilman <khilman@linaro.org>

ARM: OMAP: CPUidle: Unregister drivere on device registration failure

If the CPUidle device registration fails for some reason, we should
unregister the driver on error path.

Fix the code accordingly. Also when at it, check of the driver registration
failure too.

Acked-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Kevin Hilman <khilman@linaro.org>

ARM: OMAP4: CPUidle: Avoid double idle driver registration

OMAP4 CPUidle driver registration call is under a loop which leads
to calling cpuidle_register_driver twice which is not intended.

Fix it by moving the driver registration outside the loop.

Reported-by: Nishanth Menon <nm@ti.com>
Acked-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Kevin Hilman <khilman@linaro.org>

ARM: OMAP4+: PM: Restore CPU power state to ON with clockdomain force wakeup method

While waking up CPU from off state using clock domain force wakeup, restore
the CPU power state to ON state before putting CPU clock domain under
hardware control. Otherwise CPU wakeup might fail. The change is recommended
for all OMAP4+ devices though the PRCM weakness was observed on OMAP5
devices first.

As a result of weakness, lock-up is observed inside the hardware state
machine of local CPU PRCM and results are UN-predictable as per designers.
In software testing, we have seen hard-locks most of the time where system
gets frozen. With power domain state restored, system behaves correctly.

So update the code accordingly.

Acked-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Kevin Hilman <khilman@linaro.org>

cpuidle: OMAP4: remove timer broadcast initialization

The initialization is done from the cpuidle framework.

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

cpuidle / omap4 : use CPUIDLE_FLAG_TIMER_STOP flag

Use the CPUIDLE_FLAG_TIMER_STOP and let the cpuidle framework
to handle the CLOCK_EVT_NOTIFY_BROADCAST_ENTER/EXIT when entering
this state.

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

ARM: OMAP2+: PM: Remove bogus fiq_[enable/disable] tuple

On OMAP platform, FIQ is reserved for secure environment only. If at all
the FIQ needs to be disabled, it involves going through security
API call. Hence the local_fiq_[enable/disable]() in the OMAP code is bogus.
On GP devices too, the fiq is disabled for non-secure software.

So just get rid of it.

Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>

ARM: OMAP3/4: cpuidle: fix sparse and checkpatch warnings

Fix the following sparse warnings in the OMAP3/4 CPUIdle code:

arch/arm/mach-omap2/cpuidle34xx.c:272:1: warning: symbol 'omap3_idle_dev' was not declared. Should it be static?
arch/arm/mach-omap2/cpuidle34xx.c:274:23: warning: symbol 'omap3_idle_driver' was not declared. Should it be static?
arch/arm/mach-omap2/cpuidle44xx.c:164:1: warning: symbol 'omap4_idle_dev' was not declared. Should it be static?
arch/arm/mach-omap2/cpuidle44xx.c:166:23: warning: symbol 'omap4_idle_driver' was not declared. Should it be static?

Also fix the following checkpatch warnings:

WARNING: please, no space before tabs
#44: FILE: arch/arm/mach-omap2/cpuidle34xx.c:105:
+^I.name = ^I"omap3_idle",$

WARNING: please, no space before tabs
#45: FILE: arch/arm/mach-omap2/cpuidle34xx.c:106:
+^I.owner = ^ITHIS_MODULE,$

ERROR: code indent should use tabs where possible
#211: FILE: arch/arm/mach-omap2/cpuidle44xx.c:74:
+ /* C2 - CPU0 OFF + CPU1 OFF + MPU CSWR */$


Signed-off-by: Paul Walmsley <paul@pwsan.com>
Cc: Kevin Hilman <khilman@ti.com>
Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>

ARM: omap: allow building omap44xx without SMP

The new omap4 cpuidle implementation currently requires
ARCH_NEEDS_CPU_IDLE_COUPLED, which only works on SMP.

This patch makes it possible to build a non-SMP kernel
for that platform. This is not normally desired for
end-users but can be useful for testing.

Without this patch, building rand-0y2jSKT results in:

drivers/cpuidle/coupled.c: In function 'cpuidle_coupled_poke':
drivers/cpuidle/coupled.c:317:3: error: implicit declaration of function '__smp_call_function_single' [-Werror=implicit-function-declaration]

It's not clear if this patch is the best solution for
the problem at hand. I have made sure that we can now
build the kernel in all configurations, but that does
not mean it will actually work on an OMAP44xx.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Cc: Kevin Hilman <khilman@ti.com>
Cc: Tony Lindgren <tony@atomide.com>

ARM: OMAP4: CPUidle: Open broadcast clock-event device.

OMAP4 idle driver uses CLOCK_EVT_NOTIFY_BROADCAST_[ENTER/EXIT]
for broadcast clock events. But _ENTER/_EXIT doesn't really open
broadcast clock events and to explicitly setup the broadcast device,
CLOCK_EVT_NOTIFY_BROADCAST_ON should be used.

Add the missing CLOCK_EVT_NOTIFY_BROADCAST_ON clockevent notifications.
This will setup the broadcast timer in either periodic/oneshot modes
correctly. Recent clockevent infrastructure change 77b0d60 {leave the
broadcast device in shutdown mode when not needed} exposed this bug
leading to boot hangs in oneshot mode. Prior to this, periodic broadcast
mode was also broken. This change fixes both the periodic/oneshot broadcast
modes.

Discussion thread :
https://lkml.org/lkml/2012/4/9/13

Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>

ARM: OMAP4: CPUidle: add synchronization for coupled idle states

With coupled idle states, a failure for any CPU to hit a low power
state must be coordinated such that all CPUs abort.

On OMAP4, when entering a coupled state, CPU0 has to wait for CPU1 to
enter its low power state before it can enter its low power state.

This is implemented by letting CPU0 wait for the CPU1 powerdomain to
hit off. However, there are conditions where CPU1 might abort/fail
and not hit off while CPU0 is waiting for it. For example, a CPU1
wakeup or a failed attempt to hit off due to hardware conditions.

To avoid the deadlock where CPU0 would continually wait for CPU1 to
hit off-mode, this patch adds a flag to signal when each CPU has come
out of its low-power state. CPU0 then checks whether CPU1 has hit off
*or* has already completed its attempt to hit off. If the latter,
CPU0 must abort its attempt to hit a low-power state so the coupled
state enter method can return.

In addition, cpuidle_coupled_parallel_barrier() is used to ensure the
clearing of the 'done' flag is synchronized on all CPUs.

Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>

ARM: OMAP4: CPUidle: Use coupled cpuidle states to implement SMP cpuidle.

OMAP4 CPUDILE driver is converted mainly based on notes from the
coupled cpuidle patch series.

The changes include :
- Register both CPUs and C-states to cpuidle driver.
- Set struct cpuidle_device.coupled_cpus
- Set struct cpuidle_device.safe_state to non coupled state.
- Set CPUIDLE_FLAG_COUPLED in struct cpuidle_state.flags for each
state that affects multiple cpus.
- Separate ->enter hooks for coupled & simple idle.
- CPU0 wait loop for CPU1 power transition.
- CPU1 wakeup mechanism for the idle exit.
- Enabling ARCH_NEEDS_CPU_IDLE_COUPLED for OMAP4.

Thanks to Kevin Hilman and Colin Cross on the suggestions/fixes
on the intermediate version of this patch.

Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>

ARM: OMAP3/4: consolidate cpuidle Makefile

The current Makefile compiles the cpuidle34xx.c and cpuidle44xx.c files
even if the cpuidle option is not set in the kernel.

This patch fixes this by creating a section in the Makefile where these
files are compiled only if the CONFIG_CPU_IDLE option is set.

This modification breaks an implicit dependency between CPU_IDLE and PM as
they belong to the same block in the Makefile. This is fixed in the Kconfig
by selecting explicitely PM is CPU_IDLE is set.

The linux coding style recommend to use no-op functions in the headers
when the subsystem is disabled instead of adding big section in C files.

This patch fix this also.

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Reviewed-by: Jean Pihet <j-pihet@ti.com>
Reviewed-by: Rajendra Nayak <rnayak@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>

ARM: OMAP4: cpuidle - remove omap4_idle_data initialization at boot time

We initialized it at compile time, no need to do that at boot
time.

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Reviewed-by: Jean Pihet <j-pihet@ti.com>
Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Kevin Hilman <khilman@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>

ARM: OMAP4: cpuidle - use the omap4_idle_data variable directly

We are storing the 'omap4_idle_data' in the private data field
of the cpuidle device. As we are using this variable only in this file,
that does not really make sense. Let's use the global variable directly.

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Reviewed-by: Jean Pihet <j-pihet@ti.com>
Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Kevin Hilman <khilman@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>

ARM: OMAP4: cpuidle - Initialize omap4_idle_data at compile time

We initialize the omap4_idle_data variable at compile time allowing us
to remove in the next patch the initialization done at boot time.

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Reviewed-by: Jean Pihet <j-pihet@ti.com>
Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Kevin Hilman <khilman@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>

ARM: OMAP4: cpuidle - fix static omap4_idle_data declaration

Add the static declaration for the omap4_idle_data variable because its scope
is in the file only.

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Reviewed-by: Jean Pihet <j-pihet@ti.com>
Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Kevin Hilman <khilman@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>

ARM: OMAP4: cpuidle - Remove the cpuidle_params_table table

We do not longer need this table as we defined the values
in the driver states.

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Reviewed-by: Jean Pihet <j-pihet@ti.com>
Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Kevin Hilman <khilman@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>

ARM: OMAP4: cpuidle - Declare the states with the driver declaration

The cpuidle API allows to declare statically the states in the driver
structure. Let's use it.
We do no longer need the fill_cstate function called at runtime and
by the way adding more instructions at boot time.

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Reviewed-by: Jean Pihet <j-pihet@ti.com>
Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Kevin Hilman <khilman@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>

ARM: OMAP4: cpuidle - Remove unused valid field

The 'valid' field is never used in the code, let's remove it.

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Reviewed-by: Jean Pihet <j-pihet@ti.com>
Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Kevin Hilman <khilman@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>

ARM: omap: Consolidate OMAP4 time keeping and irq enable

Enable core cpuidle timekeeping and irq enabling and remove that
handling from this code.

Signed-off-by: Robert Lee <rob.lee@linaro.org>
Reviewed-by: Kevin Hilman <khilman@ti.com>
Reviewed-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Acked-by: Jean Pihet <j-pihet@ti.com>
Signed-off-by: Len Brown <len.brown@intel.com>

ARM: OMAP4: cpuidle: Fix the C-state reporting to cpuidle governor.

OMAP4 cpuidle driver is reporting the state requested by governor
rather than the actually attempted one.

This is obviously misleading sysfs and powertop cpuidle statistics.
Fix it so that stats are reported correctly.

Reported-by: Kevin Hilman <khilman@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
[khilman@ti.com: minor changelog edits]
Signed-off-by: Kevin Hilman <khilman@ti.com>

ARM: OMAP4: cpuidle: Switch to gptimer from twd in deeper C-states.

CPU local timer(TWD) stops when the CPU is transitioning into
deeper C-States. Since these timers are not wakeup capable, we
need the wakeup capable global timer to program the wakeup time
depending on the next timer expiry.

It can be handled by registering a global wakeup capable timer along
with local timers marked with (mis)feature flag CLOCK_EVT_FEAT_C3STOP.
Then notify the clock events layer from idle code using
CLOCK_EVT_NOTIFY_BROADCAST_ENTER/EXIT).

ARM local timers are already marked with C3STOP feature. Add the
notifiers to OMAP4 CPU idle code for the broadcast entry and exit.

Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Acked-by: Jean Pihet <j-pihet@ti.com>
Acked-by: Kevin Hilman <khilman@ti.com>
Tested-by: Vishwanath BS <vishwanath.bs@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>

ARM: OMAP4: PM: Add CPUidle support

Add OMAP4 CPUIDLE support. CPU1 is left with defualt idle and
the low power state for it is managed via cpu-hotplug.

This patch adds MPUSS low power states in cpuidle.

C1 - CPU0 ON + CPU1 ON + MPU ON
C2 - CPU0 OFF + CPU1 OFF + MPU CSWR
C3 - CPU0 OFF + CPU1 OFF + MPU OSWR

OMAP4460 onwards, MPUSS power domain doesn't support OFF state any more
anymore just like CORE power domain. The deepest state supported is OSWr.
Ofcourse when MPUSS and CORE PD transitions to OSWR along with device
off mode, even the memory contemts are lost which is as good as
the PD off state.

On OMAP4 because of hardware constraints, no low power states are
targeted when both CPUs are online and in SMP mode. The low power
states are attempted only when secondary CPU gets offline to OFF
through hotplug infrastructure.

Thanks to Nicole Chalhoub <n-chalhoub@ti.com> for doing exhaustive
C-state latency profiling.

Signed-off-by: Rajendra Nayak <rnayak@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Acked-by: Jean Pihet <j-pihet@ti.com>
Reviewed-by: Kevin Hilman <khilman@ti.com>
Tested-by: Vishwanath BS <vishwanath.bs@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>