ARM: smp: Switch to hotplug core state synchronization

Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Michael Kelley <mikelley@microsoft.com>
Tested-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Tested-by: Helge Deller <deller@gmx.de> # parisc
Tested-by: Guilherme G. Piccoli <gpiccoli@igalia.com> # Steam Deck
Link: https://lore.kernel.org/r/20230512205256.635326070@linutronix.de

ARM: remove some dead code

This code appears to be no longer used so let's get rid of it.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Tested-by: Keith Packard <keithpac@amazon.com>
Tested-by: Marc Zyngier <maz@kernel.org>
Tested-by: Vladimir Murzin <vladimir.murzin@arm.com> # ARMv7M

ARM: smp: Pass task to secondary_start_kernel

This avoids needing to compute the task pointer in this function, which
will no longer be possible once we move thread_info off the stack.

Signed-off-by: Keith Packard <keithpac@amazon.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Tested-by: Amit Daniel Kachhap <amit.kachhap@arm.com>

treewide: Convert macro and uses of __section(foo) to __section("foo")

Use a more generic form for __section that requires quotes to avoid
complications with clang and gcc differences.

Remove the quote operator # from compiler_attributes.h __section macro.

Convert all unquoted __section(foo) uses to quoted __section("foo").
Also convert __attribute__((section("foo"))) uses to __section("foo")
even if the __attribute__ has multiple list entry forms.

Conversion done using the script at:

https://lore.kernel.org/lkml/75393e5ddc272dc7403de74d645e6c6e0f4e70eb.camel@perches.com/2-convert_section.pl

Signed-off-by: Joe Perches <joe@perches.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@gooogle.com>
Reviewed-by: Miguel Ojeda <ojeda@kernel.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

ARM: Kill __smp_cross_call and co

The old IPI registration interface is now unused on arm, so let's
get rid of it.

Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>

ARM: Allow IPIs to be handled as normal interrupts

In order to deal with IPIs as normal interrupts, let's add
a new way to register them with the architecture code.

set_smp_ipi_range() takes a range of interrupts, and allows
the arch code to request them as if the were normal interrupts.
A standard handler is then called by the core IRQ code to deal
with the IPI.

This means that we don't need to call irq_enter/irq_exit, and
that we don't need to deal with set_irq_regs either. So let's
move the dispatcher into its own function, and leave handle_IPI()
as a compatibility function.

On the sending side, let's make use of ipi_send_mask, which
already exists for this purpose.

One of the major difference is that we end up, in some cases
(such as when performing IRQ time accounting on the scheduler
IPI), end up with nested irq_enter()/irq_exit() pairs.
Other than the (relatively small) overhead, there should be
no consequences to it (these pairs are designed to nest
correctly, and the accounting shouldn't be off).

Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500

Based on 2 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation #

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-only

has been chosen to replace the boilerplate/reference in 4122 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Enrico Weigelt <info@metux.net>
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Allison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

ARM: smp: remove arch-provided "pen_release"

Consolidating the "pen_release" stuff amongst the various SoC
implementations gives credence to having a CPU holding pen for
secondary CPUs. However, this is far from the truth.

Many SoC implementations cargo-cult copied various bits of the pen
release implementation from the initial Realview/Versatile Express
implementation without understanding what it was or why it existed.
The reason it existed is because these are _development_ platforms,
and some board firmware is unable to individually control the
startup of secondary CPUs. Moreover, they do not have a way to
power down or reset secondary CPUs for hot-unplug. Hence, the
pen_release implementation was designed for ARM Ltd's development
platforms to provide a working implementation, even though it is
very far from what is required.

It was decided a while back to reduce the duplication by consolidating
the "pen_release" variable, but this only made the situation worse -
we have ended up with several implementations that read this variable
but do not write it - again, showing the cargo-cult mentality at work,
lack of proper review of new code, and in some cases a lack of testing.

While it would be preferable to remove pen_release entirely from the
kernel, this is not possible without help from the SoC maintainers,
which seems to be lacking. However, I want to remove pen_release from
arch code to remove the credence that having it gives.

This patch removes pen_release from the arch code entirely, adding
private per-SoC definitions for it instead, and explicitly stating
that write_pen_release() is cargo-cult copied and should not be
copied any further. Rename write_pen_release() in a similar fashion
as well.

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

ARM: 8708/1: NOMMU: Rework MPU to be mostly done in C

Currently, there are several issues with how MPU is setup:

1. We won't boot if MPU is missing
2. We won't boot if use XIP
3. Further extension of MPU setup requires asm skills

The 1st point can be relaxed, so we can continue with boot CPU even if
MPU is missed and fail boot for secondaries only. To address the 2nd
point we could create region covering CONFIG_XIP_PHYS_ADDR - _end and
that might work for the first stage of MPU enable, but due to MPU's
alignment requirement we could cover too much, IOW we need more
flexibility in how we're partitioning memory regions... and it'd be
hardly possible to archive because of the 3rd point.

This patch is trying to address 1st and 3rd issues and paves the path
for 2nd and further improvements.

The most visible change introduced with this patch is that we start
using mpu_rgn_info array (as it was supposed?), so change in MPU setup
done by boot CPU is recorded there and feed to secondaries. It
allows us to keep minimal region setup for boot CPU and do the rest in
C. Since we start programming MPU regions in C evaluation of MPU
constrains (number of regions supported and minimal region order) can
be done once, which in turn open possibility to free-up "probe"
region early.

Tested-by: Szemző András <sza@esh.hu>
Tested-by: Alexandre TORGUE <alexandre.torgue@st.com>
Tested-by: Benjamin Gaignard <benjamin.gaignard@linaro.org>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

ARM: 8423/1: add const qualifier to smp_operations member in structures

The core framework does not modify smp_operations structures.
To clarify it, this commit adds 'const' qualifier to the 'ops'
member of struct of_cpu_method and the 'smp' member of struct
machine_desc.

This change allows each SoC code to add 'const' qualifier to its
smp_operation structure.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: 8424/1: add const qualifier to the argument of smp_set_ops()

This function just copies '*ops' to 'smp_ops', so the given
structure '*ops' is not modified at all.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: 8421/1: smp: Collapse arch_cpu_idle_dead() into cpu_die()

The only caller of cpu_die() on ARM is arch_cpu_idle_dead(), so
let's simplify the code by renaming cpu_die() to
arch_cpu_idle_dead(). While were here, drop the __ref annotation
because __cpuinit is gone nowadays.

Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: 8392/3: smp: Only expose /sys/.../cpuX/online if hotpluggable

Writes to /sys/.../cpuX/online fail if we determine the platform
doesn't support hotplug for that CPU. Furthermore, if the cpu_die
op isn't specified the system hangs when we try to offline a CPU
and it comes right back online unexpectedly. Let's figure this
stuff out before we make the sysfs nodes so that the online file
doesn't even exist if it isn't (at least sometimes) possible to
hotplug the CPU.

Add a new 'cpu_can_disable' op and repoint all 'cpu_disable'
implementations at it because all implementers use the op to
indicate if a CPU can be hotplugged or not in a static fashion.
With PSCI we may need to add a 'cpu_disable' op so that the
secure OS can be migrated off the CPU we're trying to hotplug.
In this case, the 'cpu_can_disable' op will indicate that all
CPUs are hotpluggable by returning true, but the 'cpu_disable' op
will make a PSCI migration call and occasionally fail, denying
the hotplug of a CPU. This shouldn't be any worse than x86 where
we may indicate that all CPUs are hotpluggable but occasionally
we can't offline a CPU due to check_irq_vectors_for_cpu_disable()
failing to find a CPU to move vectors to.

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Nicolas Pitre <nico@linaro.org>
Cc: Dave Martin <Dave.Martin@arm.com>
Acked-by: Simon Horman <horms@verge.net.au> [shmobile portion]
Tested-by: Simon Horman <horms@verge.net.au>
Cc: Magnus Damm <magnus.damm@gmail.com>
Cc: <linux-sh@vger.kernel.org>
Tested-by: Tyler Baker <tyler.baker@linaro.org>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: 8360/1: add secondary_startup_arm prototype in header file

Put secondary_startup_arm() prototype in arch/arm/include/asm/smp.h
so users doesn't have to add extern prototype in their code.

Signed-off-by: Yingjoe Chen <yingjoe.chen@mediatek.com>
Reviewed-by: Matthias Brugger <matthias.bgg@gmail.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: redo TTBR setup code for LPAE

Re-engineer the LPAE TTBR setup code. Rather than passing some shifted
address in order to fit in a CPU register, pass either a full physical
address (in the case of r4, r5 for TTBR0) or a PFN (for TTBR1).

This removes the ARCH_PGD_SHIFT hack, and the last dangerous user of
cpu_set_ttbr() in the secondary CPU startup code path (which was there
to re-set TTBR1 to the appropriate high physical address space on
Keystone2.)

Tested-by: Murali Karicheri <m-karicheri2@ti.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: 8131/1: arm/smp: Absorb boot_secondary()

After becoming a mandatory function, boot_secondary() is no longer used
outside arch/arm/kernel/smp.c. Hence remove its public prototype, and,
as suggested by Arnd, let it be absorbed by its single caller.

Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: sunxi: fix build for THUMB2_KERNEL

Building an SMP kernel for the sunxi platform with THUMB2 instructions
fails with this error at the moment:

headsmp.S:7: Error: Thumb encoding does not support an immediate here -- `msr cpsr_fsxc,#0xd3'

Since the generic secondary_startup function already does
the same thing in a safe way, we can just drop the private
sunxi implementation and jump straight to secondary_startup.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Cc: Maxime Ripard <maxime.ripard@free-electrons.com>

ARM: Introduce CPU_METHOD_OF_DECLARE() for cpu hotplug/smp

The goal of multi-platform kernels is to remove the need for mach
directories and machine descriptors. To further that goal,
introduce CPU_METHOD_OF_DECLARE() to allow cpu hotplug/smp
support to be separated from the machine descriptors.
Implementers should specify an enable-method property in their
cpus node and then implement a matching set of smp_ops in their
hotplug/smp code, wiring it up with the CPU_METHOD_OF_DECLARE()
macro. When the kernel is compiled we'll collect all the
enable-method smp_ops into one section for use at boot.

At boot time we'll look for an enable-method in each cpu node and
try to match that against all known CPU enable methods in the
kernel. If there are no enable-methods in the cpu nodes we
fallback to the cpus node and try to use any enable-method found
there. If that doesn't work we fall back to the old way of using
the machine descriptor.

Acked-by: Mark Rutland <mark.rutland@arm.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: <devicetree@vger.kernel.org>
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: Kumar Gala <galak@codeaurora.org>

ARM: SMP: basic IPI triggered completion support

We need a mechanism to let an inbound CPU signal that it is alive before
even getting into the kernel environment i.e. from early assembly code.
Using an IPI is the simplest way to achieve that.

This adds some basic infrastructure to register a struct completion
pointer to be "completed" when the dedicated IPI for this task is
received.

Signed-off-by: Nicolas Pitre <nico@linaro.org>

ARM: mpu: add MPU initialisation for secondary cores

The MPU initialisation on the primary core is performed in two stages, one
minimal stage to ensure the CPU can boot and a second one after
sanity_check_meminfo. As the memory configuration is known by the time we
boot secondary cores only a single step is necessary, provided the values
for DRSR are passed to secondaries.

This patch implements this arrangement. The configuration generated for the
MPU regions is made available to the secondary core, which can then use the
asm MPU intialisation code to program a complete region configuration.

This is necessary for SMP configurations without an MMU, as the MPU
initialisation is the only way to ensure that memory is specified as
'shared'.

Signed-off-by: Jonathan Austin <jonathan.austin@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
CC: Nicolas Pitre <nico@linaro.org>

ARM: 7571/1: SMP: add function arch_send_wakeup_ipi_mask()

Add function arch_send_wakeup_ipi_mask(), so that platform code can
use it as an easy way to wake up cores that are in WFI.

Signed-off-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: consolidate pen_release instead of having per platform definitions

Almost each SMP platform defines pen_release to manage booting secondary
CPUs. This of course clashes with the single zImage effort.

Add the pen_release definition to the ARM SMP code, and remove all others.
This should only be used by platforms which lack any kind of CPU power
management...

Reported-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>

ARM: smp: Make SMP operations mandatory

Now that all SMP platforms have been converted to use struct
smp_operations, remove the "weak" attribute from the hooks
in smp.c, and make the functions static wherever possible.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>

ARM: SoC: add per-platform SMP operations

This adds a 'struct smp_operations' to abstract the CPU initialization
and hot plugging functions on SMP systems, which otherwise conflict
in a multiplatform kernel. This also helps shmobile and potentially
others that have more than one method to do these.

To allow the kernel to continue building, the platform hooks are
defined as weak symbols which are overrided by the platform code.
Once all platforms are converted, the "weak" attribute will be
removed and the function made static.

Unlike the original version from Marc, this new version from Arnd
does not use a generalized abstraction for per-soc data structures
but only tries to solve the problem for the SMP operations. This
way, we can collapse the previous four data structures into a
single struct, which is less systematic but also easier to follow
as a causal reader.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>

ARM: 7293/1: logical_cpu_map: decouple CPU mapping from SMP

It turns out that the logical CPU mapping is useful even when !CONFIG_SMP
for manipulation of devices like interrupt and power controllers when
running a UP kernel on a CPU other than 0. This can happen when kexecing
a UP image from an SMP kernel.

In the future, multi-cluster systems running AMP configurations will
require something similar for mapping cluster IDs, so it makes sense to
decouple this logic in preparation for this support.

Acked-by: Yang Bai <hamo.by@gmail.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Reported-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: gic: consolidate PPI handling

PPI handling is a bit of an odd beast. It uses its own low level
handling code and is hardwired to the local timers (hence lacking
a registration interface).

Instead, switch the low handling to the normal SPI handling code.
PPIs are handled by the handle_percpu_devid_irq flow.

This also allows the removal of some duplicated code.

Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: David Brown <davidb@codeaurora.org>
Cc: Bryan Huntsman <bryanh@codeaurora.org>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: Magnus Damm <magnus.damm@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Acked-by: David Brown <davidb@codeaurora.org>
Tested-by: David Brown <davidb@codeaurora.org>
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>

ARM: 7123/1: smp: Add an IPI handler callable from C code

In order to be able to handle IPI directly from C code instead of
assembly code, introduce handle_IPI(), which is modeled after handle_IRQ().

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: 7060/1: smp: populate logical CPU mapping during boot

To allow booting Linux on a CPU with physical ID != 0, we need to
provide a mapping from the logical CPU number to the physical CPU
number.

This patch adds such a mapping and populates it during boot.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: 6942/1: mm: make TTBR1 always point to swapper_pg_dir on ARMv6/7

This patch makes TTBR1 point to swapper_pg_dir so that global, kernel
mappings can be used exclusively on v6 and v7 cores where they are
needed.

Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

Remove unused PROC_CHANGE_PENALTY constant

This constant hasn't been used since before the git era (2.6.12) and thus
can be dropped.

Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Richard Weinberger <richard@nod.at>
Cc: Hirokazu Takata <takata@linux-m32r.org>
Cc: Kyle McMartin <kyle@mcmartin.ca>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Matt Turner <mattst88@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

ARM: consolidate SMP cross call implementation

Rather than having each platform class provide a mach/smp.h header for
smp_cross_call(), arrange for them to register the function with the
core ARM SMP code instead.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: SMP: consolidate the common parts of smp_prepare_cpus()

There is a certain amount of smp_prepare_cpus() which doesn't belong
in the platform support code - that is, code which is invariant to the
SMP implementation. Move this code into arch/arm/kernel/smp.c, and
add a platform_ prefix to the original function.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: fix /proc/interrupts formatting

As per x86, align the initial column according to how many IRQs we
have. Also, provide an english explaination for the 'LOC:' and
'IPI:' lines.

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: SMP: pass an ipi number to smp_cross_call()

This allows us to use smp_cross_call() to trigger a number of different
software generated interrupts, rather than combining them all on one
SGI. Recover the SGI number via do_IPI.

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: rename mach_cpu_disable() to platform_cpu_disable()

Consistently name all SMP platform related functions.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

cpumask: remove arch_send_call_function_ipi

Now everyone is converted to arch_send_call_function_ipi_mask, remove
the shim and the #defines.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

[ARM] smp: use new cpumask functions

Convert cpu_*_mask bit twiddling to the new set_cpu_*() API.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

[ARM] smp: move core localtimer support out of platform specific files

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

[ARM] smp: fix cpumask usage in ARM SMP code

The ARM SMP code wasn't properly updated for the cpumask changes, which
results in smp_timer_broadcast() broadcasting ticks to non-online CPUs.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

RealView: Use only the shadow mapping of ARM11MPCore local timers

All the cases where the local timer for a CPU is accessed happen on the
corresponding current CPU, hence no need to access the per-CPU local
timer mappings.

Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>

[ARM] Move include/asm-arm/arch-* to arch/arm/*/include/mach

This just leaves include/asm-arm/plat-* to deal with.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

[ARM] move include/asm-arm to arch/arm/include/asm

Move platform independent header files to arch/arm/include/asm, leaving
those in asm/arch* and asm/plat* alone.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>