sys: Remove $FreeBSD$: one-line .h pattern

Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

Wrap the default SPE config in its own #define

No functional change. Cleans up the code a little.

Clean up spr.h

Remove SPRs for CPUs FreeBSD doesn't run on
Add debug register SPRs from the Freescale EREF

Fix powerpc and powerpcspe builds

This change fixes 32-bit PowerPC builds, that r367390 broke
(shift count >= width of type).

[PowerPC] Make PPC 970 PMC SPRs the standard ones

And add a _74XX suffix to 74XX SPRs.

This is a preparation for adding support to POWER8/9 PMCs, which have most
SPRs equal to 970 ones.

Reviewed by: jhibbits
Sponsored by: Eldorado Research Institute (eldorado.org.br)
Differential Revision: https://reviews.freebsd.org/D26532

[PowerPC64LE] powernv ILE setup code.

When running without a hypervisor, we need to set the ILE bit in the LPCR
ourselves.

For the boot processor, handle it in powernv_attach() like we do for other
LPCR bits.

No change for the APs, as they will use the lpcr global to set up their own
LPCR when they do their own cpudep_ap_early_bootstrap() and pick up this
automatically.

Sponsored by: Tag1 Consulting, Inc.

powerpc: clean up empty lines in .c and .h files

[PowerPC] Fix booke64 qemu infinite loop in L2 cache enable

Since qemu does not implement the L2 cache, we get stuck forever waiting
for a bit to be set when trying to invalidate it.

To prevent that, we should bail out if the L2 cache is missing.
One easy way to check this is L2CFG0 == 0 (since L2CSIZE always has at
least one bit set in a valid implementation)

(tested on qemu, rb800, and x5000)

Reviewed by: jhibbits
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D25225

powerpc64: Implement Radix MMU for POWER9 CPUs

Summary:
POWER9 supports two MMU formats: traditional hashed page tables, and Radix
page tables, similar to what's presesnt on most other architectures. The
PowerISA also specifies a process table -- a table of page table pointers--
which on the POWER9 is only available with the Radix MMU, so we can take
advantage of it with the Radix MMU driver.

Written by Matt Macy.

Differential Revision: https://reviews.freebsd.org/D19516

powerpc: Add a CPU-custom machine check handler

Summary:
Some machine checks are process-recoverable, others are not. Let a
CPU-specific handler decide what to do.

This works around a machine check error hit while building www/firefox
and mail/thunderbird, which would otherwise cause the build to fail.

More work is needed to handle all possible machine check conditions, but
this is sufficient to unblock some ports building.

Differential Revision: https://reviews.freebsd.org/D23731

powerpc: Axe PPC4xx support.

Summary:
The support was added almost a decade ago, and never completed. Just axe
it. It was also inadvertently broken 5 years ago, and nobody noticed.

Reviewed by: bdragon
Differential Revision: https://reviews.freebsd.org/D23753

Add support for building Book-E kernels with clang/lld.

This involved several changes:

* Since lld does not like text relocations, replace SMP boot page text relocs
in booke/locore.S with position-independent math, and track the virtual base
in the SMP boot page header.

* As some SPRs are interpreted differently on clang due to the way it handles
platform-specific SPRs, switch m*dear and m*esr mnemonics out for regular
m*spr. Add both forms of SPR_DEAR to spr.h so the correct encoding is selected.

* Change some hardcoded 32 bit things in the boot page to be pointer-sized, and
fix alignment.

* Fix 64-bit build of booke/pmap.c when enabling pmap debugging.

Additionally, I took the opportunity to document how the SMP boot page works.

Approved by: jhibbits (mentor)
Differential Revision: https://reviews.freebsd.org/D21999

powerpc: Add support for additional FSCR-managed facilities

Add support to enable, save, and restore the following facilities:
* Target Address Register (bctar) -- seemingly just another register to
branch to.
* Event-based branching -- an interrupt-like userspace event handler
subsystem.
* Load-monitored facility -- A facility that allows monitoring a range of
physical memory, and triggering an event on access. Targeted to garbage
collection software features.

powerpc: Add SPR definitions for additional POWER8/POWER9 facilities

This only adds the new SPR definitions and the associated FSCR bits. The
facilities themselves will be added in separate commits.

powerpc64: Add the DSCR facility on POWER8 and later

The Data Stream Control Register (DSCR) is privileged on POWER7, but
unprivileged (different register) on POWER8 and later. However, it's now
guarded by a new register, the Facility Status and Control Register, instead of
the MSR like other pre-existing facilities (FPU, Altivec). The FSCR must be
managed explicitly, since it's effectively an extension of the MSR.

Tested by: Brandon Bergren

powerpc: Add POWER8NVL definition

The POWER8NVL (POWER8 NVLink) architecturally behaves identically to the
POWER8, with a different PVR identifier. Mark it as such, so it shows up
appropriately to the user.

Reported by: Alexey Kardashevskiy
MFC after: 2 weeks

powerpc: Fix whitespace in SPR header.

powerpc64: Increase the nap level on power9 idling

The POWER9 documentation specifies that levels 0-3 are the 'lightest' sleep
level, meaning lowest latency and with no state loss. However, state 3 is
not implemented, and is instead reserved for future chips. This now
properly configures the PSSCR, specifying state 2 as the lowest level to
enter, but request level 0 for quickest sleep level. If the OCC determines
that the CPU can enter states 1 or 2 it will trigger the transition to those
states on demand.

MFC after: 1 week

powernv: Add Hypervisor Maintenance Interrupt handler

Attempting to build www/firefox on POWER9 resulted in a HMI exception being
thrown, a fatal trap currently. This is typically caused by timer facility
errors, but examination of the Hypervisor Maintenance Exception Register
(HMER) yielded only that an exception had recovered, with no information of
the actual exception cause.

When an HMI occurs, OPAL_HANDLE_HMI or OPAL_HANDLE_HMI2 must be called to
handle the exception at the firmware level. If the exception is handled, we
can continue.

This adds only the preliminary handler, enough to prevent package building
from panicking. An enhancement in the future is to use the flags returned
by OPAL_HANDLE_HMI2 to print more useful error messages, and log maintenance
events.

Reviewed by: luporl
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19634

powerpcspe: Implement SPE exception handling

The Signal Processing Engine (SPE) found in Freescale e500 cores (and
others) offloads IEEE-754 compliance (NaN, Inf handling, overflow,
underflow) to software, most likely as a means of simplifying the APU
silicon. Some software, like AbiWord, needs full IEEE-754 compliance,
including NaN handling. Implement the necessary bits to enable it.

Differential Revision: https://reviews.freebsd.org/D17446

Sort SPR_SPEFSCR in the SPR list

Also remove duplicate definition of SPR_IBAT0U.

Add the rest of the files for r335481

Missed hooking PMCR cpufreq(4) to the build, and adding the SPR to the header.

Stop idle threads on power9 in the idle task until an interrupt.

This reduces the CPU cycle wastage on power9, which is SMT4. Any idle
thread that's spinning is simply starving working threads on the same core
of valuable resources.

This can be reduced further by taking more advantage of the PSSCR supported
states, as well as permitting state loss, as is currently done for power8.
The currently implemented stop state is the lowest latency, which may still
consume resources.

Add support for the XIVE XICS emulation mode for POWER9 systems

Summary:
POWER9 systems use a new interrupt controller, XIVE, managed through OPAL
firmware calls. The OPAL firmware includes support for emulating the previous
generation XICS presentation layer in addition to a new "XIVE Exploitation"
mode. As a stopgap until we have XIVE exploitation mode, enable XICS emulation
mode so that we at least have an interrupt controller.

Since the CPPR is local to the current CPU, it cannot be updated for APs when
initializing on the BSP. This adds a new function, directly called by the
powernv platform code, to initialize the CPPR on AP bringup.

Reviewed by: nwhitehorn
Differential Revision: https://reviews.freebsd.org/D15492

Add SPR_HSRR0/SPR_HSRR1 definitions

Reported by: Mark Millard
Pointy-hat to: jhibbits

Add support for powernv POWER9 MMU initialization

The POWER9 MMU (PowerISA 3.0) is slightly different from current
configurations, using a partition table even for hypervisor mode, and
dropping the SDR1 register. Key off the newly early-enabled CPU features
flags for the new architecture, and configure the MMU appropriately.

The POWER9 MMU ignores the "PSIZ" field in the PTCR, and expects a 64kB
table. As we are enabled for powernv (hypervisor mode, no VMs), only
initialize partition table entry 0, and zero out the rest. The actual
contents of the register are identical to SDR1 from previous architectures.

Along with this, fix a bug in the page table allocation with very large
memory. The table can be allocated on any 256k boundary. The
bootstrap_alloc alignment argument is an int, and with large amounts of
memory passing the size of the table as the alignment will overflow an
integer. Hard-code the alignment at 256k as wider alignment is not
necessary.

Reviewed by: nwhitehorn
Tested by: Breno Leitao
Relnotes: Yes

powerpc64: Add DSCR support

Summary:
Powerpc64 has support for a register called Data Stream Control Register
(DSCR), which basically controls how the hardware controls the caching and
prefetch for stream operations.

Since mfdscr and mtdscr are privileged instructions, we need to emulate them,
and
keep the custom DSCR configuration per thread.

The purpose of this feature is to change DSCR depending on the operation, set
to DSCR Default Prefetch Depth to deepest on string operations, as memcpy.

Submitted by: Breno Leitao
Differential Revision: https://reviews.freebsd.org/D15081

Fix typo

Reserved cause is 6, not 5.

Reported by: cem

Powerpc64: Add the facility unavailable trap subsystem

Summary:
This code adds the basic infrastructure for the facility subsystem. A facility
trap is raised when an unavailable instruction is executed. One example is
executing a Hardware Transactional Memory instruction while the MSR[TM] is
disabled. In the past, there was a specific interrupt for it (FP, VEC), but the
new instructions seem to be multiplexed on this facility interrupt.

The root cause of the trap is provided on Facility Status and Control Register
(FSCR) register.

Submitted by: Breno Leitao
Reviewed by: nwhitehorn
Differential Revision: https://reviews.freebsd.org/D14566

Add definition for the PowerPC A2.

PowerNV: Put processor to power-save state in idle thread

When processor enters power-save state it releases resources shared with other
cpu threads which makes other cores working much faster.

This patch also implements saving and restoring registers that might get
corrupted in power-save state.

Submitted by: Patryk Duda <pdk@semihalf.com>
Obtained from: Semihalf
Reviewed by: jhibbits, nwhitehorn, wma
Sponsored by: IBM, QCM Technologies
Differential revision: https://reviews.freebsd.org/D14330

Remove some unused AIM register declarations that existed to support some
CPUs we have never run on. As a side-effect, removes some #ifdef AIM/#else.

PowerNV: set LPCR[LPES] correctly

Make sure to set LPCR[LPES] so that external interrupts set SRR0 and SRR1
instead of HSRR0 and HSRR1. Without this, external interrupt handlers would
get the wrong MSR value when executing, causing eventual madness.

Created by: Nathan Whitehorn <nwhitehorn@freebsd.org>
Submitted by: Wojciech Macek <wma@freebsd.org>
Sponsored by: FreeBSD Foundation

sys/powerpc: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Definitions for registers and trap types found on new POWER CPUs.

MFC after: 3 weeks

Add P5010/P5010E for completeness

Add some more PVR and SVR defines

These processors may not be supported yet, but add them for completion.

POWER9 is planned for support. e300 may work (based on 603e core).
P5040/P5021 are similar to P5020, so should work as well. One addition is
needed for P5040, to support the number of LAWs, and will be a separate commit.

Introduce 64-bit PowerPC Book-E support

Extend the Book-E pmap to support 64-bit operation. Much of this was taken from
Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory
list -- PP2D, page directory, page table), but has gaps in the page directory
list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap
between the two parts of the index). In practice this may not be a problem
given the expanded address space. However, an alternative to this would be to
use a 4-level page table, like Linux, and possibly reduce the available address
space; Linux appears to use a 46-bit address space. Alternatively, a cache of
page directory pointers could be used to keep the overall design as-is, but
remove the gaps in the address space.

This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with
the following notes:
* The DPAA driver has not yet been ported to 64-bit so is not included in the
kernel config.
* This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in
(total size kernel+mdroot must be under 64MB).
* This can run both 32-bit and 64-bit processes, and has even been tested to run
a 32-bit init with 64-bit children.

Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open
sourced to be used here, and to stevek for reviewing, and providing some
historical contexts on quirks of the code.

Reviewed by: stevek
Obtained from: Juniper (in part)
MFC after: 2 months
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D9433

Add Book-E Enhanced Debug (E.D) profile debug support

Freescale added the E.D profile to e500mc and derivative cores. From
Freescale's EREF reference manual this is enabled by a bit in HID0 and should
otherwise default to traditional debug. However, none of the Freescale cores
support that bit, and instead always use E.D. This results in kernel panics
using the standard debug on e500mc+ cores.

Enhanced debug allows debugging of interrupts, including critical interrupts,
as it uses a different save/restore registers (srr*). At this time we don't use
this ability, so instead share the core of the debug handler code between both
handlers.

MFC after: 3 weeks

Add P1022 and compatible SVR IDs

The eSDHC driver requires these IDs. Missed in r308188.

Create a new MACHINE_ARCH for Freescale PowerPC e500v2

Summary:
The Freescale e500v2 PowerPC core does not use a standard FPU.
Instead, it uses a Signal Processing Engine (SPE)--a DSP-style vector processor
unit, which doubles as a FPU. The PowerPC SPE ABI is incompatible with the
stock powerpc ABI, so a new MACHINE_ARCH was created to deal with this.
Additionaly, the SPE opcodes overlap with Altivec, so these are mutually
exclusive. Taking advantage of this fact, a new file, powerpc/booke/spe.c, was
created with the same function set as in powerpc/powerpc/altivec.c, so it
becomes effectively a drop-in replacement. setjmp/longjmp were modified to save
the upper 32-bits of the now-64-bit GPRs (upper 32-bits are only accessible by
the SPE).

Note: This does _not_ support the SPE in the e500v1, as the e500v1 SPE does not
support double-precision floating point.

Also, without a new MACHINE_ARCH it would be impossible to provide binary
packages which utilize the SPE.

Additionally, no work has been done to support ports, work is needed for this.
This also means no newer gcc can yet be used. However, gcc's powerpc support
has been refactored which would make adding a powerpcspe-freebsd target very
easy.

Test Plan:
This was lightly tested on a RouterBoard RB800 and an AmigaOne A1222
(P1022-based) board, compiled against the new ABI. Base system utilities
(/bin/sh, /bin/ls, etc) still function appropriately, the system is able to boot
multiuser.

Reviewed By: bdrewery, imp
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D5683

Print machine check address for Book-E.

Bits in mcsr indicate if the address is valid, and whether it's a physical
address or effective address.

Sponsored by: Alex Perez/Inertial Computing

Add PVR identifier for E6500, from the reference.

Add machine check register printing

This will print out the Memory Subsystem Status Register on MPC745x (G4+ class),
and the Machine Check Status Register on Book-E class CPUs, to aid in debugging
machine checks. Other relevant registers, for other CPUs, can be added in the
future.

Add a new CPU id for a POWER8 variant.

Add hwpmc(4) support for the PowerPC 970 class processors, direct events.
This also fixes asserts on removal of the module for the mpc74xx.

The PowerPC 970 processors have two different types of events: direct events
and indirect events. Thus far only direct events are supported. I included
some documentation in the driver on how indirect events work, but support is
for the future.

MFC after: 1 month

Add PMU-based CPU frequency scaling. This method is used on most Titanium
PowerBooks.

MFC after: 1 month

Add POWER7+ and POWER8 to the CPU ID table.

Approved by: re (kib)

Remove duplicate definition of SPR MMCR0.

MFC after: 3 days

On Nintendo Wii CPUs, the mdp value will be garbage. Set it to NULL
so as to not confuse things.

Submitted by: Margarida Gouveia

Let us manage differences of Book-E PowerPC variations i.e. vendor /
implementation specific vs. the common architecture definition.

Bring PPC4XX defines (PSL, SPR, TLB). Note the new definitions under
BOOKE_PPC4XX are not used in the code yet.

This change set is not supposed to affect existing E500 support, it's just
another reorg step before bringing support for E500mc, E5500 and PPC465.

Obtained from: AppliedMicro, Freescale, Semihalf

Provide SPR definitions for newer Book-E (E500mc, E5500, PPC465).

Obtained from: Freescale, Semihalf.

Unify SPR defines formatting, no funtional changes.

Implement hwpmc counting PMC support for PowerPC G4+ (MPC745x/MPC744x).
Sampling is in progress.

Approved by: nwhitehorn (mentor)
MFC after: 9.0-RELEASE

o Add system versions for the P4040(E) and P4080(E).
o In bare_probe(), change the logic that determines the maximum
number of processors/cores into a switch statement and take
advantage of the fact that bit 3 of the SVR value indicates
whether we're running on a security enabled version. Since we
don't care about that here, mask the bit. All -E versions
are taken care of automatically.

o Swap the SVR numbers for MPC8533 & MPC8533E
o Add SVR defines for P1011(E), P1020(E), P2010(E) & P2020(E)

Fix handling of NX pages on capable CPUs. Thanks to kib for prodding me
in the right direction.

Add CPU support code for the IBM Cell Broadband Engine.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

MFppc64:

Kernel sources for 64-bit PowerPC, along with build-system changes to keep
32-bit kernels compiling (build system changes for 64-bit kernels are
coming later). Existing 32-bit PowerPC kernel configurations must be
updated after this change to specify their architecture.

The NetBSD Foundation has granted permission to remove clause 3 and 4 from
their software.

Obtained from: NetBSD

MFC r198212,198378,198427,198428,198723,198724,198725,198731:
SMP support for PowerPC G5 systems.

r198724:
Fix a race in casuword() exposed by csup. casuword() non-atomically
read the current value of its argument before atomically replacing it,
which could occasionally return the wrong value on an SMP system. This
resulted in user mutex operations hanging when using threaded applications.

r198723,198725,198731:
Loop on blocked threads when using ULE scheduler, removing an
XXX MP comment.

r198427:
Add some more paranoia to setting HID registers, and update the AIM
clock routines to work better with SMP.

r198378:
Add SMP support on U3-based G5 systems. While here, correct the
64-bit tlbie function to set the CPU to 64-bit mode correctly.

r198212:
Don't assume that physical addresses are identity mapped. This
allows the second processor on G5 systems to start.

Add SMP support on U3-based G5 systems. This does not yet work perfectly:
at least on my Xserve, getting the decrementer and timebase on APs to tick
requires setting up a clock chip over I2C, which is not yet done.

While here, correct the 64-bit tlbie function to set the CPU to 64-bit
mode correctly.

Hardware donated by: grehan

Fix copy/paste typo in last revision. PMC0 control should be shifted 8
bits, not 6, on the PPC 970.

Teach cpu_est_clockrate() about the G5's slightly different PMC. This
allows the boot messages to include the CPU speed and makes possible
the forthcoming cpufreq support for the PPC 970.

Initial support for SMP on PowerPC MPC85xx.

Tested with Freescale dual-core MPC8572DS development system.

Obtained from: Freescale, Semihalf

PowerPC common SMP startup and time base rework.

- make mftb() shared, rewrite in C, provide complementary mttb()
- adjust SMP startup per the above, additional comments, minor naming
changes
- eliminate redundant TB defines, other minor cosmetics

Reviewed by: marcel, nwhitehorn
Obtained from: Freescale, Semihalf

Factor out platform dependent things unrelated to device drivers into a
new platform module. These are probed in early boot, and have the
responsibility of determining the layout of physical memory, determining
the CPU timebase frequency, and handling the zoo of SMP mechanisms
found on PowerPC.

Reviewed by: marcel, raj
Book-E parts by: raj

Rework the way we get the cacheline size. Instead of having a table of
CPUs known to use 128 byte cache lines and defaulting to 32, use the dcbz
instruction to measure it. Also make dcbz behave the way you would
expect on PPC 970.

Add support for 64-bit PowerPC CPUs operating in the 64-bit bridge mode
provided, for example, on the PowerPC 970 (G5), as well as on related CPUs
like the POWER3 and POWER4.

This also adds support for various built-in hardware found on Apple G5
hardware (e.g. the IBM CPC925 northbridge).

Reviewed by: grehan

Make MPC85xx LAW handling and reset routines aware of the MPC8548 variant.

Inspired by discussion with Alexey V Fedorov on freebsd-powerpc@.

Improve MPC85XX helper routines.

- Move CCSR accessors to the shared MPC85XX area
- Simplify SVR version subfield handling
- Adjust OCP

- Import the HEAD csup code which is the basis for the cvsmode work.

o Remove SPR_TSR & SPR_TCR for AIM.
o Remove SPR_HID2.
o Add more SPR_L3CR bit definitions.

Move System Revision defines to a bit better place, add MPC8572 systems IDs.

Rework and extend PowerPC headers definitons towards Book-E/e500 CPUs support.

Approved by: cognet (mentor)
Obtained from: Juniper, Semihalf
MFp4: e500

Teach PowerPC CPU identification routines to recognize e500 cores. Fix style
issues in this area.

Approved by: cognet (mentor)
MFp4: e500

- add definitions for MPC7447A/7448 (i.e. miniMac)
- expand MPC745X_P macro to include these

Obtained from: NetBSD

/* -> /*- for license, minor formatting changes

Definitions for MPC7457 CPU type and HID0 bits

- Update spr.h
- Add hid.h

Obtained from: NetBSD

NOTE: This undoes some changes I'd made to prefix the processor name defines
with PVR_. This was due to my original decision to use MPC750 as a cpu name.
With this changed, the PVR_ change is no longer required.

Rename the constants for the contents of the PVR register so as not to
conflict with cpu names used in config files..

Commit of stuff that's been sitting in my tree for a while.

Highlights include:
- New low-level trap code from NetBSD. The high level code still needs a lot
of work.
- Fixes for some pmap handling in thread switching.
- The kernel will now get to attempting to jump into init in user mode. There
are some pmap/trap issues which prevent it from actually getting there though.

Obtained from: NetBSD (parts)