powerpc: Implement fpu_kern_enter/fpu_kern_leave

Summary:
Provide an implementation of fpu_kern_enter/fpu_kern_leave for PPC to
enable FPU, VSX, and Altivec usage in-kernel. The functions currently
only support FPU_KERN_NOCTX, but this is sufficient for ossl(1) and many
other users of the API.

This patchset has been tested on powerpc64le using a modified version of
the in-tree tools/tools/crypto/cryptocheck.c tool to check for FPU/Vec
register clobbering along with a follow-up patch to enable ossl(4) on
powerpc64*.

Reviewed by: jhibbits
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D41540

Revert "powerpc: Implement fpu_kern_enter/fpu_kern_leave"

This reverts commit 6a47fa697ace42851b44498a53446b29b2593316.

Need to correct authorship.

powerpc: Implement fpu_kern_enter/fpu_kern_leave

Summary:
Provide an implementation of fpu_kern_enter/fpu_kern_leave for PPC to
enable FPU, VSX, and Altivec usage in-kernel. The functions currently
only support FPU_KERN_NOCTX, but this is sufficient for ossl(1) and many
other users of the API.

This patchset has been tested on powerpc64le using a modified version of
the in-tree tools/tools/crypto/cryptocheck.c tool to check for FPU/Vec
register clobbering along with a follow-up patch to enable ossl(4) on
powerpc64*.

Reviewed by: jhibbits
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D41540

sys: Remove $FreeBSD$: one-line .h pattern

Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/

[PowerPC64LE] Fix LE VSX/fpr interop

In the PCB struct, we need to match the VSX register file layout
correctly, as the VSRs shadow the FPRs.

In LE, we need to have a dword of padding before the fprs so they end up
on the correct side, as the struct may be manipulated by either the FP
routines or the VSX routines.

Additionally, when saving and restoring fprs, we need to explicitly target
the fpr union member so it gets offset correctly on LE.

Fixes weirdness with FP registers in VSX-using programs (A FPR that was
saved by the FP routines but restored by the VSX routines was becoming 0
due to being loaded to the wrong side of the VSR.)

Original patch by jhibbits.

Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D27431

powerpc: clean up empty lines in .c and .h files

[PPC] Fix wrong comment

pcb_context[20] holds r12-r31 and not r14-r31, as the comment said.

powerpc: Add support for additional FSCR-managed facilities

Add support to enable, save, and restore the following facilities:
* Target Address Register (bctar) -- seemingly just another register to
branch to.
* Event-based branching -- an interrupt-like userspace event handler
subsystem.
* Load-monitored facility -- A facility that allows monitoring a range of
physical memory, and triggering an event on access. Targeted to garbage
collection software features.

powerpc64: Add the DSCR facility on POWER8 and later

The Data Stream Control Register (DSCR) is privileged on POWER7, but
unprivileged (different register) on POWER8 and later. However, it's now
guarded by a new register, the Facility Status and Control Register, instead of
the MSR like other pre-existing facilities (FPU, Altivec). The FSCR must be
managed explicitly, since it's effectively an extension of the MSR.

Tested by: Brandon Bergren

powerpc: Sync icache on SIGILL, in case of cache issues

The update of jemalloc to 5.1.0 exposed a cache syncing issue on a Freescale
e500 base system. There was already code in the FPU emulator to address
this, but it was limited to a single static variable, and did not attempt to
sync the cache. This pulls that out to the higher level program exception
handler, and syncs the cache.

If a SIGILL is hit a second time at the same address, it will be treated as
a real illegal instruction, and handled accordingly.

powerpc64: Add initial support for HTM (kABI)

This patch adds the very initial support for HTM that might come at FreeBSD
version 12.1. This basic support defines a new kABI, so, we do not need to change
it later during 12.1 time frame, when the full implementation will come.

Reviewed by: jhibbits
Approved by: re(marius), jhibbits (mentor)
Differential Revision: https://reviews.freebsd.org/D16889

powerpc64: Add DSCR support

Summary:
Powerpc64 has support for a register called Data Stream Control Register
(DSCR), which basically controls how the hardware controls the caching and
prefetch for stream operations.

Since mfdscr and mtdscr are privileged instructions, we need to emulate them,
and
keep the custom DSCR configuration per thread.

The purpose of this feature is to change DSCR depending on the operation, set
to DSCR Default Prefetch Depth to deepest on string operations, as memcpy.

Submitted by: Breno Leitao
Differential Revision: https://reviews.freebsd.org/D15081

Do not include float interfaces when using libsa.

We don't support float in the boot loaders, so don't include
interfaces for float or double in systems headers. In addition, take
the unusual step of spiking double and float to prevent any more
accidental seepage.

sys: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Use setjmp() instead of the identical-except-for-having-a-wrong-prototype
setfault() when testing for faults. This should also help the compiler
do the right thing with this complicated-to-optimize function.

Kernel support for the Vector-Scalar eXtension (VSX) found on the POWER7
and POWER8. This instruction set unifies the 32 64-bit scalar floating
point registers with the 32 128-bit vector registers into a single bank
of 64 128-bit registers. Kernel support mostly amounts to saving and
restoring the wider version of the floating point registers and making
sure that both scalar FP and vector registers are enabled once a VSX
instruction is executed. get_mcontext() and friends currently cannot
see the high bits, which will require a little more work.

As the system compiler (GCC 4.2) does not support VSX, making use of this
from userland requires either newer GCC or clang.

Relnotes: yes
Sponsored by: FreeBSD Foundation

Resort and resize the altivec registers in the pcb. vrsave and vscr are both
32-bit registers via the PowerPC spec.

X-MFC-with: r276634
MFC after: 2 weeks

Split the function of the PCB_FPU flags into two: PCB_FPU now indicates that
the actual FPU is enabled, while PCB_FPREGS indicates that the FPU state
structure in the PCB is valid. This separation reflects the situation on
FPU-less systems in which the FP state is used by the emulator but we don't
actually want to try to turn on the non-existant FPU.

Use this flag to save and restore FP regs properly on both AIM and Book-E.
As a side effect, this sets up hard-FP and Altivec on Book-E CPUs with such
abilities except for a trap handler to call enable_fpu()/enable_altivec().

There is no reason Book-E needs to save XER and CTR on context switches.
They aren't Book-E specific registers to begin with and, even if they were,
are defined volatile by the ABI.

Move the prototype for savectx from cpu.h to pcb.h, as it is on other
platforms, as well as putting it in an #ifdef KERNEL block.

MFC after: 2 weeks

Restructure the way the copyin/copyout segment is stored to prevent a
concurrency bug. Since all SLB/SR entries were invalidated during an
exception, a decrementer exception could cause the user segment to be
invalidated during a copyin()/copyout() without a thread switch that
would cause it to be restored from the PCB, potentially causing the
operation to continue on invalid memory. This is now handled by explicit
restoration of segment 12 from the PCB on 32-bit systems and a check in
the Data Segment Exception handler on 64-bit.

While here, cause copyin()/copyout() to check whether the requested
user segment is already installed, saving some pipeline flushes, and
fix the synchronization primitives around the mtsr and slbmte
instructions to prevent accessing stale segments.

MFC after: 2 weeks

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

Handle vector assist traps without a kernel panic, by setting denormalized
values to zero. A correct solution would involve emulating vector
operations on denormalized values, but this has little effect on accuracy
and is much less complicated for now.

MFC after: 2 weeks

Replace the SLB backing store splay tree used on 64-bit PowerPC AIM
hardware with a lockless sparse tree design. This marginally improves
the performance of PMAP and allows copyin()/copyout() to run without
acquiring locks when used on wired mappings.

Submitted by: mdf

MFppc64:

Kernel sources for 64-bit PowerPC, along with build-system changes to keep
32-bit kernels compiling (build system changes for 64-bit kernels are
coming later). Existing 32-bit PowerPC kernel configurations must be
updated after this change to specify their architecture.

Make Book-E debug register state part of the PCB context.

Previously, DBCR0 flags were set "globally", but this leads to problems
because Book-E fine grained debug settings work only in conjuction with the
debug master enable bit in MSR: in scenarios when the DBCR0 was set with
intention to debug one process, but another one with MSR[DE] set got
scheduled, the latter would immediately cause debug exceptions to occur upon
execution of its own code instructions (and not the one intended for
debugging).

To avoid such problems and properly handle debugging context, DBCR0 state
should be managed individually per process.

Submitted by: Grzegorz Bernacki gjb ! semihalf dot com
Reviewed by: marcel

Add Altivec support for supported CPUs. This is derived from the FPU support
code, and also reducing the size of trapcode to fit inside a 32 byte handler
slot.

Reviewed by: grehan
MFC after: 2 weeks

- Import the HEAD csup code which is the basis for the cvsmode work.

Unify and generalize PowerPC headers, adjust AIM code accordingly.

Rework of this area is a pre-requirement for importing e500 support (and
other PowerPC core variations in the future). Mainly the following
headers are refactored so that we can cover for low-level differences between
various machines within PowerPC architecture:

<machine/pcpu.h>
<machine/pcb.h>
<machine/kdb.h>
<machine/hid.h>
<machine/frame.h>

Areas which use the above are adjusted and cleaned up.

Credits for this rework go to marcel@

Approved by: cognet (mentor)
MFp4: e500

Forward declare struct trapframe.

Add prototype for KDB's makectx routine

Add the USER_SR segment register to pcb state. Initialize correctly,
and save/restore during a context switch.

The USER_SR could be overwritten when the current thread was switched
out with a faulting copyin/copyout.

Approved by: Benno

The a.out md_coredump stuff isn't referenced anywhere anymore, and
hasn't been filled in for ages.. Nuked.

To quote Peter:

The case in cpu_switch() where there isn't a higher priority thread
(choosethread() == curthread) uses r4 as the PCB context pointer. However, the
use of r4 after the label L2 is incorrect, since it was probably trashed by
the call to choosethread, and in any case was set up to curthread at the start
of the routine.

This condition will occur when an interrupt thread schedules a netisr, which
is a lower priority thread.

Another (probably unnecessary) difference is that I was paranoid about
register trashing, so I decided to save r2 and r13 as well.

Submitted by: Peter Grehan <peterg@ptree32.com.au>

FPU support.

Obtained from: NetBSD (portions)

Make fork work, at least for kthreads. Switching still has some issues.

Save WIP. Partial rewrite of cpu_switch() and savectx(). This makes it closer
to working but still needs some work to properly switch the full context
(such as saving the fpu registers, switch stacks, etc.). Also, remove some
dead code that was mixed in.

Bring in NetBSD code used in the PowerPC port.

Reviewed by: obrien, dfr
Obtained from: NetBSD