sys: Remove $FreeBSD$: two-line .h pattern

Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/

Add CFI start/end proc directives to arm64, i386, and ppc

Follow-up to r353959 and r368070: do the same for other architectures.

arm32 already seems to use its own .fnstart/.fnend directives, which
appear to be ARM-specific variants of the same thing. Likewise, MIPS
uses .frame directives.

Reviewed by: arichardson
Differential Revision: https://reviews.freebsd.org/D27387

powerpc/booke: Fix 32-bit Book-E SMP AP bringup

r354266 changed the type of bp_kernload to vm_paddr_t in platform_mpc85xx.c,
but not the variable itself in locore.S. This caused the AP to not come up,
due to overwriting the following variable (bp_virtaddr). Also, properly
load bp_kernload into MAS3 and MAS7. Prior to r354266, we required loading
into the low 4GB, but now we can load from anywhere in memory that ubldr can
access.

Add support for building Book-E kernels with clang/lld.

This involved several changes:

* Since lld does not like text relocations, replace SMP boot page text relocs
in booke/locore.S with position-independent math, and track the virtual base
in the SMP boot page header.

* As some SPRs are interpreted differently on clang due to the way it handles
platform-specific SPRs, switch m*dear and m*esr mnemonics out for regular
m*spr. Add both forms of SPR_DEAR to spr.h so the correct encoding is selected.

* Change some hardcoded 32 bit things in the boot page to be pointer-sized, and
fix alignment.

* Fix 64-bit build of booke/pmap.c when enabling pmap debugging.

Additionally, I took the opportunity to document how the SMP boot page works.

Approved by: jhibbits (mentor)
Differential Revision: https://reviews.freebsd.org/D21999

powerpc/booke: Don't zero MAS8, it's unnecessary

MAS8 is hypervisor privileged, defining the logical partition (VM) to
operate on for TLB accesses. It's already guaranteed to be cleared when
booting bare metal (bootloader needs it zeroed to work), and we can't touch
it from a guest. Assume that if/when we eventually port bhyve to PowerPC
(and Book-E) the hypervisor module will take care of managing MAS8. This
saves several (tens) of clocks on each TLB miss.

MFC after: 2 weeks

powerpc/booke64: Align initial stack setting to match that of aim64's

Clang9/LLD9 appears to get quite confused with the instruction stream used
to obtain the tmpstack pointer, almost as though it thinks this is a C
function, so tries to optimize it. Since the AIM64 method doesn't use the
TOC to obtain the tmpstack, just follow that model, and lld won't get
confused.

Reported by: bdragon
MFC after: 2 weeks

powerpc: Stop pretending we run on e500v1 cores

Unconditional writing to MAS7, which doesn't exist on the e500v1 core, in a
TLB miss handler has been in the code for several years now. Since this has
gone unnoticed for so long, it's easily concluded that e500v1 is not in use
with FreeBSD. Simplify the code path a bit, by unconditionally zeroing MAS7
instead of calling a subroutine to do it.

powerpc64: Use medium code model in asm files for TOC references

Summary:
With a sufficiently large TOC, it's possible to index out of range, as
the immediate load instructions only permit 16-bit indices, allowing up
to 64kB range (signed) from the base pointer. Allow +/- 2GB range, with
the medium code model TOC accesses in asm.

Patch originally by Brandon Bergren. The issue appears to impact ELFv2
more than ELFv1.

Reviewed by: luporl
Differential Revision: https://reviews.freebsd.org/D19708

powerpc/booke: Depessimize MAS register updates even more

Remove isyncs between MAS register updates in the TLB miss handler, since
it's only needed before the TLB update instructions.

powerpc/booke: Don't get and use the load offset for TOC on APs

The code was a near exact copy of the code in startup, but it doesn't need
the complexity since the kernel is already relocated. With
VM_MIN_KERNEL_ADDRESS as currently set to KERNBASE, this doesn't cause a
problem, because it's a zero offset. However, when KERNBASE is changed to a
physical load address, it then has a non-zero offset, and ends up with an
invalid stack pointer, causing the AP to hang.

powerpc: Prepare Book-E kernels for KERNBASE != run base

Book-E kernels really run at VM_MIN_KERNEL_ADDRESS, which currently happens to
be the same as KERNBASE. KERNBASE is the linked address, which the loader also
takes to be the physical load address. Treat KERNBASE as a physical address,
not a virtual, and change virtual address references for KERNBASE to use
something more appropriate.

Remove dead errata fixup code

This code caused more problems than it should have fixed (boot failures) on
the machines I tested, so has been commented out for a while now. Remove
it, and assume the errata fixups were done by the bootloader where they
belong.

Rename assym.s to assym.inc

assym is only to be included by other .s files, and should never
actually be assembled by itself.

Reviewed by: imp, bdrewery (earlier)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D14180

kernel: Fix several typos and minor errors

- duplicate words
- typos
- references to old versions of FreeBSD

Reviewed by: imp, benno

Fix debug interrupts on 64-bit Book-E

Use a WORD_SIZE macro to define the correct offset to the second word
needed. This corrects the offset calculation in 64-bit builds.

Introduce 64-bit PowerPC Book-E support

Extend the Book-E pmap to support 64-bit operation. Much of this was taken from
Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory
list -- PP2D, page directory, page table), but has gaps in the page directory
list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap
between the two parts of the index). In practice this may not be a problem
given the expanded address space. However, an alternative to this would be to
use a 4-level page table, like Linux, and possibly reduce the available address
space; Linux appears to use a 46-bit address space. Alternatively, a cache of
page directory pointers could be used to keep the overall design as-is, but
remove the gaps in the address space.

This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with
the following notes:
* The DPAA driver has not yet been ported to 64-bit so is not included in the
kernel config.
* This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in
(total size kernel+mdroot must be under 64MB).
* This can run both 32-bit and 64-bit processes, and has even been tested to run
a 32-bit init with 64-bit children.

Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open
sourced to be used here, and to stevek for reviewing, and providing some
historical contexts on quirks of the code.

Reviewed by: stevek
Obtained from: Juniper (in part)
MFC after: 2 months
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D9433

Skip HID1 initialization on e6500 cores, it doesn't exist.

With this, and some drivers removed, a T2080 dev board boots to mountroot.

Submitted by: Ivan Krivonos <int0dster_AT_gmail.com>

Add a kdb show command to print arbitrary SPRs on PowerPC

Summary:
There is often a need at the debugger to print arbitrary special
purpose registers (SPRs) on PowerPC. Using a rewritable asm stub, print any SPR
provided on the command line.

Note, as there is no checking in this, attempting to print a nonexistent SPR
may cause a Program exception (illegal instruction, or boundedly undefined).

Note also that this relies on the kernel text pages being writable. If in the
future this is made not the case, this will need to be reworked.

Test Plan:
Printing the Processor Version Register (PVR, SPR 287):

db> show spr 11f
SPR 287(11f): 80240012

Differential Revision: https://reviews.freebsd.org/D7403

Add ePAPR boot support for PowerPC book-E (MPC85xx) hardware

Summary:
u-boot, following the ePAPR specification, puts secondary cores into a
spinloop at boot, rather than leaving them shut off. It then relies on the host
OS to write the correct values to a special spin table, located in coherent
memory (on newer implementations), or noncoherent memory (older
implementations).

This supports both implementations of ePAPR, as well as continuing to support
non-ePAPR booting, by first attempting to use the spintable, and falling back to
expecting non-started CPUs.

Test Plan:
Booted on a P5020 board. Tested before and after the changes.
Before the changes, prints the error "SMP: CPU 1 already out of hold-off state!"
and panics shortly thereafter. After the changes, same boot method lets it
complete boot.

Reviewed by: nwhitehorn
MFC after: 2 weeks
Relnotes: Yes
Sponsored by: Alex Perez/Inertial Computing
Differential Revision: https://reviews.freebsd.org/D7494

Use label math instead of hard-coding offsets for return addresses.

Though the chances of the code in these sections changing are low, future-proof
the sections and use label math.

Renumber the surrounding areas to avoid duplicate label numbers.

Fix SMP booting for PowerPC Book-E

Summary:
PowerPC Book-E SMP is currently broken for unknown reasons. Pull in
Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP.

This eliminates the shared software TLB1 table, replacing it with
tlb1_read_entry() function.

This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs
already released (ePAPR boot releases APs to a spin loop waiting on a specific
address). This will be addressed in the near future by using the MPIC to reset
the AP into our own alternate boot address.

This does include a change to the dpaa/dtsec(4) driver, to mark the portals as
CPU-private.

Test Plan:
Tested on Amiga X5000/20 (P5020). Boots, prints the following
messages:

Adding CPU 0, pir=0, awake=1
Waking up CPU 1 (dev=1)
Adding CPU 1, pir=20, awake=1
SMP: AP CPU #1 launched

top(1) shows CPU1 active.

Obtained from: Semihalf
Relnotes: Yes
Differential Revision: https://reviews.freebsd.org/D5945

Remove dead code and dead comments, most notably the implemenation of the
now-obsolete setfault(). No NetBSD code exists in the AIM locore files, so
update the copyrights there.

Add platform support for QorIQ SoCs.

This includes the following changes:
* SMP kickoff for QorIQ (tested on P5020)
* Errata fixes for some silicon revisions
* Enables L2 (and L3 if available) caches
Obtained from: Semihalf
Sponsored by: Alex Perez/Inertial Computing

Rewrite tid_flush() in C.

There's no need for it to be in asm. Also, by writing in C, and marking it
static in pmap.c, it saves a branch to the function itself, as it's only used in
one location. The generated asm is virtually identical to the handwritten code.

Add initial boot support for e500mc and e5500.

* Since r257190 the kernel must actually be loaded at a 64MB boundary, not 16MB.
* Don't program HID1 register on e500mc or e5500, they don't have this SPR.
* Set proper HID0 defaults for these new architectures.

There is still more work to be done for the various SoCs, and the PMAP code
still needs to be extended to 36-bit paddr, coming soon.

Obtained from: Semihalf
Sponsored by: Alex Perez/Inertial Computing

Fix copy&paste.

Save the registers at the correct offsets.

When merging the AIM and BookE trap.c files, the offsets for BookE's setfault
inadvertantly got munged.

Implement hwpmc(4) for Freescale e500 core.

This supports e500v1, e500v2, and e500mc. Tested only on e500v2, but the
performance counters are identical across all, with e500mc having some
additional events.

Relnotes: Yes

Unbreak book-e, broken by the trap.c merge (missed this file).

Make 32-bit PowerPC kernels, like 64-bit PowerPC kernels, position-independent
executables. The goal here, not yet accomplished, is to let the e500 kernel
run under QEMU by setting KERNBASE to something that fits in low memory and
then having the kernel relocate itself at runtime.

Move IVOR setup from assembler to C, decreasing required assumptions about
address formats for trap handlers.

Refactor PowerPC (especially AIM) init sequence to be less baroque.

MFC after: 2 months

Follow up r223485, which made AIM use the ABI thread pointer instead of
PCPU fields for curthread, by doing the same to Book-E. This closes
some potential races switching between CPUs. As a side effect, it turns out
the AIM and Book-E swtch.S implementations were the same to within a few
registers, so move that to powerpc/powerpc.

MFC after: 3 months

Bump initial TLB size. The kernel is not necessarily less than 16 MB any
more.

1. Have the APs initialize the TLB1 entries from what has been
programmed on the BSP during (early) boot. This makes sure
that the APs get configured the same as the BSP, irrspective
of how FreeBSD was loaded.
2. Make sure to flush the dcache after writing the TLB1 entries
to the boot page. The APs aren't part of the coherency domain
just yet.
3. Set pmap_bootstrapped after calling pmap_bootstrap(). The
FDT code now maps the devices (like OF), and this resulted
in a panic.
4. Since we pre-wire the CCSR, make sure not to map chunks of
it in pmap_mapdev().

Let us manage differences of Book-E PowerPC variations i.e. vendor /
implementation specific vs. the common architecture definition.

Bring PPC4XX defines (PSL, SPR, TLB). Note the new definitions under
BOOKE_PPC4XX are not used in the code yet.

This change set is not supposed to affect existing E500 support, it's just
another reorg step before bringing support for E500mc, E5500 and PPC465.

Obtained from: AppliedMicro, Freescale, Semihalf

o Rename kernload_ap to bp_kernelload. This to introduce a common prefix
for variables that live in the boot page.
o Add bp_trace (yes, it's in the boot page) that gets zeroed before we
try to wake a core and to which the core being woken can write markers
so that we know where the core was in case it doesn't wake up. The
boot code does not yet write markers (too follow).
o Disable the boot page translation to allow the last 4K page to be used
for whatever we please. It would get mapped otherwise.
o Fix kernstart in the case of SMP. The start argument is typically page
aligned due to the alignment requirements that come with having a boot
page. The point of using trunc_page is that we get the actual load
address given that the entry point is immediately following the ELF
headers. In the SMP case this ended up exactly 4K after the load
address. Hence subtracting 1 from start.

It's invalid to use GLOBAL() for kernload_ap, as the macro switches
to the .data section. We need kernload_ap in the boot page.

Approved by: re (blanket)

There's no ':' after GLOBAL(). Missed due to no SMP testing.

Approved by: re (blanket)

Add support for Juniper's loader. The difference between FreeBSD's and
Juniper's loader is that Juniper's loader maps all of the kernel and
preloaded modules at the right virtual address before jumping into the
kernel. FreeBSD's loader simply maps 16MB using the physical address
and expects the kernel to jump through hoops to relocate itself to
it's virtual address. The problem with the FreeBSD loader's approach is
that it typically maps too much or too little. There's no harm if it's
too much (other than wasting space), but if it's too little then the
kernel will simply not boot, because the first thing the kernel needs
is the bootinfo structure, which is never mapped in that case. The page
fault that early is fatal.

The changes constitute:
1. Do not remap the kernel in locore.S. We're mapped where we need to
be so we can pretty much call into C code after setting up the
stack.
2. With kernload and kernload_ap not set in locore.S, we need to set
them in pmap.c: kernload gets defined when we preserve the TLB1.
Here we also determine the size of the kernel mapped. kernload_ap
is set first thing in the pmap_bootstrap() method.
3. Fix tlb1_map_region() and its use to properly externd the mapped
kernel size to include low-level data structures.

Approved by: re (blanket)
Obtained from: Juniper Networks, Inc

Fix r224187: .word defines a 16-bit object and size_t is defined as
a 32-bit intergal. Use .long to define sintrcnt and sintrname.

Approved by: re (blanket)

- Remove the eintrcnt/eintrnames usage and introduce the concept of
sintrcnt/sintrnames which are symbols containing the size of the 2
tables.
- For amd64/i386 remove the storage of intr* stuff from assembly files.
This area can be widely improved by applying the same to other
architectures and likely finding an unified approach among them and
move the whole code to be MI. More work in this area is expected to
happen fairly soon.

No MFC is previewed for this patch.

Tested by: pluknet
Reviewed by: jhb
Approved by: re (kib)

Better support different kernel hand-offs. When loaded directly
from U-Boot, the kernel is passed a standard argc/argv pair.
The Juniper loader passes the metadata pointer as the second
argument and passes 0 in the first. The FreeBSD loader passes
the metadata pointer in the first argument.

As such, have locore preserve the first 2 arguments in registers
r30 & r31. Change e500_init() to accept these arguments. Don't
pass global offsets (i.e. kernel_text and _end) as arguments to
e500_init(). We can reference those directly.

Rename e500_init() to booke_init() now that we're changing the
prototype.

In booke_init(), "decode" arg1 and arg2 to obtain the metadata
pointer correctly. For the U-Boot case, clear SBSS and BSS and
bank on having a static FDT for now. This allows loading the
ELF kernel and jumping to the entry point without trampoline.

Wire the kernel using TLB1 entry 0 rather than entry 1. A more recent
U-Boot as found on the P1020RDB doesn't like it when we use entry 1
(for some reason) whereas an older U-Boot doesn't mind if we use entry
0. If anything else, this simplifies the code a bit.

Remove unused includes of <sys/mutex.h> and <machine/mutex.h>.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

MFppc64:

Kernel sources for 64-bit PowerPC, along with build-system changes to keep
32-bit kernels compiling (build system changes for 64-bit kernels are
coming later). Existing 32-bit PowerPC kernel configurations must be
updated after this change to specify their architecture.

Convert Freescale PowerPC platforms to FDT convention.

The following systems are affected:

- MPC8555CDS
- MPC8572DS

This overhaul covers the following major changes:

- All integrated peripherals drivers for Freescale MPC85XX SoC, which are
currently in the FreeBSD source tree are reworked and adjusted so they
derive config data out of the device tree blob (instead of hard coded /
tabelarized values).

- This includes: LBC, PCI / PCI-Express, I2C, DS1553, OpenPIC, TSEC, SEC,
QUICC, UART, CFI.

- Thanks to the common FDT infrastrucutre (fdtbus, simplebus) we retire
ocpbus(4) driver, which was based on hard-coded config data.

Note that world for these platforms has to be built WITH_FDT.

Reviewed by: imp
Sponsored by: The FreeBSD Foundation

Improve style(9), clean up.

Initial support for SMP on PowerPC MPC85xx.

Tested with Freescale dual-core MPC8572DS development system.

Obtained from: Freescale, Semihalf

Centralize setting HID0/1 for E500. Rename HID defines which are specific
to E500 rather than shared within Book-E family.

Obtained from: Freescale, Semihalf

Rework BookE pmap towards multi-core support.

o Eliminate tlb0[] (a s/w copy of TLB0)
- The table contents cannot be maintained reliably in multiple MMU
environments, where asynchronous events (invalidations from other cores)
can change our local TLB0 contents underneath.
- Simplify and optimize TLB flushing: system wide invalidations are
performed using tlbivax instruction (propagates to other cores), for
local MMU invalidations a new optimized routine (assembly) is introduced.

o Improve and simplify TID allocation and management.
- Let each core keep track of its TID allocations.
- Simplify TID recycling, eliminate dead code.
- Drop the now unused powerpc/booke/support.S file.

o Improve page tables management logic.

o Simplify TLB1 manipulation routines.

o Other improvements and polishing.

Obtained from: Freescale, Semihalf

Minor spelling fix in E500 locore.

Fix E500 cache invalidation routines.

When invalidating the i/d-cache we need to wait until the core complex is
really finished with the operation.

Obtained from: Semihalf

Rework E500 locore.

- split bootstrap code into more modular routines, which will also be used for
the non-booting cores
- clean up registers usage
- improve comments to better reflect reality
- eliminate dead or redundant code
- other minor fixes

This refactoring is a preliminary step before importing dual-core (MPC8572)
support.

Obtained from: Freescale, Semihalf

Add support for kernel profiling for both AIM and BookE.

Obtained from: Juniper Networks, Inc (BookE support).

- Import the HEAD csup code which is the basis for the cvsmode work.

Improve kernel stack handling on e500.

- Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page
- Switch to thread0.td_kstack as soon as possible i.e. right after return
from e500_init() and before mi_startup() happens
- Clean up temp stack area
- Other minor cosmetics in machdep.c

Obtained from: Semihalf

Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family.

The PQ3 is a high performance integrated communications processing system
based on the e500 core, which is an embedded RISC processor that implements
the 32-bit Book E definition of the PowerPC architecture. For details refer
to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E

This port was tested and successfully run on the following members of the PQ3
family: MPC8533, MPC8541, MPC8548, MPC8555.

The following major integrated peripherals are supported:

* On-chip peripherals bus
* OpenPIC interrupt controller
* UART
* Ethernet (TSEC)
* Host/PCI bridge
* QUICC engine (SCC functionality)

This commit brings the main functionality and will be followed by individual
drivers that are logically separate from this base.

Approved by: cognet (mentor)
Obtained from: Juniper, Semihalf
MFp4: e500