MFC r328467:

Implement mitigation for Spectre version 2 attacks on ARMv7.

Revert r330897:

This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.

Revert with prejudice.

This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.

Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.

Requested by: gjb (re)

Partial merge of the SPDX changes

These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.

No objections from: pfg

MFC r327222:

Add a new ARM kernel option, LOCORE_MAP_MB, to control the size of the
kernel VA mapping in the temporary page tables set up by locore-v6.S.

The number used to be hard-coded to 64MB, which is still the default if
the kernel option is not specified. However, 64MB is insufficient for
using a large mdroot filesystem. The hard-coded number can't be safely
increased because too large a number may run into memory-mapped IO space
on some SoCs that must not be mapped as ordinary memory.

MFC r327367:

Make kernel option KERNVIRTADDR optional, remove it from std.<platform>
files that can use the default value.

It used to be required that the low-order bits of KERNVIRTADDR matched
the low-order bits of the physical load address for all arm platforms.
That hasn't been a requirement for armv6 platforms since FreeBSD 10.
There is no longer any relationship between load addr and KERNVIRTADDR
except that both must be aligned to a 2 MiB boundary.

This change makes the default KERNVIRTADDR value 0xc0000000, and removes the
options from all the platforms that can use the default value. The default
is now defined in vmparam.h, and that file is now included in a few new
places that reference KERNVIRTADDR, since it may not come in via the
forced-include of opt_global.h on the compile command line.

MFC r306631,r306640,r306641,r306650,r306656:

r306631:
Use C99 designated initializers to create the armv6 cpu_functions structs.
This will help with a later cleanup of what functions we implement.
r306640:
Only define the CF_* macros on ARMv4/v5. They are unused on armv6.
r306641:
Remove the parts of cpu_functions from armv6 that are unused on that
architecture.
r306650:
Add the Cortex-A{53,57,72} ID register values. These can all run 32-bit
code so could run a 32-bit kernel.
r306656:
Use the cortex functions when booting on one of the Cortex-A ARMv8 CPUs.
This list is incomplete, however we don't have the ID values for the
missing Cortex-A32 or A35.

MFC r306162:

Make it possible to safely use TPIDRURW from userspace.

On amd64, arm64 and i386, we have the possibility to switch between TLS
areas in userspace. The nice thing about this is that it makes it easier
to do light-weight threading, if we ever feel like doing that. On armv6,
let's go into the same direction by making it possible to safely use the
TPIDRURW register, which is intended for this purpose.

Clean up the ARMv6 code to remove md_tp entirely. Simply add a dedicated
field to the PCB to hold the value of TPIDRURW across context switches,
like we do for any other register. As userspace currently uses the
read-only TPIDRURO register, simply ensure that we keep both values in
sync where possible. The system calls for modifying the read-only
register will simply write the intended value into both registers, so
that it lazily ends up in the PCB during the next context switch.

Approved by: andrew
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D7951

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Remove the ARMv4/ARMv5 userland atomic support from struct proc on armv6.
Nothing should use this on armv6 as we use the atomic instructions added in
ARMv6k.

Sponsored by: ABT Systems Ltd

Include machine/acle-compat.h in cdefs.h on arm if the compiler doesn't
have ACLE support built in. The ACLE (ARM C Language Extensions) defines
a set of standardized symbols which indicate the architecture version and
features available. ACLE support is built in to modern compilers (both
clang and gcc), but absent from gcc prior to 4.4.

ARM (the company) provides the acle-compat.h header file to define the
right symbols for older versions of gcc. Basically, acle-compat.h does
for arm about the same thing cdefs.h does for freebsd: defines
standardized macros that work no matter which compiler you use. If ARM
hadn't provided this file we would have ended up with a big #ifdef __arm__
section in cdefs.h with our own compatibility shims.

Remove #include <machine/acle-compat.h> from the zillion other places (an
ever-growing list) that it appears. Since style(9) requires sys/types.h
or sys/param.h early in the include list, and both of those lead to
including cdefs.h, only a couple special cases still need to include
acle-compat.h directly.

Loves it: imp

Fix <sys/_bitset.h> and <sys/_cpuset.h> to not require <sys/param.h>.

- Hardcode '8' instead of NBBY in _BITSET_BITS.
- Define a private version of 'howmany' for use in __bitset_words().
- While here, move a few more things out of _bitset.h and _cpuset.h to
bitset.h and cpuset.h, respectively. The only things left in
_bitset.h and _cpuset.h are the bits needed to define a bitset
structure.

Reviewed by: bde, kib (ish)

As <machine/vmparam.h> is included from <vm/vm_param.h>, there is no
need to include it explicitly when <vm/vm_param.h> is already included.

Suggested by: alc
Reviewed by: alc
Differential Revision: https://reviews.freebsd.org/D5379

Do not use PMAP_DOMAIN_KERNEL definition for __ARM_ARCH >= 6 as domains
are not utilized there. Only domain #0 is used and there is no reference
to it in the whole pmap-v6.c. Thus initialize domain access register in
locore-v6.c without reference too.

ARM: Introduce new cpu-v4.h header and move all ARMv4 specific code
from cpu-v6.h to it.
Remove unneeded cpu-v6.h includes.

ARM: Remove unused symbols from genassym.c.

ARM: Replace only once used cpu_icache_sync_all() by ranged equivalent.
Remove it from cpu_functions table.

Remove not needed <machine/pte.h> includes.

ARM: remove old pmap-v6 code. The new pmap-v6 is mature enough, and
dual implementation is showstopper for major cleanup.

This patch only removes old code from tree. Cleanups will follow asap.

Move the check to see if we are tracing a function with the DTrace Function
Boundary Trace to assembly to reduce the overhead of these checks.

Submitted by: Howard Su <howard0su@gmail.com>
Relnotes: Yes
Differential Revision: https://reviews.freebsd.org/D4266

ARM userspace accessors, e.g. {s,f}uword(9), copy{in,out}(9),
casuword(9) and others, use LDRT and STRT instructions to access
memory with the privileges of userspace. If the *RT instruction
faults on the kernel address, then additional checks must be done to
not confuse the VM system with invalid kernel-mode faults.

Put ARM on line with other FreeBSD architectures and disallow usermode
buffers which intersect with the kernel address space in advance,
before any accesses are performed. In other words, vm_fault(9) is no
longer called when e.g. suword(9) stores to invalid (i.e. not
userspace) address.

Also, switch ARM to use fueword(9) and casueword(9).

Note: there is a pending patch in D3617, which adds the special
processing for faults from LDRT and STRT. The addition of the
processing is useful for potential other uses of the instructions and
for completeness, but standard userspace accessors are better served
by not allowing such faults beforehand.

Reviewed by: andrew
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3816
MFC after: 2 weeks

pc_curpmap is only in the armv6 pcpu data.

Fix a misplaced #endif (maybe a mismerge?). Emitting the symbol for CURPMAP
is not dependent on whether VFP (hardware floating point) is enabled.

Stop checking for ARM_TP_ADDRESS when we mean to check if building for
ARMv6 or later.

Remove pc_cpu, it was duplicating pc_cpuid so was unneeded.

New pmap code for armv6. Disabled by default, option ARM_NEW_PMAP enables it.

This is pretty much a complete rewrite based on the existing i386 code. The
patches have been circulating for a couple years and have been looked at by
plenty of people, but I'm not putting anybody on the hook as having reviewed
this in any formal sense except myself.

After this has gotten wider testing from the user community, ARM_NEW_PMAP
will become the default and various dregs of the old pmap code will be
removed.

Submitted by: Svatopluk Kraus <onwahe@gmail.com>,
Michal Meloun <meloun@miracle.cz>

Add minimum cache line sizes to struct cpuinfo, use them in the new cache
maintenance routines. Also add a routine to invalidate the branch cache.

Submitted by: Michal Meloun

Export MAXCPU to the assembler code, needing by upcoming changes to locore.S.

Cleanup up ARM *frame structures...

- Eliminate unused irqframe
- Eliminate unused saframe
- Instead of splitting r4-sp storage between the stack and switchframe,
just put all the registers in switchframe and eliminate the un_32 struct.

Submitted by: Svatopluk Kraus <onwahe@gmail.com>,
Michal Meloun <meloun@miracle.cz>

Remove all dregs of a per-thread undefined-exception-mode stack. This is
a leftover from the days when a low-level debugger had hooks in the
undefined exception vector and needed stack space to function. These days
it effectively isn't used because we switch immediately to the svc32 mode
stack on exception entry. For that, the single undef mode stack per core
that gets set up at init time works fine.

The stack wasn't necessary but it was harmful, because the space for it
was carved out of the normal per-thread svc32 stack, in effect cutting
that 8K stack in half. If svc32 mode used more than 4k of stack space it
wandered down into the undef mode stack, and then an undef exception would
overwrite a couple words on the stack while switching to svc32 mode,
corrupting the scv32 stack. Having another stack abut the bottom of the
svc32 stack also effectively mooted the guard page below the stack.

This work is based on analysis and patches submitted by Juergen Weiss.

Rework the VFP code that handles demand-based save and restore of state.

The old code was full of complexity that would only matter if the
kernel itself used the VFP hardware. Now that's reduced to either killing
the userland process or panicking the kernel on an illegal VFP instruction.

This removes most of the complexity from the assembler code, reducing it
to just calling the save code if the outgoing thread used the VFP.

The routine that stores the VFP state now takes a flag that indicates
whether the hardware should be disabled after saving state. Right now it
always is, but this makes the code ready to be used by get/set_mcontext()
(doing so will be addressed in a future commit).

Remove the arm-specific pc_vfpcthread from struct pcpu and use the MI
field pc_fpcurthread instead.

Reviewed by: cognet

Fix the arm sys_sigreturn(): its argument is a struct ucontext, not a
struct sigframe containing the struct ucontext.

The signal trampoline return code on the other hand DOES have just a
struct sigframe on the stack to work with, so have it get a pointer to
the ucontext out of there to pass along to sys_sigreturn.

In other words, make everything work right whether sys_sigreturn is
invoked from the trampoline or from userland code calling sigreturn(2).

Submitted by: Takashi Komatsu <komatsu.taka@jp.panasonic.com>
Reviewed by: cognet

Change the way pcpu and curthread are stored per-core:
the old way was to store pcpu in a register, and get curthread from pcpu,
which is not very atomic, and led to issues if the thread was migrated
to another core between the time we got the pcpu address and the time we
got curthread.
Instead, we now store curthread where pcpu used to be store, and we
calculate the pcpu address based on the cpu id.

Fix clang warning.

Remove #include <machine/frame.h> from all the arm code that doesn't
really need it. That would be almost everywhere it was included. Add
it in a couple files that really do need it and were previously getting
it by accident via another header.

Rename device vfp to option VFP and retire the ARM_VFP_SUPPORT option. This
simplifies enabling as previously both options were required to be enabled,
now we only need a single option.

While here enable VFP on the PandaBoard.

Merging projects/armv6, part 1

Cummulative patch of changes that are not vendor-specific:
- ARMv6 and ARMv7 architecture support
- ARM SMP support
- VFP/Neon support
- ARM Generic Interrupt Controller driver
- Simplification of startup code for all platforms

To prevent various race conditions in the RAS code, store and restore the
values in ARM_RAS_START and ARM_RAS_END at context switch time.

MFC after: 1 week

Provide L2 cache synchronization (write back + invalidation) on ARM.

Note the cpu_l2cache_wbinv_* routines are no-ops on systems not populated with
L2 caches.

Obtained from: Marvell, Semihalf

Improve ARM_TP_ADDRESS and RAS area.

De-hardcode usage of ARM_TP_ADDRESS and RAS local storage, and move this
special purpose page to a more convenient place i.e. after the vectors high
page, more towards the end of address space. Previous location (0xe000_0000)
caused grief if KVA was to go beyond the default limit.

Note that ARM world rebuilding is required after this change since the
location of ARM_TP_ADDRESS is shared between kernel and userland.

Submitted by: Grzegorz Bernacki (gjb AT semihalf dot com)
Reviewed by: imp
Approved by: cognet (mentor)

Ooops, we need to define TD_LOCK here.

Approved by: re (blanket)
Pointy hat to: cognet

Remove duplicate includes.

Submitted by: Cyril Nguyen Huu <cyril ci0 org>

Turn console printf buffering into a kernel option and only on
by default for sun4v where it is absolutely required.

This change moves the buffer from struct pcpu to the stack to avoid
using the critical section which created a LOR in a couple of cases
due to interaction with the tty code and kqueue. The LOR can't be
fixed with the critical section and the pcpu buffer can't be used
without the critical section.

Putting the buffer on the stack was my initial solution, but it was
pointed out that the stress on the stack might cause problems
depending on the call path. We don't have a way of creating tests
for those possible cases, so it's best to leave this as an option
for the time being. In time we may get enough data to enable this
option more generally.

Add a cnputs() function to write a string to the console with
a lock to prevent interspersed strings written from different CPUs
at the same time.

To avoid putting a buffer on the stack or having to malloc one,
space is incorporated in the per-cpu structure. The buffer
size if 128 bytes; chosen because it's the next power of 2 size
up from 80 characters.

String writes to the console are buffered up the end of the line
or until the buffer fills. Then the buffer is flushed to all
console devices.

Existing low level console output via cnputc() is unaffected by
this change. ithread calls to log() are also unaffected to avoid
blocking those threads.

A minor change to the behaviour in a panic situation is that
console output will still be buffered, but won't be written to
a tty as before. This should prevent interspersed panic output
as a number of CPUs panic before we end up single threaded
running ddb.

Reviewed by: scottl, jhb
MFC after: 2 weeks

Export PAGE_SIZE from genassym.c, and include assym.s in bcopy_page.S,
instead of <machine/param.h>.

Instead of using sysarch() to store-retrieve the tp, add a magic address,
ARM_TP_ADDRESS, where the tp will be stored. On CPUs that support it, a cache
line will be allocated and locked for this address, so that it will never go
to RAM. On CPUs that does not, a page is allocated for it (it will be a bit
slower, and is wrong for SMP, but should be fine for UP).
The tp is still stored in the mdthread struct, and at each context switch,
ARM_TP_ADDRESS gets updated.

Suggested by: davidxu

Remove references to U area and garbage collect includes.

Reviewed by: arch@

Implement interrupt counting, so that vmstat -i work.

Remove definitions related to the pmap cache state, and add TDF_NEEDRESCHED.

Import FreeBSD/arm kernel bits.
It only supports sa1110 (on simics) right now, but xscale support should come
soon.
Some of the initial work has been provided by :
Stephane Potvin <sepotvin at videotron.ca>
Most of this comes from NetBSD.