MFC 340020: Don't enter DDB for fatal traps before panic by default.

Add a new 'debugger_on_trap' knob separate from 'debugger_on_panic'
and make the calls to kdb_trap() in MD fatal trap handlers prior to
calling panic() conditional on this new knob instead of
'debugger_on_panic'. Disable the new knob by default. Developers who
wish to recover from a fatal fault by adjusting saved register state
and retrying the faulting instruction can still do so by enabling the
new knob. However, for the more common case this makes the user
experience for panics due to a fatal fault match the user experience
for other panics, e.g. 'c' in DDB will generate a crash dump and
reboot the system rather than being stuck in an infinite loop of fatal
fault messages and DDB prompts.

MFC r333667: followup to r332730/r332752: set kdb_why to "trap" for fatal traps

This change updates arm, arm64 and mips achitectures. Additionally, it
removes redundant checks for kdb_active where it already results in
kdb_reenter() and adds kdb_reenter() calls where they were missing.

Some architectures check the return value of kdb_trap(), but some don't.
I haven't changed any of that.

Some trap handling routines have a return code. I am not sure if I
provided correct ones for returns after kdb_reenter(). kdb_reenter
should never return unless kdb_jmpbufp is NULL for some reason.

MFC r319873:
Move struct syscall_args syscall arguments parameters container into
struct thread.

MFC r317197:

Restrict the arm64 supervisor all instructions to only allow a zero
immediate value for system calls. We may wish to use other values in the
future for other purposes.

MFC r317192:

Push loading curthread into assembly in the synchronous exception handlers.
This will help investigating the performance impact of moving parts of the
switch statement in do_el0_sync into assembly.

MFC r313396:

Push reading of ESR_EL1 to assembly. Among other uses this will allow us
to expose this to signal handlers, e.g. for the clang sanitizers.

MFC 305607:
Trap msr/mrs instructions. These are privileged arm64 instructions and
shouldn't normally be used.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

MFC 305128:
Also handle instruction traps. We might hit these when the page we are
executing is being promoted to a superpage.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

MFC 304004, 304596, 304598, 304599, 304600, 304604, 304620, 304685, 304687,
304688, 304689, 304746, 304749, 304750, 304806, 305071, 305191:
Merge arm64 superpage support, however leave it disabled by default.

MFC after: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

MFC 305605:
Don't panic when we don't handle a userland exception, not all we may see
are currently handled.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Fix a race when the hardware has raised an exception with an unknown or
uncategorised reason. We need to read the fault address register before
enabling interrupts as the interrupt handler may cause this register to
change.

Approved by: re (marius, kib)
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

As <machine/vmparam.h> is included from <vm/vm_param.h>, there is no
need to include it explicitly when <vm/vm_param.h> is already included.

Suggested by: alc
Reviewed by: alc
Differential Revision: https://reviews.freebsd.org/D5379

Handle a misaligned stack pointer exception from userspace. The exception
still needs to be enabled, but this will help with testing.

Sponsored by: ABT Systems Ltd

Implement single stepping on arm64. We need to set the single step bits in
the processor and debug state registers. A flag has been added to the pcb
to tell us when to enable single stepping for a given thread.

Reviewed by: kib
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D4730

Print useful information when we hit a data abort we can't handle. This
prints the trap frame, along with the exception syndrome and fault address
registers. Even though esr is 64-bits here it is only 32-bits in hardware
so only print the valid 32-bits.

While here also print esr and far when appropriate after printing the trap
frame.

Sponsored by: ABT Systems Ltd

arm64: remove exception instruction length assertion

From the (now removed) comment:

* It is unclear in some cases if the bit is implementation defined.
* The Foundation Model and QEMU disagree on if the IL bit should
* be set when we are in a data fault from the same EL and the ISV
* bit (bit 24) is also set.

Instead of adding even more special cases just remove the assertion.

Approved by: andrew
Sponsored by: The FreeBSD Foundation

When trying to execute from a misaligned address raise a SIGBUS with the
invalid address alignment code.

Obtained from: EuroBSDCon
Sponsored by: ABT Systems Ltd

When asserting IL bit in exception syndrome register, print the raw
register value.

Reviewed by: andrew
Sponsored by: The FreeBSD Foundation

Do not execute exception handlers with disabled interrupts.

We should not call vm_fault(), or send a signal, with interrupts
disabled. MI kernel code is not prepared for such environment, not to
mention that this increases system latency, since code appears to be
executing as being under spinlock.

The FAR register for data aborts is read before the interrupts are
enabled, to avoid its corruption due to nested exception or context
switch.

Add asserts, similar to the checks done by other architectures, about
not taking page faults in non-sleepable contexts, rather than die with
late and somewhat confusing witness diagnostic.

Reviewed by: andrew
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3669

Clear exclusive monitors when handling data aborts, the monitors are
in unknown state per spec.

Reviewed by: andrew (previous version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3668

Do not hold the process around the vm_fault() call from the trap()s.
The only operation which is prevented by the hold is the kernel stack
swapout for the faulted thread, which should be fine to allow.

Remove useless checks for NULL curproc or curproc->p_vmspace from the
trap_pfault() wrappers on x86 and powerpc.

Reviewed by: alc (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Call into kdb on a data abort when it's enabled. This is needed to
correctly handle trying to access an invalid address in the debugger.

While here document that the breakpoint handler is supposed to fall
through to the following case.

Sponsored by: ABT Systems Ltd

Handle the breakpoint instruction in userland by sending a SIGTRAP signal.

Correct comment typo noted by erik

Remove register dump from arm64 el0 unknown exception

An exception with an unknown reasion is the expected result of the
attempted execution of an instruction bit pattern that has no allocated
instruction.

Sponsored by: The FreeBSD Foundation

arm64: turn unknown el0 exception into a SIGILL

It seems we get EXCP_UNKNOWN from QEMU when executing zeroed memory.
Print a register dump here and signal illegal instruction. Also print
a register dump for other invalid exceptions, before panic.

Reviewed by: andrew
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3370

Clean out some externally visible "more then" grammar

MFC after: 3 days

Add missing exception number to EL0 sync. abort on ARM64

When doing a data abort from userland it is possible to get
more than one data abort inside the same exception level.
Add an appropriate exception number to allow nesting of
data_abort handler for EL0.

Reviewed by: andrew
Obtained from: Semihalf
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3276

Add ARM64TODO markers to unimplemented functionality

Reviewed by: andrew
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D2389

Enable IRQ during syscalls on ARM64

FreeBSD provides a feature called Adaptive Mutexes, which allows
a thread to spin for a while when the mutex is taken instead of
immediately going to sleep. This causes issues when called from
syscall handler if interrupts are masked. If every other core
also attempts to access the same mutex there is a chance that
all of them are spinning on the same lock at the same time.
If interrupts are disabled, no kernel preemtion can occur and
the system becomes unresponsive.

This patch enables interrupts when syscall is being executed
and masks them as soon as it is completed.

Reviewed by: andrew
Obtained from: Semihalf
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3246

Add logging of synchronous exceptions.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

Send the correct signal when vm_fault fails. While here also set the code
and address fields.

Sponsored by: ABT Systems Ltd

First cut of DTrace for AArch64.

Reviewed by: andrew, emaste
Sponsored by: ARM Limited
Differential Revision: https://reviews.freebsd.org/D2738

Use the address of fsu_intr_fault, not the first two instructions in it as
the address to check if we are handling a trap for fuswintr or suswintr.

Sponsored by: The FreeBSD Foundation

Bring in the start of the arm64 kernel.

This is only the minimum set of files needed to boot in qemu. As such it is
missing a few things.

The bus_dma code is currently only stub functions with a full implementation
from the development tree to follow.

The gic driver has been copied as the interrupt framework is different. It
is expected the two drivers will be merged by the arm intrng project,
however this will need to be imported into the tree and support for arm64
would need to be added.

This includes code developed by myself, SemiHalf, Ed Maste, and Robin
Randhawa from ARM. This has been funded by the FreeBSD Foundation, with
early development by myself in my spare time with assistance from Robin.

Differential Revision: https://reviews.freebsd.org/D2199
Reviewed by: emaste, imp
Relnotes: yes
Sponsored by: The FreeBSD Foundation