MFC r348798, r348813:
amd64 trap.c: Modernize syntax around trap_msg[].

MFC 340020: Don't enter DDB for fatal traps before panic by default.

Add a new 'debugger_on_trap' knob separate from 'debugger_on_panic'
and make the calls to kdb_trap() in MD fatal trap handlers prior to
calling panic() conditional on this new knob instead of
'debugger_on_panic'. Disable the new knob by default. Developers who
wish to recover from a fatal fault by adjusting saved register state
and retrying the faulting instruction can still do so by enabling the
new knob. However, for the more common case this makes the user
experience for panics due to a fatal fault match the user experience
for other panics, e.g. 'c' in DDB will generate a crash dump and
reboot the system rather than being stuck in an infinite loop of fatal
fault messages and DDB prompts.

MFC 338976: Don't clear DR6 for debug exceptions from userland.

This reverts part of r333368. The attempt to clear DR6 was occuring
too soon as trapsignal() does not pause to let the debugger notice the
SIGTRAP and query DR6. The signal exchange does not occur until much
later during ast(). As a result, GDB was no longer recognizing
hardware breakpoints and watchpoints on x86.

In addition, any userland programs that want to inspect DR6 in a
SIGTRAP handler don't have a way to do this if we clear DR6 in the
exception handler.

Instead of relying on the kernel to clear DR6, debuggers will have to
explicitly clear it after a trace trap (which they needed to do on
older kernels anyway).

MFC r338711:
Make the PTI violation check to follow style of the SMAP check.

MFC r338699:
Remove unneeded new line from the panic string.

MFC 332454,334009,334122: Various fixes for x86 debug exceptions.

332454:
Fix PSL_T inheritance on exec for x86.

The miscellaneous x86 sysent->sv_setregs() implementations tried to
migrate PSL_T from the previous program to the new executed one, but
they evaluated regs->tf_eflags after the whole regs structure was
bzeroed. Make this functional by saving PSL_T value before zeroing.

Note that if the debugger is not attached, executing the first
instruction in the new program with PSL_T set results in SIGTRAP, and
since all intercepted signals are reset to default dispostion on
exec(2), this means that non-debugged process gets killed immediately
if PSL_T is inherited. In particular, since suid images drop
P_TRACED, attempt to set PSL_T for execution of such program would
kill the process.

Another issue with userspace PSL_T handling is that it is reset by
trap(). It is reasonable to clear PSL_T when entering SIGTRAP
handler, to allow the signal to be handled without recursion or
delivery of blocked fault. But it is not reasonable to return back to
the normal flow with PSL_T cleared. This is too late to change, I
think.

334009:
Cleanups related to debug exceptions on x86.

- Add constants for fields in DR6 and the reserved fields in DR7. Use
these constants instead of magic numbers in most places that use DR6
and DR7.
- Refer to T_TRCTRAP as "debug exception" rather than a "trace trap"
as it is not just for trace exceptions.
- Always read DR6 for debug exceptions and only clear TF in the flags
register for user exceptions where DR6.BS is set.
- Clear DR6 before returning from a debug exception handler as
recommended by the SDM dating all the way back to the 386. This
allows debuggers to determine the cause of each exception. For
kernel traps, clear DR6 in the T_TRCTRAP case and pass DR6 by value
to other parts of the handler (namely, user_dbreg_trap()). For user
traps, wait until after trapsignal to clear DR6 so that userland
debuggers can read DR6 via PT_GETDBREGS while the thread is stopped
in trapsignal().

334122:
x86: stop unconditionally clearing PSL_T on the trace trap.

We certainly should clear PSL_T when calling the SIGTRAP signal
handler, which is already done by all x86 sendsig(9) ABI code. On the
other hand, there is no obvious reason why PSL_T needs to be cleared
when returning from the signal handler. For instance, Linux allows
userspace to set PSL_T and keep tracing enabled for the desired
period. There are userspace programs which would use PSL_T if we make
it possible, for instance sbcl.

Remember if PSL_T was set by PT_STEP or PT_SETSTEP by mean of TDB_STEP
flag, and only clear it when the flag is set.

MFC r334856, r338434:
Don't bother looking for non-executable pages when a process is
excluded from PTI.

MFC r338068, r338113:
Update L1TF workaround to sustain L1D pollution from NMI.

MFC r333059 (by tychon):
Expand the checks for UCR3 == PMAP_NO_CR3 to enable processes to be
excluded from PTI.

MFC r333368: Prepare DB# handler for deferred trigger of watchpoints.

Prepare DB# handler for deferred trigger of watchpoints.

Since pop %ss/mov %ss instructions defer all interrupts and exceptions
for the next instruction, it is possible that the userspace watchpoint
trap executes on the first instruction of the kernel entry for
syscall/bpt.

In this case, DB# should be treated similarly to NMI: on amd64 we must
always load GSBASE even if the trap comes from kernel mode, and load
the kernel page table root into %cr3. Moreover, the trap must
use the dedicated stack, because we are still on the user stack when
trapped on syscall entry.

For i386, we must reload %cr3. The syscall instruction is not configured,
so there is no issue with executing on user stack when trapping.

Due to some CPU erratas it is not always possible to detect that the
userspace watchpoint triggered by inspecting %dr6. In trap(), compare the
trap %rip with the known unsafe entry points and if matched pretend that
the watchpoint did not fire at all.

Thank you to the MSRC Incident Response Team, and in particular Greg
Lenti and Nate Warfield, for coordinating the response to this issue
across multiple vendors.

Thanks to Computer Recycling at The Working Center of Kitchener for
making hardware available to allow us to test the patch on additional
CPU families.

Reviewed by: jhb
Discussed with: Matthew Dillon
Tested by: emaste
Approved by: re (so blanket)
Security: CVE-2018-8897
Security: FreeBSD-SA-18:06.debugreg
Sponsored by: The FreeBSD Foundation

MFC r332752: set kdb_why to "trap" when calling kdb_trap from trap_fatal

This will allow to hook a ddb script to "kdb.enter.trap" event.
Previously there was no specific name for this event, so it could only
be handled by either "kdb.enter.unknown" or "kdb.enter.default" hooks.
Both are very unspecific.

Having a specific event is useful because the fatal trap condition is
very similar to panic but it has an additional property that the current
stack frame is the frame where the trap occurred. So, both a register
dump and a stack bottom dump have additional information that can help
analyze the problem.

I have added the event only on architectures that have trap_fatal()
function defined. I haven't looked at other architectures. Their
maintainers can add support for the event later.

Sample script:
kdb.enter.trap=bt; show reg; x/aS $rsp,20; x/agx $rsp,20

Sponsored by: Panzura

MFC r332730: don't check for kdb reentry in trap_fatal(), it's impossible

Sponsored by: Panzura

Revert r330897:

This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.

Revert with prejudice.

This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.

Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.

Requested by: gjb (re)

Partial merge of the SPDX changes

These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.

No objections from: pfg

MFC r328083,328096,328116,328119,328120,328128,328135,328153,328157,
328166,328177,328199,328202,328205,328468,328470,328624,328625,328627,
328628,329214,329297,329365:

Meltdown mitigation by PTI, PCID optimization of PTI, and kernel use of IBRS
for some mitigations of Spectre.

Tested by: emaste, Arshan Khanifar <arshankhanifar@gmail.com>
Discussed with: jkim
Sponsored by: The FreeBSD Foundation

MFC r327472:
Avoid re-check of usermode condition.

MFC r326774, r326811:
Pass the trap frame to fasttrap hooks.

MFC r321922:
amd64: annotate the syscall return address check with __predict_false

before:
0xffffffff80b03ebb <+2059>: mov 0x460(%r14),%rax
0xffffffff80b03ec2 <+2066>: mov 0x98(%rax),%rax
0xffffffff80b03ec9 <+2073>: shr $0x2f,%rax
0xffffffff80b03ecd <+2077>: je 0xffffffff80b03edd <amd64_syscall+2093>
0xffffffff80b03ecf <+2079>: mov 0x3f8(%r14),%rax
0xffffffff80b03ed6 <+2086>: orl $0x1,0xc8(%rax)
0xffffffff80b03edd <+2093>: add $0xf8,%rsp

after:
0xffffffff80b03ebb <+2059>: mov 0x460(%r14),%rax
0xffffffff80b03ec2 <+2066>: mov 0x98(%rax),%rax
0xffffffff80b03ec9 <+2073>: shr $0x2f,%rax
0xffffffff80b03ecd <+2077>: jne 0xffffffff80b03eef <amd64_syscall+2111>
0xffffffff80b03ecf <+2079>: add $0xf8,%rsp

MFC r322720,r322723:
Simplify amd64 trap().

MFC r322719:
Trim excessive 'extern' and remove unused declaration.

MFC r322718:
Use ANSI C declaration for trap_pfault(). Style.

MFC r322496:
Print whole machine state on double fault.

MFC r322494:
Style.

MFC r321919:
Do not call trapsignal() after handling usermode fault or interrupt,
when a signal is not intended to be sent.

MFC r319873:
Move struct syscall_args syscall arguments parameters container into
struct thread.

MFC 308820,308821: Fixes for fatal page faults on x86.

308820:
Report page faults due to reserved bits in PTEs as a separate fault type.

Rather than reporting a page fault due to a bad PTE as a protection
violation with the "rsv" flag, treat these faults as a separate type of
fault altogether.

308821:
MFamd64: Various fatal page fault fixes.

- If a page fault is triggered due to reserved bits in a PTE, treat it
as a fatal fault and panic.
- If PG_NX is in use, report whether a fatal page fault is due to an
instruction fetch or a data access.
- If a fatal page fault is due to reserved bits in a PTE, report that as
the page fault type rather than a protection violation.

MFC r310205:
Fix typo. Remove spurious blank line.

MFC r307866:
Handle broadcast NMIs.

MFC r307880:
Follow-up to r307866.

MFC r308030:
Use correct cpu id in the banner.

Merge bde improvements for ddb on x86, mostly for single-stepping and
vm86 mode.

MFC r304085 (by bde):
Fix the variables $esp, $ds, $es, $fs, $gs and $ss in vm86 mode. Fix
PC_REGS() so that printing of instructions works in some useful cases.

MFC r304962 (by bde):
Expand error messages: print symbol names, parentheses and shift tokens,
and negative shift counts. Fix error messages.

MFC r305612 (by bde):
Fix single-stepping of instructions emulated by vm86.

MFC r305661 (by bde):
Give the full syntax of the 'count' arg for all commmands that support
it. Give the full syntax of the 'addr' arg for these commands and some
others. Rename it from 'address' for the generic command. Fix
description of how 'count' is supposed to work for the 'break'
command.

Don't (mis)describe the syntax of the comma for the 'step' command.

Expand the description for the generic command.

Give the full syntax for the 'examine' command. It was also missing
the possible values for the modifier.

MFC r305663 (by bde):
Fix stopping when the specified breakpoint count is reached.

MFC r305665 (by bde):
Pass the trap type and code down from db_trap() to db_stop_at_pc() so
that the latter can easily determine what the trap type actually is
after callers are fixed to encode the type unambigously.

MFC r305807 (by bde):
Use the MI macro TRAPF_USERMODE() instead of open-coded checks for
SEL_UPL and sometimes PSL_VM. Fix logic errors in treating vm86
bioscall mode as kernel mode. The main place checked all the
necessary flags, but put the necessary parentheses for the PSL_VM and
PCB_VM86CALL checks in the wrong place.

MFC r305811 (by bz):
Try to fix LINT builds after r305807.

MFC r305840 (by bde):
Abort single stepping in ddb if the trap is not for single-stepping.

MFC r305862 (by bde):
Ifdef the new dr6 variable for KDB.

MFC r305864 (by bde):
Statically initialize the run mode to the one that will become current
on first entry. Don't reset to the run mode to STEP_NONE when
stopping, and remove STEP_NONE.

MFC r305865 (by bde):
Fix decoding of tf_rsp on amd64, and move TF_HAS_STACKREGS() to the
i386-only section, and fix a comment about the amd64 kernel trapframe
not having stackregs.

MFC r305897 (by bde):
Silently ignore unexpected single-step traps.

MFC r306311 (by bde):
Determine the operand/address size of %cs in a new function
db_segsize(). Use db_segsize() to set the default operand/address
size for disassembling.

Fix db_print_loc_and_inst() to ask for the normal format and not the
alternate in normal operation. Use db_segsize() to avoid trying to
print a garbage stack trace if %cs is 16 bits.

MFC r303913:
Unconditionally perform checks that FPU region was entered, when #NM
exception is caught in kernel mode.