kernel/x86_64: align the stack before calling user_debug_pre_syscall

Fixes #17531

Change-Id: Ia406ca15438bb2b432e9ebbe9f6db074f973af95
Reviewed-on: https://review.haiku-os.org/c/haiku/+/5113
Reviewed-by: waddlesplash <waddlesplash@gmail.com>
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>

kernel/x86_64: remove lfence after swapgs on syscall entry and exit

in a syscall, swapgs is always executed and can't be speculatively executed or bypassed.
it's also not needed on exception/interrupt exit, only on exception/interrupt entry.
follow-up on commit 84f6e2d39f55a1eeac8387e9cb456018d522afbd by waddlesplash

(https://github.com/freebsd/freebsd-src/commit/7aa47cace14948a7b8277a4b24a0ca9e0308990a)

Change-Id: I56de9526a1acd0075c4a12147ae782f0366dec52
Reviewed-on: https://review.haiku-os.org/c/haiku/+/4557
Tested-by: Commit checker robot <no-reply+buildbot@haiku-os.org>
Reviewed-by: Fredrik Holmqvist <fredrik.holmqvist@gmail.com>
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

kernel/x86_64: AVX support

xsave or xsavec are supported.
breaks vregs compatibility.
change the thread structure object cache alignment to 64
the xsave fpu_state size isn't defined, it is for instance 832 here, thus I picked 1024.

Change-Id: I4a0cab0bc42c1d37f24dcafb8259f8ff24a330d2
Reviewed-on: https://review.haiku-os.org/c/haiku/+/2849
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

kernel/x86_64: Always lfence after swapgs.

This works around "yet another" Intel CPU bug. AMD is unaffected.
Linux chose to use an altcodepatch for Intel only; FreeBSD did
a bunch of benchmarks and determined the effect this had (if any)
was so far out in the decimal places that just adding it unconditionally
was the easier and simplest solution.

(FreeBSD commit: https://github.com/freebsd/freebsd/commit/f4038696064b86260)

Fixes #15236.

:kernel: Fix stack alignment in syscall entry on x86_64.

The user iframe and associated data that the syscall entry pushes to the
stack directly were causing the stack to be mis-aligned by 8 bytes. Since
we re-aligned %rsp afterwards, for most usecases this wasn't a problem.
However, since we stored the pre-realinged %rsp in %rbp (as we need it to
access the iframe data), this also meant that anything which depended on
%rbp being 16-byte-aligned would run into serious problems.

As it turned out, GCC 7 assumed that %rbp was indeed 16-byte-aligned, and
so optimized certain accesses to use SSE instructions that depended on this
alignment. Since inside any callstack begining with a syscall this was not
the case, a "General Protection Exception" resulted (see #14160 for an example)
at the first usage of such an instruction. I wasn't really sure what was going
on when it first came up, and so "fixed" it by disabling the GCC optimization
that used such instructions. Replacing the -fdisable... with -mstackrealign thus
also "fixes" the problem, as I discovered earlier today, as it forces GCC to
realign the stack in function prologues.

So instead of rounding %rsp down to the nearest aligned address after the
pushes are complete, we offset %rsp by the amount the pushes are not,
thus fixing both %rsp and %rbp in syscall handling routines. This of course
depends on syscall_rsp being already aligned, which it is.

Thanks to PulkoMandy and js for the advice and guidance (and PulkoMandy
for the ASCII art), as this is essentially my first time working with
kernel assembly.

kernel/x86_64: add ia32 syscall entry and commpage syscall code.

* only for Intel sysenter/sysexit.
* the entry function processes stack arguments as follows:
we look up the syscall in the table, find the number of arguments.
reserve place on the stack for the arguments.
copy arguments on the stack
pop register arguments
call the syscall function
place the return value in ax and dx registers.
* TODO: we need to store the arguments somewhere for the post-syscall tracing.
* the thread exit stub is 32-bit, for the time being use hexadecimal instructions.

Change-Id: Ie5c502eb596d4fa7613d238de80643566bc19ed8

white space cleanup

kernel: support for Intel SMAP and SMEP on x86_64.

SMAP will generated page faults when the kernel tries to access user pages unless overriden.
If SMAP is enabled, the override instructions are written where needed in memory with
binary "altcodepatches".
Support is enabled by default, might be disabled per safemode setting.

Change-Id: Ife26cd765056aeaf65b2ffa3cadd0dcf4e273a96

kernel: Add core dump facility

* Add function core_dump_write_core_file(). It writes a core file for
the current thread's team. The file format is similar to that of
other OSs (i.e. ELF with PT_LOAD segments and a PT_NOTE segment), but
most of the notes are Haiku specific (infos for team, areas, images,
threads). More data will probably need to be added.
* Add team flag TEAM_FLAG_DUMP_CORE, thread flag
THREAD_FLAGS_TRAP_FOR_CORE_DUMP, and Team property coreDumpCondition,
a condition variable available while a core dump is progress. A
thread that finds its flag THREAD_FLAGS_TRAP_FOR_CORE_DUMP set before
exiting the kernel to userland calls core_dump_trap_thread(), which
blocks on the condition variable until the core dump has finished. We
need the team's threads to stop so we can get their CPU state (and
have a generally unchanging team state while writing the core file).
* Add user debugger message B_DEBUG_WRITE_CORE_FILE. It causes
core_dump_write_core_file() to be called for the team.
* Dumping core as an immediate effect of a terminal signal has not been
implemented yet, but that should be fairly straight forward.

kernel/x86_64: clear xmm0-15 registers on syscall exit

As Alex pointed out we can leak possibly sensitive data in xmm registers
when returning from the kernel. To prevent that xmm0-15 are zeroed
before sysret or iret. The cost is negligible.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>

kernel/x86_64: save fpu state at interrupts

The kernel is allowed to use fpu anywhere so we must make sure that
user state is not clobbered by saving fpu state at interrupt entry.
There is no need to do that in case of system calls since all fpu
data registers are caller saved.

We do not need, though, to save the whole fpu state at task swich
(again, thanks to calling convention). Only status and control
registers are preserved. This patch actually adds xmm0-15 register
to clobber list of task swich code, but the only reason of that is
to make sure that nothing bad happens inside the function that
executes that task swich. Inspection of the generated code shows
that no xmm registers are actually saved.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>

kernel/x86_64: make sure stack is properly aligned in syscalls

Just following the path of least resistance and adding andq $~15, %rsp
where appropriate. That should also make things harder to break
when changing the amount of stuff placed on stack before calling the
actual syscall routine.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>

x86: Let each CPU have its own GDT

x86_64: Set vector in iframe/debug state to 99 for syscalls.

Although syscalls are done through SYSCALL and therefore don't actually
have an interrupt number, set it to 99 (the syscall vector on 32-bit)
in the iframe so that a syscall frame can be identified. Also added
vector/error_code to x86_64_debug_cpu_state for Debugger to use, not
sure why I didn't put them there in the first place.

Implemented user debugging support for x86_64.

Reused x86 arch_user_debugger.cpp, with a few minor changes to make
the code work for both 32 and 64 bit. Something isn't quite working
right, if a breakpoint is hit the kernel will hang. Other than that
everything appears to work correctly.

System call restart support.

Improved safety for user memory accesses.

* Changed IS_USER_ADDRESS to check an address using USER_BASE and
USER_SIZE, rather than just !IS_KERNEL_ADDRESS. The old check would
allow user buffers to point into the physical memory map area.
* Added an unmapped hole at the end of the bottom half of the address
space which catches buffers that cross into the uncanonical address
region. This also removes the need to check for uncanonical return
addresses in the syscall handler, it is no longer possible for the
return address to be uncanonical under normal circumstances. All
cases in which the return address might be changed by the kernel
are still handled via the IRET path.

Implemented signals for x86_64.

Finished system call handler.

The interrupt and system call handlers now perform all the necessary
kernel entry/exit work, and the system call handler now handles calls
with more than 6 arguments. Debugging and system call tracing hooks
are not yet called, will be added when user debugging gets implemented.

Initial userland support for x86_64.

Userland switch is implemented, as is basic system call support (using
SYSCALL/SYSRET). The system call handler is not yet complete: it doesn't
handle more than 6 arguments, and does not perform all the necessary kernel
entry/exit work (neither does the interrupt handler). However, this is
sufficient for runtime_loader to start and print some debug output.

Implemented threading for x86_64.

* Thread creation and switching is working fine, however threads do not yet
get interrupted because I've not implemented hardware interrupt handling
yet (I'll do that next).
* I've made some changes to struct iframe: I've removed the e/r prefixes
from the member names for both 32/64, so now they're just named ip, ax,
bp, etc. This makes it easier to write code that works with both 32/64
without having to deal with different iframe member names.

Merged in the x86_64 exception handling.

I've split the 32-bit dependent IDT setup code and ASM interrupt handlers to
the 32 subdirectory, arch_int.cpp now contains only the generic hardware
interrupt handling code.

kernel/x86_64: clear xmm0-15 registers on syscall exit

As Alex pointed out we can leak possibly sensitive data in xmm registers
when returning from the kernel. To prevent that xmm0-15 are zeroed
before sysret or iret. The cost is negligible.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>

kernel/x86_64: save fpu state at interrupts

The kernel is allowed to use fpu anywhere so we must make sure that
user state is not clobbered by saving fpu state at interrupt entry.
There is no need to do that in case of system calls since all fpu
data registers are caller saved.

We do not need, though, to save the whole fpu state at task swich
(again, thanks to calling convention). Only status and control
registers are preserved. This patch actually adds xmm0-15 register
to clobber list of task swich code, but the only reason of that is
to make sure that nothing bad happens inside the function that
executes that task swich. Inspection of the generated code shows
that no xmm registers are actually saved.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>

kernel/x86_64: make sure stack is properly aligned in syscalls

Just following the path of least resistance and adding andq $~15, %rsp
where appropriate. That should also make things harder to break
when changing the amount of stuff placed on stack before calling the
actual syscall routine.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>

x86: Let each CPU have its own GDT

x86_64: Set vector in iframe/debug state to 99 for syscalls.

Although syscalls are done through SYSCALL and therefore don't actually
have an interrupt number, set it to 99 (the syscall vector on 32-bit)
in the iframe so that a syscall frame can be identified. Also added
vector/error_code to x86_64_debug_cpu_state for Debugger to use, not
sure why I didn't put them there in the first place.

Implemented user debugging support for x86_64.

Reused x86 arch_user_debugger.cpp, with a few minor changes to make
the code work for both 32 and 64 bit. Something isn't quite working
right, if a breakpoint is hit the kernel will hang. Other than that
everything appears to work correctly.

System call restart support.

Improved safety for user memory accesses.

* Changed IS_USER_ADDRESS to check an address using USER_BASE and
USER_SIZE, rather than just !IS_KERNEL_ADDRESS. The old check would
allow user buffers to point into the physical memory map area.
* Added an unmapped hole at the end of the bottom half of the address
space which catches buffers that cross into the uncanonical address
region. This also removes the need to check for uncanonical return
addresses in the syscall handler, it is no longer possible for the
return address to be uncanonical under normal circumstances. All
cases in which the return address might be changed by the kernel
are still handled via the IRET path.

Implemented signals for x86_64.

Finished system call handler.

The interrupt and system call handlers now perform all the necessary
kernel entry/exit work, and the system call handler now handles calls
with more than 6 arguments. Debugging and system call tracing hooks
are not yet called, will be added when user debugging gets implemented.

Initial userland support for x86_64.

Userland switch is implemented, as is basic system call support (using
SYSCALL/SYSRET). The system call handler is not yet complete: it doesn't
handle more than 6 arguments, and does not perform all the necessary kernel
entry/exit work (neither does the interrupt handler). However, this is
sufficient for runtime_loader to start and print some debug output.

Implemented threading for x86_64.

* Thread creation and switching is working fine, however threads do not yet
get interrupted because I've not implemented hardware interrupt handling
yet (I'll do that next).
* I've made some changes to struct iframe: I've removed the e/r prefixes
from the member names for both 32/64, so now they're just named ip, ax,
bp, etc. This makes it easier to write code that works with both 32/64
without having to deal with different iframe member names.

Merged in the x86_64 exception handling.

I've split the 32-bit dependent IDT setup code and ASM interrupt handlers to
the 32 subdirectory, arch_int.cpp now contains only the generic hardware
interrupt handling code.