Add TLB bypass for instruction emulation

copyinsn() fetches a userland instruction through the direct map.
This lets emulation work with execute-only virtual memory mappings.

OK deraadt@

spelling

Push KERNEL_LOCK/UNLOCK() dance inside trapsignal().

ok kettenis@, visa@

Wrap floating point instructions with .set hardfloat to make clang's
integrated assembler accept them in the kernel.

Move fsr access code into dedicated inline functions to improve
readability.

Set floating point condition code even if the result of an emulated
floating point comparison is unordered. The setting should be skipped
only if an invalid operation exception is taken.

This fixes incorrect emulated compare behaviour with NaN values.

NaN issue on octeon reported by afresh1@; OK miod@

More "explicitely" -> "explicitly" in various comments.

ok guenther@ tb@ deraadt@

Make sure that `fsr' is always initialized. Non-DEBUG kernels
have not been affected.

From miod@

Let the kernel utilize the FPU if one is available, even when the
FPUEMUL option is enabled. This benefits OCTEON III systems which can
run floating-point operations natively.

Feedback from and OK miod@; he also helped with testing.

Tested on octeon without FPU (CN5020, CN6120) and with FPU (CN7130),
as well as on sgi/IP27 (MP R16000), sgi/IP32 (R5000), and
loongson (3A1000).

Prefer copyin32() to copyin() when fetching instructions
from user space. This improves performance slightly.

Discussed with miod@

Use copyin32() instead of a direct memory load when fetching a branch
instruction for branch emulation. This ensures the userspace memory
access is properly guarded and that TLB faults are handled.

In order not to complicate the interface of MipsEmulateBranch(), each
caller now has to provide the branch instruction for the function.

Feedback from miod@

p_comm is the process's command and isn't per thread, so move it from
struct proc to struct process.

ok deraadt@ kettenis@

Rename mips64's trap_frame into trapframe.

For coherency with other archs and in order to use it in MI code.

ok visa@, tobiasu@

Initialize `pc' earlier so that the siginfo pc value is correct in case of
genuine FPU exception.

Access the image of the floating point registers via p_md.md_regs, instead of
directly on the frame, or any updates will get lost. This went unnoticed for
so long because FPU emulation traps usually use the first frame of the U area,
so values kind of stick.

This fixes a rare occurrence of wrong floating-point values with MP kernels
on Octeon.

ok visa@

no need to swizzle load/store addresses for 32bit values on big endian systems

ok miod@

Fix a few format string warnings, allow to build DEBUG kernel on sgi

OK miod@

It's been a quarter century: we can assume volatile is present with that name.

ok dlg@ mpi@ deraadt@

Split ever-growing mips <machine/cpu.h> into what 99% of the kernel needs,
which will remain in <machine/cpu.h>, and a new mips_cpu.h containing only the
goriest md details, which are only of interest to a handful set of files; this
is similar in spirit to what alpha does, but here <machine/cpu.h> does not
include the new file.

Handle the coprocessor 0 cause and status registers as a 64 bit value now,
as some odd mips designs need moro than 32 bits in there. This causes a lot
of mechanical changes everywhere getsr() is used.

Revert art@'s moving around of the KERNEL_LOCK()/KERNEL_UNLOCK() calls,
as it causes hangs in some ports, including libsigsegv's configure script

confirmed by krw@, landry@

There is a bunch of places in the kernel entry points where we don't
hold the kernel lock, but still need call one function that needs it.

Instead of grabbing the lock all over the place, move the locks into
the affected functions: trapsignal, scdebug*, ktrsyscall, ktrsysret,
systrace_redirect and ADDUPROF. In the cases we already hold the biglock
we'll just recurse.

kettenis@, beck@ ok

Clean up after P_BIGLOCK removal.
KERNEL_PROC_LOCK -> KERNEL_LOCK
KERNEL_PROC_UNLOCK -> KERNEL_UNLOCK

oga@ ok

Floating-point emulation code for systems lacking proper FPU (i.e. Octeon),
enabled by option FPUEMUL.

This is pretty straightforward, except for conditional branch on FPU condition
codes emulation (bc1f/bc1fl/bc1t/bc1tl instructions): unlike most
RISC-with-delay-slots designs (m88k, sparc), the branch pipeline is not exposed
to the kernel on Mips, therefore we can not resume a branch without losing the
delay slot instruction.

Some other operating systems work around this issue by emulating the delay
slot instruction, but this is error-prone (and requires the kernel code to
be aware of all supported instructions of the processor it is currently running
on), some use dedicated breakpoints to single-step through the delay slot and
then resume the branch as expected, but this causes a lot of copy-on-write
allocations.

This code chooses a third path, of copying the delay slot instructions to run toa special `magic' page, followed by a special trap instruction to give control
back to the kernel. This makes sure the instruction will actually be run by the
processor, and that no more than one page per process is wasted, regardless of
the number of branches to emulate.

Tested on octeon (big-endian) by syuu@ and on loongson (little-endian) by me.
Note that enabling option FPUEMUL in the kernel will completely disable the
hardware FPU, if there is one; there is currently no way to build a kernel
supporting both hardware and software FPU, and there is no reason to change
this until there is a strong need to support both.

Fix a few logic errors in comparison instruction emulation: make sure the
less than relation is correctly computed, and check for both operands being
signaling NaNs, instead of only the first NaN found, to decide whether to
raise an invalid exception or not.

Replace the old floating point completion code with a C interface to the
MI softfloat code, implementing all MIPS IV specified floating point
operations.
Tested on R5000, R10000, R14000 and Loongson2F.

spelling

Push KERNEL_LOCK/UNLOCK() dance inside trapsignal().

ok kettenis@, visa@

Wrap floating point instructions with .set hardfloat to make clang's
integrated assembler accept them in the kernel.

Move fsr access code into dedicated inline functions to improve
readability.

Set floating point condition code even if the result of an emulated
floating point comparison is unordered. The setting should be skipped
only if an invalid operation exception is taken.

This fixes incorrect emulated compare behaviour with NaN values.

NaN issue on octeon reported by afresh1@; OK miod@

More "explicitely" -> "explicitly" in various comments.

ok guenther@ tb@ deraadt@

Make sure that `fsr' is always initialized. Non-DEBUG kernels
have not been affected.

From miod@

Let the kernel utilize the FPU if one is available, even when the
FPUEMUL option is enabled. This benefits OCTEON III systems which can
run floating-point operations natively.

Feedback from and OK miod@; he also helped with testing.

Tested on octeon without FPU (CN5020, CN6120) and with FPU (CN7130),
as well as on sgi/IP27 (MP R16000), sgi/IP32 (R5000), and
loongson (3A1000).

Prefer copyin32() to copyin() when fetching instructions
from user space. This improves performance slightly.

Discussed with miod@

Use copyin32() instead of a direct memory load when fetching a branch
instruction for branch emulation. This ensures the userspace memory
access is properly guarded and that TLB faults are handled.

In order not to complicate the interface of MipsEmulateBranch(), each
caller now has to provide the branch instruction for the function.

Feedback from miod@

p_comm is the process's command and isn't per thread, so move it from
struct proc to struct process.

ok deraadt@ kettenis@

Rename mips64's trap_frame into trapframe.

For coherency with other archs and in order to use it in MI code.

ok visa@, tobiasu@

Initialize `pc' earlier so that the siginfo pc value is correct in case of
genuine FPU exception.

Access the image of the floating point registers via p_md.md_regs, instead of
directly on the frame, or any updates will get lost. This went unnoticed for
so long because FPU emulation traps usually use the first frame of the U area,
so values kind of stick.

This fixes a rare occurrence of wrong floating-point values with MP kernels
on Octeon.

ok visa@

no need to swizzle load/store addresses for 32bit values on big endian systems

ok miod@

Fix a few format string warnings, allow to build DEBUG kernel on sgi

OK miod@

It's been a quarter century: we can assume volatile is present with that name.

ok dlg@ mpi@ deraadt@

Split ever-growing mips <machine/cpu.h> into what 99% of the kernel needs,
which will remain in <machine/cpu.h>, and a new mips_cpu.h containing only the
goriest md details, which are only of interest to a handful set of files; this
is similar in spirit to what alpha does, but here <machine/cpu.h> does not
include the new file.

Handle the coprocessor 0 cause and status registers as a 64 bit value now,
as some odd mips designs need moro than 32 bits in there. This causes a lot
of mechanical changes everywhere getsr() is used.

Revert art@'s moving around of the KERNEL_LOCK()/KERNEL_UNLOCK() calls,
as it causes hangs in some ports, including libsigsegv's configure script

confirmed by krw@, landry@

There is a bunch of places in the kernel entry points where we don't
hold the kernel lock, but still need call one function that needs it.

Instead of grabbing the lock all over the place, move the locks into
the affected functions: trapsignal, scdebug*, ktrsyscall, ktrsysret,
systrace_redirect and ADDUPROF. In the cases we already hold the biglock
we'll just recurse.

kettenis@, beck@ ok

Clean up after P_BIGLOCK removal.
KERNEL_PROC_LOCK -> KERNEL_LOCK
KERNEL_PROC_UNLOCK -> KERNEL_UNLOCK

oga@ ok

Floating-point emulation code for systems lacking proper FPU (i.e. Octeon),
enabled by option FPUEMUL.

This is pretty straightforward, except for conditional branch on FPU condition
codes emulation (bc1f/bc1fl/bc1t/bc1tl instructions): unlike most
RISC-with-delay-slots designs (m88k, sparc), the branch pipeline is not exposed
to the kernel on Mips, therefore we can not resume a branch without losing the
delay slot instruction.

Some other operating systems work around this issue by emulating the delay
slot instruction, but this is error-prone (and requires the kernel code to
be aware of all supported instructions of the processor it is currently running
on), some use dedicated breakpoints to single-step through the delay slot and
then resume the branch as expected, but this causes a lot of copy-on-write
allocations.

This code chooses a third path, of copying the delay slot instructions to run toa special `magic' page, followed by a special trap instruction to give control
back to the kernel. This makes sure the instruction will actually be run by the
processor, and that no more than one page per process is wasted, regardless of
the number of branches to emulate.

Tested on octeon (big-endian) by syuu@ and on loongson (little-endian) by me.
Note that enabling option FPUEMUL in the kernel will completely disable the
hardware FPU, if there is one; there is currently no way to build a kernel
supporting both hardware and software FPU, and there is no reason to change
this until there is a strong need to support both.

Fix a few logic errors in comparison instruction emulation: make sure the
less than relation is correctly computed, and check for both operands being
signaling NaNs, instead of only the first NaN found, to decide whether to
raise an invalid exception or not.

Replace the old floating point completion code with a C interface to the
MI softfloat code, implementing all MIPS IV specified floating point
operations.
Tested on R5000, R10000, R14000 and Loongson2F.

Push KERNEL_LOCK/UNLOCK() dance inside trapsignal().

ok kettenis@, visa@

Wrap floating point instructions with .set hardfloat to make clang's
integrated assembler accept them in the kernel.

Move fsr access code into dedicated inline functions to improve
readability.

Set floating point condition code even if the result of an emulated
floating point comparison is unordered. The setting should be skipped
only if an invalid operation exception is taken.

This fixes incorrect emulated compare behaviour with NaN values.

NaN issue on octeon reported by afresh1@; OK miod@

More "explicitely" -> "explicitly" in various comments.

ok guenther@ tb@ deraadt@

Make sure that `fsr' is always initialized. Non-DEBUG kernels
have not been affected.

From miod@

Let the kernel utilize the FPU if one is available, even when the
FPUEMUL option is enabled. This benefits OCTEON III systems which can
run floating-point operations natively.

Feedback from and OK miod@; he also helped with testing.

Tested on octeon without FPU (CN5020, CN6120) and with FPU (CN7130),
as well as on sgi/IP27 (MP R16000), sgi/IP32 (R5000), and
loongson (3A1000).

Prefer copyin32() to copyin() when fetching instructions
from user space. This improves performance slightly.

Discussed with miod@

Use copyin32() instead of a direct memory load when fetching a branch
instruction for branch emulation. This ensures the userspace memory
access is properly guarded and that TLB faults are handled.

In order not to complicate the interface of MipsEmulateBranch(), each
caller now has to provide the branch instruction for the function.

Feedback from miod@

p_comm is the process's command and isn't per thread, so move it from
struct proc to struct process.

ok deraadt@ kettenis@

Rename mips64's trap_frame into trapframe.

For coherency with other archs and in order to use it in MI code.

ok visa@, tobiasu@

Initialize `pc' earlier so that the siginfo pc value is correct in case of
genuine FPU exception.

Access the image of the floating point registers via p_md.md_regs, instead of
directly on the frame, or any updates will get lost. This went unnoticed for
so long because FPU emulation traps usually use the first frame of the U area,
so values kind of stick.

This fixes a rare occurrence of wrong floating-point values with MP kernels
on Octeon.

ok visa@

no need to swizzle load/store addresses for 32bit values on big endian systems

ok miod@

Fix a few format string warnings, allow to build DEBUG kernel on sgi

OK miod@

It's been a quarter century: we can assume volatile is present with that name.

ok dlg@ mpi@ deraadt@

Split ever-growing mips <machine/cpu.h> into what 99% of the kernel needs,
which will remain in <machine/cpu.h>, and a new mips_cpu.h containing only the
goriest md details, which are only of interest to a handful set of files; this
is similar in spirit to what alpha does, but here <machine/cpu.h> does not
include the new file.

Handle the coprocessor 0 cause and status registers as a 64 bit value now,
as some odd mips designs need moro than 32 bits in there. This causes a lot
of mechanical changes everywhere getsr() is used.

Revert art@'s moving around of the KERNEL_LOCK()/KERNEL_UNLOCK() calls,
as it causes hangs in some ports, including libsigsegv's configure script

confirmed by krw@, landry@

There is a bunch of places in the kernel entry points where we don't
hold the kernel lock, but still need call one function that needs it.

Instead of grabbing the lock all over the place, move the locks into
the affected functions: trapsignal, scdebug*, ktrsyscall, ktrsysret,
systrace_redirect and ADDUPROF. In the cases we already hold the biglock
we'll just recurse.

kettenis@, beck@ ok

Clean up after P_BIGLOCK removal.
KERNEL_PROC_LOCK -> KERNEL_LOCK
KERNEL_PROC_UNLOCK -> KERNEL_UNLOCK

oga@ ok

Floating-point emulation code for systems lacking proper FPU (i.e. Octeon),
enabled by option FPUEMUL.

This is pretty straightforward, except for conditional branch on FPU condition
codes emulation (bc1f/bc1fl/bc1t/bc1tl instructions): unlike most
RISC-with-delay-slots designs (m88k, sparc), the branch pipeline is not exposed
to the kernel on Mips, therefore we can not resume a branch without losing the
delay slot instruction.

Some other operating systems work around this issue by emulating the delay
slot instruction, but this is error-prone (and requires the kernel code to
be aware of all supported instructions of the processor it is currently running
on), some use dedicated breakpoints to single-step through the delay slot and
then resume the branch as expected, but this causes a lot of copy-on-write
allocations.

This code chooses a third path, of copying the delay slot instructions to run toa special `magic' page, followed by a special trap instruction to give control
back to the kernel. This makes sure the instruction will actually be run by the
processor, and that no more than one page per process is wasted, regardless of
the number of branches to emulate.

Tested on octeon (big-endian) by syuu@ and on loongson (little-endian) by me.
Note that enabling option FPUEMUL in the kernel will completely disable the
hardware FPU, if there is one; there is currently no way to build a kernel
supporting both hardware and software FPU, and there is no reason to change
this until there is a strong need to support both.

Fix a few logic errors in comparison instruction emulation: make sure the
less than relation is correctly computed, and check for both operands being
signaling NaNs, instead of only the first NaN found, to decide whether to
raise an invalid exception or not.

Replace the old floating point completion code with a C interface to the
MI softfloat code, implementing all MIPS IV specified floating point
operations.
Tested on R5000, R10000, R14000 and Loongson2F.

Wrap floating point instructions with .set hardfloat to make clang's
integrated assembler accept them in the kernel.

Move fsr access code into dedicated inline functions to improve
readability.

Set floating point condition code even if the result of an emulated
floating point comparison is unordered. The setting should be skipped
only if an invalid operation exception is taken.

This fixes incorrect emulated compare behaviour with NaN values.

NaN issue on octeon reported by afresh1@; OK miod@

More "explicitely" -> "explicitly" in various comments.

ok guenther@ tb@ deraadt@

Make sure that `fsr' is always initialized. Non-DEBUG kernels
have not been affected.

From miod@

Let the kernel utilize the FPU if one is available, even when the
FPUEMUL option is enabled. This benefits OCTEON III systems which can
run floating-point operations natively.

Feedback from and OK miod@; he also helped with testing.

Tested on octeon without FPU (CN5020, CN6120) and with FPU (CN7130),
as well as on sgi/IP27 (MP R16000), sgi/IP32 (R5000), and
loongson (3A1000).

Prefer copyin32() to copyin() when fetching instructions
from user space. This improves performance slightly.

Discussed with miod@

Use copyin32() instead of a direct memory load when fetching a branch
instruction for branch emulation. This ensures the userspace memory
access is properly guarded and that TLB faults are handled.

In order not to complicate the interface of MipsEmulateBranch(), each
caller now has to provide the branch instruction for the function.

Feedback from miod@

p_comm is the process's command and isn't per thread, so move it from
struct proc to struct process.

ok deraadt@ kettenis@

Rename mips64's trap_frame into trapframe.

For coherency with other archs and in order to use it in MI code.

ok visa@, tobiasu@

Initialize `pc' earlier so that the siginfo pc value is correct in case of
genuine FPU exception.

Access the image of the floating point registers via p_md.md_regs, instead of
directly on the frame, or any updates will get lost. This went unnoticed for
so long because FPU emulation traps usually use the first frame of the U area,
so values kind of stick.

This fixes a rare occurrence of wrong floating-point values with MP kernels
on Octeon.

ok visa@

no need to swizzle load/store addresses for 32bit values on big endian systems

ok miod@

Fix a few format string warnings, allow to build DEBUG kernel on sgi

OK miod@

It's been a quarter century: we can assume volatile is present with that name.

ok dlg@ mpi@ deraadt@

Split ever-growing mips <machine/cpu.h> into what 99% of the kernel needs,
which will remain in <machine/cpu.h>, and a new mips_cpu.h containing only the
goriest md details, which are only of interest to a handful set of files; this
is similar in spirit to what alpha does, but here <machine/cpu.h> does not
include the new file.

Handle the coprocessor 0 cause and status registers as a 64 bit value now,
as some odd mips designs need moro than 32 bits in there. This causes a lot
of mechanical changes everywhere getsr() is used.

Revert art@'s moving around of the KERNEL_LOCK()/KERNEL_UNLOCK() calls,
as it causes hangs in some ports, including libsigsegv's configure script

confirmed by krw@, landry@

There is a bunch of places in the kernel entry points where we don't
hold the kernel lock, but still need call one function that needs it.

Instead of grabbing the lock all over the place, move the locks into
the affected functions: trapsignal, scdebug*, ktrsyscall, ktrsysret,
systrace_redirect and ADDUPROF. In the cases we already hold the biglock
we'll just recurse.

kettenis@, beck@ ok

Clean up after P_BIGLOCK removal.
KERNEL_PROC_LOCK -> KERNEL_LOCK
KERNEL_PROC_UNLOCK -> KERNEL_UNLOCK

oga@ ok

Floating-point emulation code for systems lacking proper FPU (i.e. Octeon),
enabled by option FPUEMUL.

This is pretty straightforward, except for conditional branch on FPU condition
codes emulation (bc1f/bc1fl/bc1t/bc1tl instructions): unlike most
RISC-with-delay-slots designs (m88k, sparc), the branch pipeline is not exposed
to the kernel on Mips, therefore we can not resume a branch without losing the
delay slot instruction.

Some other operating systems work around this issue by emulating the delay
slot instruction, but this is error-prone (and requires the kernel code to
be aware of all supported instructions of the processor it is currently running
on), some use dedicated breakpoints to single-step through the delay slot and
then resume the branch as expected, but this causes a lot of copy-on-write
allocations.

This code chooses a third path, of copying the delay slot instructions to run toa special `magic' page, followed by a special trap instruction to give control
back to the kernel. This makes sure the instruction will actually be run by the
processor, and that no more than one page per process is wasted, regardless of
the number of branches to emulate.

Tested on octeon (big-endian) by syuu@ and on loongson (little-endian) by me.
Note that enabling option FPUEMUL in the kernel will completely disable the
hardware FPU, if there is one; there is currently no way to build a kernel
supporting both hardware and software FPU, and there is no reason to change
this until there is a strong need to support both.

Fix a few logic errors in comparison instruction emulation: make sure the
less than relation is correctly computed, and check for both operands being
signaling NaNs, instead of only the first NaN found, to decide whether to
raise an invalid exception or not.

Replace the old floating point completion code with a C interface to the
MI softfloat code, implementing all MIPS IV specified floating point
operations.
Tested on R5000, R10000, R14000 and Loongson2F.

Set floating point condition code even if the result of an emulated
floating point comparison is unordered. The setting should be skipped
only if an invalid operation exception is taken.

This fixes incorrect emulated compare behaviour with NaN values.

NaN issue on octeon reported by afresh1@; OK miod@

More "explicitely" -> "explicitly" in various comments.

ok guenther@ tb@ deraadt@

Make sure that `fsr' is always initialized. Non-DEBUG kernels
have not been affected.

From miod@

Let the kernel utilize the FPU if one is available, even when the
FPUEMUL option is enabled. This benefits OCTEON III systems which can
run floating-point operations natively.

Feedback from and OK miod@; he also helped with testing.

Tested on octeon without FPU (CN5020, CN6120) and with FPU (CN7130),
as well as on sgi/IP27 (MP R16000), sgi/IP32 (R5000), and
loongson (3A1000).

Prefer copyin32() to copyin() when fetching instructions
from user space. This improves performance slightly.

Discussed with miod@

Use copyin32() instead of a direct memory load when fetching a branch
instruction for branch emulation. This ensures the userspace memory
access is properly guarded and that TLB faults are handled.

In order not to complicate the interface of MipsEmulateBranch(), each
caller now has to provide the branch instruction for the function.

Feedback from miod@

p_comm is the process's command and isn't per thread, so move it from
struct proc to struct process.

ok deraadt@ kettenis@

Rename mips64's trap_frame into trapframe.

For coherency with other archs and in order to use it in MI code.

ok visa@, tobiasu@

Initialize `pc' earlier so that the siginfo pc value is correct in case of
genuine FPU exception.

Access the image of the floating point registers via p_md.md_regs, instead of
directly on the frame, or any updates will get lost. This went unnoticed for
so long because FPU emulation traps usually use the first frame of the U area,
so values kind of stick.

This fixes a rare occurrence of wrong floating-point values with MP kernels
on Octeon.

ok visa@

no need to swizzle load/store addresses for 32bit values on big endian systems

ok miod@

Fix a few format string warnings, allow to build DEBUG kernel on sgi

OK miod@

It's been a quarter century: we can assume volatile is present with that name.

ok dlg@ mpi@ deraadt@

Split ever-growing mips <machine/cpu.h> into what 99% of the kernel needs,
which will remain in <machine/cpu.h>, and a new mips_cpu.h containing only the
goriest md details, which are only of interest to a handful set of files; this
is similar in spirit to what alpha does, but here <machine/cpu.h> does not
include the new file.

Handle the coprocessor 0 cause and status registers as a 64 bit value now,
as some odd mips designs need moro than 32 bits in there. This causes a lot
of mechanical changes everywhere getsr() is used.

Revert art@'s moving around of the KERNEL_LOCK()/KERNEL_UNLOCK() calls,
as it causes hangs in some ports, including libsigsegv's configure script

confirmed by krw@, landry@

There is a bunch of places in the kernel entry points where we don't
hold the kernel lock, but still need call one function that needs it.

Instead of grabbing the lock all over the place, move the locks into
the affected functions: trapsignal, scdebug*, ktrsyscall, ktrsysret,
systrace_redirect and ADDUPROF. In the cases we already hold the biglock
we'll just recurse.

kettenis@, beck@ ok

Clean up after P_BIGLOCK removal.
KERNEL_PROC_LOCK -> KERNEL_LOCK
KERNEL_PROC_UNLOCK -> KERNEL_UNLOCK

oga@ ok

Floating-point emulation code for systems lacking proper FPU (i.e. Octeon),
enabled by option FPUEMUL.

This is pretty straightforward, except for conditional branch on FPU condition
codes emulation (bc1f/bc1fl/bc1t/bc1tl instructions): unlike most
RISC-with-delay-slots designs (m88k, sparc), the branch pipeline is not exposed
to the kernel on Mips, therefore we can not resume a branch without losing the
delay slot instruction.

Some other operating systems work around this issue by emulating the delay
slot instruction, but this is error-prone (and requires the kernel code to
be aware of all supported instructions of the processor it is currently running
on), some use dedicated breakpoints to single-step through the delay slot and
then resume the branch as expected, but this causes a lot of copy-on-write
allocations.

This code chooses a third path, of copying the delay slot instructions to run toa special `magic' page, followed by a special trap instruction to give control
back to the kernel. This makes sure the instruction will actually be run by the
processor, and that no more than one page per process is wasted, regardless of
the number of branches to emulate.

Tested on octeon (big-endian) by syuu@ and on loongson (little-endian) by me.
Note that enabling option FPUEMUL in the kernel will completely disable the
hardware FPU, if there is one; there is currently no way to build a kernel
supporting both hardware and software FPU, and there is no reason to change
this until there is a strong need to support both.

Fix a few logic errors in comparison instruction emulation: make sure the
less than relation is correctly computed, and check for both operands being
signaling NaNs, instead of only the first NaN found, to decide whether to
raise an invalid exception or not.

Replace the old floating point completion code with a C interface to the
MI softfloat code, implementing all MIPS IV specified floating point
operations.
Tested on R5000, R10000, R14000 and Loongson2F.

More "explicitely" -> "explicitly" in various comments.

ok guenther@ tb@ deraadt@

Make sure that `fsr' is always initialized. Non-DEBUG kernels
have not been affected.

From miod@

Let the kernel utilize the FPU if one is available, even when the
FPUEMUL option is enabled. This benefits OCTEON III systems which can
run floating-point operations natively.

Feedback from and OK miod@; he also helped with testing.

Tested on octeon without FPU (CN5020, CN6120) and with FPU (CN7130),
as well as on sgi/IP27 (MP R16000), sgi/IP32 (R5000), and
loongson (3A1000).

Prefer copyin32() to copyin() when fetching instructions
from user space. This improves performance slightly.

Discussed with miod@

Use copyin32() instead of a direct memory load when fetching a branch
instruction for branch emulation. This ensures the userspace memory
access is properly guarded and that TLB faults are handled.

In order not to complicate the interface of MipsEmulateBranch(), each
caller now has to provide the branch instruction for the function.

Feedback from miod@

p_comm is the process's command and isn't per thread, so move it from
struct proc to struct process.

ok deraadt@ kettenis@

Rename mips64's trap_frame into trapframe.

For coherency with other archs and in order to use it in MI code.

ok visa@, tobiasu@

Initialize `pc' earlier so that the siginfo pc value is correct in case of
genuine FPU exception.

Access the image of the floating point registers via p_md.md_regs, instead of
directly on the frame, or any updates will get lost. This went unnoticed for
so long because FPU emulation traps usually use the first frame of the U area,
so values kind of stick.

This fixes a rare occurrence of wrong floating-point values with MP kernels
on Octeon.

ok visa@

no need to swizzle load/store addresses for 32bit values on big endian systems

ok miod@

Fix a few format string warnings, allow to build DEBUG kernel on sgi

OK miod@

It's been a quarter century: we can assume volatile is present with that name.

ok dlg@ mpi@ deraadt@

Split ever-growing mips <machine/cpu.h> into what 99% of the kernel needs,
which will remain in <machine/cpu.h>, and a new mips_cpu.h containing only the
goriest md details, which are only of interest to a handful set of files; this
is similar in spirit to what alpha does, but here <machine/cpu.h> does not
include the new file.

Handle the coprocessor 0 cause and status registers as a 64 bit value now,
as some odd mips designs need moro than 32 bits in there. This causes a lot
of mechanical changes everywhere getsr() is used.

Revert art@'s moving around of the KERNEL_LOCK()/KERNEL_UNLOCK() calls,
as it causes hangs in some ports, including libsigsegv's configure script

confirmed by krw@, landry@

There is a bunch of places in the kernel entry points where we don't
hold the kernel lock, but still need call one function that needs it.

Instead of grabbing the lock all over the place, move the locks into
the affected functions: trapsignal, scdebug*, ktrsyscall, ktrsysret,
systrace_redirect and ADDUPROF. In the cases we already hold the biglock
we'll just recurse.

kettenis@, beck@ ok

Clean up after P_BIGLOCK removal.
KERNEL_PROC_LOCK -> KERNEL_LOCK
KERNEL_PROC_UNLOCK -> KERNEL_UNLOCK

oga@ ok

Floating-point emulation code for systems lacking proper FPU (i.e. Octeon),
enabled by option FPUEMUL.

This is pretty straightforward, except for conditional branch on FPU condition
codes emulation (bc1f/bc1fl/bc1t/bc1tl instructions): unlike most
RISC-with-delay-slots designs (m88k, sparc), the branch pipeline is not exposed
to the kernel on Mips, therefore we can not resume a branch without losing the
delay slot instruction.

Some other operating systems work around this issue by emulating the delay
slot instruction, but this is error-prone (and requires the kernel code to
be aware of all supported instructions of the processor it is currently running
on), some use dedicated breakpoints to single-step through the delay slot and
then resume the branch as expected, but this causes a lot of copy-on-write
allocations.

This code chooses a third path, of copying the delay slot instructions to run toa special `magic' page, followed by a special trap instruction to give control
back to the kernel. This makes sure the instruction will actually be run by the
processor, and that no more than one page per process is wasted, regardless of
the number of branches to emulate.

Tested on octeon (big-endian) by syuu@ and on loongson (little-endian) by me.
Note that enabling option FPUEMUL in the kernel will completely disable the
hardware FPU, if there is one; there is currently no way to build a kernel
supporting both hardware and software FPU, and there is no reason to change
this until there is a strong need to support both.

Fix a few logic errors in comparison instruction emulation: make sure the
less than relation is correctly computed, and check for both operands being
signaling NaNs, instead of only the first NaN found, to decide whether to
raise an invalid exception or not.

Replace the old floating point completion code with a C interface to the
MI softfloat code, implementing all MIPS IV specified floating point
operations.
Tested on R5000, R10000, R14000 and Loongson2F.

Make sure that `fsr' is always initialized. Non-DEBUG kernels
have not been affected.

From miod@

Let the kernel utilize the FPU if one is available, even when the
FPUEMUL option is enabled. This benefits OCTEON III systems which can
run floating-point operations natively.

Feedback from and OK miod@; he also helped with testing.

Tested on octeon without FPU (CN5020, CN6120) and with FPU (CN7130),
as well as on sgi/IP27 (MP R16000), sgi/IP32 (R5000), and
loongson (3A1000).

Prefer copyin32() to copyin() when fetching instructions
from user space. This improves performance slightly.

Discussed with miod@

Use copyin32() instead of a direct memory load when fetching a branch
instruction for branch emulation. This ensures the userspace memory
access is properly guarded and that TLB faults are handled.

In order not to complicate the interface of MipsEmulateBranch(), each
caller now has to provide the branch instruction for the function.

Feedback from miod@

p_comm is the process's command and isn't per thread, so move it from
struct proc to struct process.

ok deraadt@ kettenis@

Rename mips64's trap_frame into trapframe.

For coherency with other archs and in order to use it in MI code.

ok visa@, tobiasu@

Initialize `pc' earlier so that the siginfo pc value is correct in case of
genuine FPU exception.

Access the image of the floating point registers via p_md.md_regs, instead of
directly on the frame, or any updates will get lost. This went unnoticed for
so long because FPU emulation traps usually use the first frame of the U area,
so values kind of stick.

This fixes a rare occurrence of wrong floating-point values with MP kernels
on Octeon.

ok visa@

no need to swizzle load/store addresses for 32bit values on big endian systems

ok miod@

Fix a few format string warnings, allow to build DEBUG kernel on sgi

OK miod@

It's been a quarter century: we can assume volatile is present with that name.

ok dlg@ mpi@ deraadt@

Split ever-growing mips <machine/cpu.h> into what 99% of the kernel needs,
which will remain in <machine/cpu.h>, and a new mips_cpu.h containing only the
goriest md details, which are only of interest to a handful set of files; this
is similar in spirit to what alpha does, but here <machine/cpu.h> does not
include the new file.

Handle the coprocessor 0 cause and status registers as a 64 bit value now,
as some odd mips designs need moro than 32 bits in there. This causes a lot
of mechanical changes everywhere getsr() is used.

Revert art@'s moving around of the KERNEL_LOCK()/KERNEL_UNLOCK() calls,
as it causes hangs in some ports, including libsigsegv's configure script

confirmed by krw@, landry@

There is a bunch of places in the kernel entry points where we don't
hold the kernel lock, but still need call one function that needs it.

Instead of grabbing the lock all over the place, move the locks into
the affected functions: trapsignal, scdebug*, ktrsyscall, ktrsysret,
systrace_redirect and ADDUPROF. In the cases we already hold the biglock
we'll just recurse.

kettenis@, beck@ ok

Clean up after P_BIGLOCK removal.
KERNEL_PROC_LOCK -> KERNEL_LOCK
KERNEL_PROC_UNLOCK -> KERNEL_UNLOCK

oga@ ok

Floating-point emulation code for systems lacking proper FPU (i.e. Octeon),
enabled by option FPUEMUL.

This is pretty straightforward, except for conditional branch on FPU condition
codes emulation (bc1f/bc1fl/bc1t/bc1tl instructions): unlike most
RISC-with-delay-slots designs (m88k, sparc), the branch pipeline is not exposed
to the kernel on Mips, therefore we can not resume a branch without losing the
delay slot instruction.

Some other operating systems work around this issue by emulating the delay
slot instruction, but this is error-prone (and requires the kernel code to
be aware of all supported instructions of the processor it is currently running
on), some use dedicated breakpoints to single-step through the delay slot and
then resume the branch as expected, but this causes a lot of copy-on-write
allocations.

This code chooses a third path, of copying the delay slot instructions to run toa special `magic' page, followed by a special trap instruction to give control
back to the kernel. This makes sure the instruction will actually be run by the
processor, and that no more than one page per process is wasted, regardless of
the number of branches to emulate.

Tested on octeon (big-endian) by syuu@ and on loongson (little-endian) by me.
Note that enabling option FPUEMUL in the kernel will completely disable the
hardware FPU, if there is one; there is currently no way to build a kernel
supporting both hardware and software FPU, and there is no reason to change
this until there is a strong need to support both.

Fix a few logic errors in comparison instruction emulation: make sure the
less than relation is correctly computed, and check for both operands being
signaling NaNs, instead of only the first NaN found, to decide whether to
raise an invalid exception or not.

Replace the old floating point completion code with a C interface to the
MI softfloat code, implementing all MIPS IV specified floating point
operations.
Tested on R5000, R10000, R14000 and Loongson2F.