Enable EPAN if it is available.

ok patrick@

Decode remaining ID_AA64ISAR1_EL1 features.

ok jsg@

Add support for the new layout of the CCSIDR_EL1 register that was
introduced in Armv8.3 when the CCIDX feature is advertised. This
makes us properly detect the cache size on newer CPU cores like
Neoverse N2, at least when emulated by QEMU.

ok jsg@

The feature is called SSBS instead of SBSS.

Tighten up BTCFI by flipping the bits that make PACIASP and PACIBSP
behave like BTI c instead of BTI jc.

ok deraadt@, tobhe@

Fix unwanted sign-extension of ID register masks. Sign-extension of the
GPI feature mask caused misdetection of the GPI feature when some other
feature was present that was advertised in the upper 32 bits of the same
ID register. Resulting in a crash as soon as the pmap code tried to set
the PAC keys.

Fix suggested by Marc Zyngier who found and debugged the problem.

ok jsg@, deraadt@

Implement support for pointer authentication (PAC) in userland. With PAC
it is possible to "sign" pointers with a hidden key. The signature is
placed in unused bits of the pointer and can be checked later. This can
be used to provide "tail CFI" that is similar to what retguard provides.

Debuggers need to be aware of the fact that pointers can be signed. For
this purpose a new PT_PACMASK ptrace(2) request is introduced that returns
as mask that indicates the bits used for the signature. Separate masks
are provided for code and data pointers even though the masks are identical
in the current implementation. These masks are also written into a special
note section in the core dump.

ok patrick@

Clear BTYPE bits when setting up a signal handler and when handling a
PT_CONTINUE ptrace(2) request. Otherwise we would trap if userland was
interrupted at a point where it is doing an indirect branch that has set
the bits but before it has executed the BTI instruction at the branch
target.

The PT_SETREGS request may need similar treatment, at least when the
PC is changed. But Linux doesn't do this and debuggers might want full
control over the BTYPE bits. So leave this alone for now.

ok guenther@

Implement branch target protection using the branch target identification
feature introduced in Armv8.5. This provides "head-CFI" to complement
the "tail-CFI" provided by retguard. Unfortunately most arm64 machines
don't support this feature yet. But Apple M2 does support it and it
seems to work there.

ok deraadt@

Add detection of EPAN feature bit.

ok deraadt@

Add detection for Spectre-BHB related CLRBHB, ECBHB and CSV2_3/HCXT
feature bits.

ok kettenis@

Document a few more ID register bits. This should add most of the stuff
in ARMv8.5 as far as the ISAR and PFR registers are concerned.

ok deraadt@

Move definitions for CNTV_CTL_EL0 to armreg.h.

ok mpi@, jsg@, phessler@, patrick@

Add a few more PSTATE bits.

ok deraadt@

use ansi volatile keyword, not __volatile
ok miod@ guenther@

Add DIT CPU feature.

ok beck@

slignment -> alignment

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Fix the TCR_TG0_xxx definitions and add TCR_TG0_4K to the initial setting
of TCR_EL1 in locore to make clear we use 4K pages for both userland and
the kernel.

ok patrick@

Make sure that all CPUs end up with the same bits set in SCTLR_EL1.
Do this by clearing all the bits marked RES0 and set all the bits
marked RES1 for the ARMv8.0.

Any optional features introduced in later revisions of the architecture
(such as PAN) will be enabled after SCTLR_EL1 is initialized.

ok patrick@

Add ARMv8.5 instruction set related CPU features.

ok patrick@

spelling

Add code to print CPU features.

ok naddy@

Enable PAN (Privileged Access Never) on CPUs that support it. This means
that user-space access from the kernel is not allowed for "normal"
load/store instructions. Only the special "unprivileged" load/store
instructions are allowed. We already use those in copyin(9) and copyout(9).

ok patrick@, drahn@, jsg@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Cache flush operations on arm64 were being incorrectly treated as write
operations, however they should be treated as read per the design.
Switch to using bit defines, correct said defines.
Fixes cache flushing causing Firefox to abort.
ok kettenis@ kurt@

Newer ARMv8 processors now include a new CSV2 field in their processor
feature register that can indicate that a processor is not vulnarable to
Spectre v2 attacks. Use this field in favour of adding specific processors
to a whitelist. Continue to whitelist the few processors that are known
not to be vulnerable but don't set the appropriate value in the CSV2 field.

ok jsg@

Add the needed ICC_PMR_EL1 register bit defines for the previous
commit to unbreak the build.

from kettenis@
ok drahn@

Add code to print the characteristics of the caches that can be discovered
through the CLIDR_EL1 register.

ok patrick@

Implement single-stepping. Based on an earlier diff from drahn@.
Disable userland debug communication access while there.

ok patrick@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

Adjust OpenBSD/arm64 files with FreeBSD origin to show the upstream
revision. While there, update a few of those files.

Prompted by mikeb@.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Decode remaining ID_AA64ISAR1_EL1 features.

ok jsg@

Add support for the new layout of the CCSIDR_EL1 register that was
introduced in Armv8.3 when the CCIDX feature is advertised. This
makes us properly detect the cache size on newer CPU cores like
Neoverse N2, at least when emulated by QEMU.

ok jsg@

The feature is called SSBS instead of SBSS.

Tighten up BTCFI by flipping the bits that make PACIASP and PACIBSP
behave like BTI c instead of BTI jc.

ok deraadt@, tobhe@

Fix unwanted sign-extension of ID register masks. Sign-extension of the
GPI feature mask caused misdetection of the GPI feature when some other
feature was present that was advertised in the upper 32 bits of the same
ID register. Resulting in a crash as soon as the pmap code tried to set
the PAC keys.

Fix suggested by Marc Zyngier who found and debugged the problem.

ok jsg@, deraadt@

Implement support for pointer authentication (PAC) in userland. With PAC
it is possible to "sign" pointers with a hidden key. The signature is
placed in unused bits of the pointer and can be checked later. This can
be used to provide "tail CFI" that is similar to what retguard provides.

Debuggers need to be aware of the fact that pointers can be signed. For
this purpose a new PT_PACMASK ptrace(2) request is introduced that returns
as mask that indicates the bits used for the signature. Separate masks
are provided for code and data pointers even though the masks are identical
in the current implementation. These masks are also written into a special
note section in the core dump.

ok patrick@

Clear BTYPE bits when setting up a signal handler and when handling a
PT_CONTINUE ptrace(2) request. Otherwise we would trap if userland was
interrupted at a point where it is doing an indirect branch that has set
the bits but before it has executed the BTI instruction at the branch
target.

The PT_SETREGS request may need similar treatment, at least when the
PC is changed. But Linux doesn't do this and debuggers might want full
control over the BTYPE bits. So leave this alone for now.

ok guenther@

Implement branch target protection using the branch target identification
feature introduced in Armv8.5. This provides "head-CFI" to complement
the "tail-CFI" provided by retguard. Unfortunately most arm64 machines
don't support this feature yet. But Apple M2 does support it and it
seems to work there.

ok deraadt@

Add detection of EPAN feature bit.

ok deraadt@

Add detection for Spectre-BHB related CLRBHB, ECBHB and CSV2_3/HCXT
feature bits.

ok kettenis@

Document a few more ID register bits. This should add most of the stuff
in ARMv8.5 as far as the ISAR and PFR registers are concerned.

ok deraadt@

Move definitions for CNTV_CTL_EL0 to armreg.h.

ok mpi@, jsg@, phessler@, patrick@

Add a few more PSTATE bits.

ok deraadt@

use ansi volatile keyword, not __volatile
ok miod@ guenther@

Add DIT CPU feature.

ok beck@

slignment -> alignment

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Fix the TCR_TG0_xxx definitions and add TCR_TG0_4K to the initial setting
of TCR_EL1 in locore to make clear we use 4K pages for both userland and
the kernel.

ok patrick@

Make sure that all CPUs end up with the same bits set in SCTLR_EL1.
Do this by clearing all the bits marked RES0 and set all the bits
marked RES1 for the ARMv8.0.

Any optional features introduced in later revisions of the architecture
(such as PAN) will be enabled after SCTLR_EL1 is initialized.

ok patrick@

Add ARMv8.5 instruction set related CPU features.

ok patrick@

spelling

Add code to print CPU features.

ok naddy@

Enable PAN (Privileged Access Never) on CPUs that support it. This means
that user-space access from the kernel is not allowed for "normal"
load/store instructions. Only the special "unprivileged" load/store
instructions are allowed. We already use those in copyin(9) and copyout(9).

ok patrick@, drahn@, jsg@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Cache flush operations on arm64 were being incorrectly treated as write
operations, however they should be treated as read per the design.
Switch to using bit defines, correct said defines.
Fixes cache flushing causing Firefox to abort.
ok kettenis@ kurt@

Newer ARMv8 processors now include a new CSV2 field in their processor
feature register that can indicate that a processor is not vulnarable to
Spectre v2 attacks. Use this field in favour of adding specific processors
to a whitelist. Continue to whitelist the few processors that are known
not to be vulnerable but don't set the appropriate value in the CSV2 field.

ok jsg@

Add the needed ICC_PMR_EL1 register bit defines for the previous
commit to unbreak the build.

from kettenis@
ok drahn@

Add code to print the characteristics of the caches that can be discovered
through the CLIDR_EL1 register.

ok patrick@

Implement single-stepping. Based on an earlier diff from drahn@.
Disable userland debug communication access while there.

ok patrick@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

Adjust OpenBSD/arm64 files with FreeBSD origin to show the upstream
revision. While there, update a few of those files.

Prompted by mikeb@.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Add support for the new layout of the CCSIDR_EL1 register that was
introduced in Armv8.3 when the CCIDX feature is advertised. This
makes us properly detect the cache size on newer CPU cores like
Neoverse N2, at least when emulated by QEMU.

ok jsg@

The feature is called SSBS instead of SBSS.

Tighten up BTCFI by flipping the bits that make PACIASP and PACIBSP
behave like BTI c instead of BTI jc.

ok deraadt@, tobhe@

Fix unwanted sign-extension of ID register masks. Sign-extension of the
GPI feature mask caused misdetection of the GPI feature when some other
feature was present that was advertised in the upper 32 bits of the same
ID register. Resulting in a crash as soon as the pmap code tried to set
the PAC keys.

Fix suggested by Marc Zyngier who found and debugged the problem.

ok jsg@, deraadt@

Implement support for pointer authentication (PAC) in userland. With PAC
it is possible to "sign" pointers with a hidden key. The signature is
placed in unused bits of the pointer and can be checked later. This can
be used to provide "tail CFI" that is similar to what retguard provides.

Debuggers need to be aware of the fact that pointers can be signed. For
this purpose a new PT_PACMASK ptrace(2) request is introduced that returns
as mask that indicates the bits used for the signature. Separate masks
are provided for code and data pointers even though the masks are identical
in the current implementation. These masks are also written into a special
note section in the core dump.

ok patrick@

Clear BTYPE bits when setting up a signal handler and when handling a
PT_CONTINUE ptrace(2) request. Otherwise we would trap if userland was
interrupted at a point where it is doing an indirect branch that has set
the bits but before it has executed the BTI instruction at the branch
target.

The PT_SETREGS request may need similar treatment, at least when the
PC is changed. But Linux doesn't do this and debuggers might want full
control over the BTYPE bits. So leave this alone for now.

ok guenther@

Implement branch target protection using the branch target identification
feature introduced in Armv8.5. This provides "head-CFI" to complement
the "tail-CFI" provided by retguard. Unfortunately most arm64 machines
don't support this feature yet. But Apple M2 does support it and it
seems to work there.

ok deraadt@

Add detection of EPAN feature bit.

ok deraadt@

Add detection for Spectre-BHB related CLRBHB, ECBHB and CSV2_3/HCXT
feature bits.

ok kettenis@

Document a few more ID register bits. This should add most of the stuff
in ARMv8.5 as far as the ISAR and PFR registers are concerned.

ok deraadt@

Move definitions for CNTV_CTL_EL0 to armreg.h.

ok mpi@, jsg@, phessler@, patrick@

Add a few more PSTATE bits.

ok deraadt@

use ansi volatile keyword, not __volatile
ok miod@ guenther@

Add DIT CPU feature.

ok beck@

slignment -> alignment

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Fix the TCR_TG0_xxx definitions and add TCR_TG0_4K to the initial setting
of TCR_EL1 in locore to make clear we use 4K pages for both userland and
the kernel.

ok patrick@

Make sure that all CPUs end up with the same bits set in SCTLR_EL1.
Do this by clearing all the bits marked RES0 and set all the bits
marked RES1 for the ARMv8.0.

Any optional features introduced in later revisions of the architecture
(such as PAN) will be enabled after SCTLR_EL1 is initialized.

ok patrick@

Add ARMv8.5 instruction set related CPU features.

ok patrick@

spelling

Add code to print CPU features.

ok naddy@

Enable PAN (Privileged Access Never) on CPUs that support it. This means
that user-space access from the kernel is not allowed for "normal"
load/store instructions. Only the special "unprivileged" load/store
instructions are allowed. We already use those in copyin(9) and copyout(9).

ok patrick@, drahn@, jsg@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Cache flush operations on arm64 were being incorrectly treated as write
operations, however they should be treated as read per the design.
Switch to using bit defines, correct said defines.
Fixes cache flushing causing Firefox to abort.
ok kettenis@ kurt@

Newer ARMv8 processors now include a new CSV2 field in their processor
feature register that can indicate that a processor is not vulnarable to
Spectre v2 attacks. Use this field in favour of adding specific processors
to a whitelist. Continue to whitelist the few processors that are known
not to be vulnerable but don't set the appropriate value in the CSV2 field.

ok jsg@

Add the needed ICC_PMR_EL1 register bit defines for the previous
commit to unbreak the build.

from kettenis@
ok drahn@

Add code to print the characteristics of the caches that can be discovered
through the CLIDR_EL1 register.

ok patrick@

Implement single-stepping. Based on an earlier diff from drahn@.
Disable userland debug communication access while there.

ok patrick@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

Adjust OpenBSD/arm64 files with FreeBSD origin to show the upstream
revision. While there, update a few of those files.

Prompted by mikeb@.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

The feature is called SSBS instead of SBSS.

Tighten up BTCFI by flipping the bits that make PACIASP and PACIBSP
behave like BTI c instead of BTI jc.

ok deraadt@, tobhe@

Fix unwanted sign-extension of ID register masks. Sign-extension of the
GPI feature mask caused misdetection of the GPI feature when some other
feature was present that was advertised in the upper 32 bits of the same
ID register. Resulting in a crash as soon as the pmap code tried to set
the PAC keys.

Fix suggested by Marc Zyngier who found and debugged the problem.

ok jsg@, deraadt@

Implement support for pointer authentication (PAC) in userland. With PAC
it is possible to "sign" pointers with a hidden key. The signature is
placed in unused bits of the pointer and can be checked later. This can
be used to provide "tail CFI" that is similar to what retguard provides.

Debuggers need to be aware of the fact that pointers can be signed. For
this purpose a new PT_PACMASK ptrace(2) request is introduced that returns
as mask that indicates the bits used for the signature. Separate masks
are provided for code and data pointers even though the masks are identical
in the current implementation. These masks are also written into a special
note section in the core dump.

ok patrick@

Clear BTYPE bits when setting up a signal handler and when handling a
PT_CONTINUE ptrace(2) request. Otherwise we would trap if userland was
interrupted at a point where it is doing an indirect branch that has set
the bits but before it has executed the BTI instruction at the branch
target.

The PT_SETREGS request may need similar treatment, at least when the
PC is changed. But Linux doesn't do this and debuggers might want full
control over the BTYPE bits. So leave this alone for now.

ok guenther@

Implement branch target protection using the branch target identification
feature introduced in Armv8.5. This provides "head-CFI" to complement
the "tail-CFI" provided by retguard. Unfortunately most arm64 machines
don't support this feature yet. But Apple M2 does support it and it
seems to work there.

ok deraadt@

Add detection of EPAN feature bit.

ok deraadt@

Add detection for Spectre-BHB related CLRBHB, ECBHB and CSV2_3/HCXT
feature bits.

ok kettenis@

Document a few more ID register bits. This should add most of the stuff
in ARMv8.5 as far as the ISAR and PFR registers are concerned.

ok deraadt@

Move definitions for CNTV_CTL_EL0 to armreg.h.

ok mpi@, jsg@, phessler@, patrick@

Add a few more PSTATE bits.

ok deraadt@

use ansi volatile keyword, not __volatile
ok miod@ guenther@

Add DIT CPU feature.

ok beck@

slignment -> alignment

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Fix the TCR_TG0_xxx definitions and add TCR_TG0_4K to the initial setting
of TCR_EL1 in locore to make clear we use 4K pages for both userland and
the kernel.

ok patrick@

Make sure that all CPUs end up with the same bits set in SCTLR_EL1.
Do this by clearing all the bits marked RES0 and set all the bits
marked RES1 for the ARMv8.0.

Any optional features introduced in later revisions of the architecture
(such as PAN) will be enabled after SCTLR_EL1 is initialized.

ok patrick@

Add ARMv8.5 instruction set related CPU features.

ok patrick@

spelling

Add code to print CPU features.

ok naddy@

Enable PAN (Privileged Access Never) on CPUs that support it. This means
that user-space access from the kernel is not allowed for "normal"
load/store instructions. Only the special "unprivileged" load/store
instructions are allowed. We already use those in copyin(9) and copyout(9).

ok patrick@, drahn@, jsg@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Cache flush operations on arm64 were being incorrectly treated as write
operations, however they should be treated as read per the design.
Switch to using bit defines, correct said defines.
Fixes cache flushing causing Firefox to abort.
ok kettenis@ kurt@

Newer ARMv8 processors now include a new CSV2 field in their processor
feature register that can indicate that a processor is not vulnarable to
Spectre v2 attacks. Use this field in favour of adding specific processors
to a whitelist. Continue to whitelist the few processors that are known
not to be vulnerable but don't set the appropriate value in the CSV2 field.

ok jsg@

Add the needed ICC_PMR_EL1 register bit defines for the previous
commit to unbreak the build.

from kettenis@
ok drahn@

Add code to print the characteristics of the caches that can be discovered
through the CLIDR_EL1 register.

ok patrick@

Implement single-stepping. Based on an earlier diff from drahn@.
Disable userland debug communication access while there.

ok patrick@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

Adjust OpenBSD/arm64 files with FreeBSD origin to show the upstream
revision. While there, update a few of those files.

Prompted by mikeb@.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Tighten up BTCFI by flipping the bits that make PACIASP and PACIBSP
behave like BTI c instead of BTI jc.

ok deraadt@, tobhe@

Fix unwanted sign-extension of ID register masks. Sign-extension of the
GPI feature mask caused misdetection of the GPI feature when some other
feature was present that was advertised in the upper 32 bits of the same
ID register. Resulting in a crash as soon as the pmap code tried to set
the PAC keys.

Fix suggested by Marc Zyngier who found and debugged the problem.

ok jsg@, deraadt@

Implement support for pointer authentication (PAC) in userland. With PAC
it is possible to "sign" pointers with a hidden key. The signature is
placed in unused bits of the pointer and can be checked later. This can
be used to provide "tail CFI" that is similar to what retguard provides.

Debuggers need to be aware of the fact that pointers can be signed. For
this purpose a new PT_PACMASK ptrace(2) request is introduced that returns
as mask that indicates the bits used for the signature. Separate masks
are provided for code and data pointers even though the masks are identical
in the current implementation. These masks are also written into a special
note section in the core dump.

ok patrick@

Clear BTYPE bits when setting up a signal handler and when handling a
PT_CONTINUE ptrace(2) request. Otherwise we would trap if userland was
interrupted at a point where it is doing an indirect branch that has set
the bits but before it has executed the BTI instruction at the branch
target.

The PT_SETREGS request may need similar treatment, at least when the
PC is changed. But Linux doesn't do this and debuggers might want full
control over the BTYPE bits. So leave this alone for now.

ok guenther@

Implement branch target protection using the branch target identification
feature introduced in Armv8.5. This provides "head-CFI" to complement
the "tail-CFI" provided by retguard. Unfortunately most arm64 machines
don't support this feature yet. But Apple M2 does support it and it
seems to work there.

ok deraadt@

Add detection of EPAN feature bit.

ok deraadt@

Add detection for Spectre-BHB related CLRBHB, ECBHB and CSV2_3/HCXT
feature bits.

ok kettenis@

Document a few more ID register bits. This should add most of the stuff
in ARMv8.5 as far as the ISAR and PFR registers are concerned.

ok deraadt@

Move definitions for CNTV_CTL_EL0 to armreg.h.

ok mpi@, jsg@, phessler@, patrick@

Add a few more PSTATE bits.

ok deraadt@

use ansi volatile keyword, not __volatile
ok miod@ guenther@

Add DIT CPU feature.

ok beck@

slignment -> alignment

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Fix the TCR_TG0_xxx definitions and add TCR_TG0_4K to the initial setting
of TCR_EL1 in locore to make clear we use 4K pages for both userland and
the kernel.

ok patrick@

Make sure that all CPUs end up with the same bits set in SCTLR_EL1.
Do this by clearing all the bits marked RES0 and set all the bits
marked RES1 for the ARMv8.0.

Any optional features introduced in later revisions of the architecture
(such as PAN) will be enabled after SCTLR_EL1 is initialized.

ok patrick@

Add ARMv8.5 instruction set related CPU features.

ok patrick@

spelling

Add code to print CPU features.

ok naddy@

Enable PAN (Privileged Access Never) on CPUs that support it. This means
that user-space access from the kernel is not allowed for "normal"
load/store instructions. Only the special "unprivileged" load/store
instructions are allowed. We already use those in copyin(9) and copyout(9).

ok patrick@, drahn@, jsg@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Cache flush operations on arm64 were being incorrectly treated as write
operations, however they should be treated as read per the design.
Switch to using bit defines, correct said defines.
Fixes cache flushing causing Firefox to abort.
ok kettenis@ kurt@

Newer ARMv8 processors now include a new CSV2 field in their processor
feature register that can indicate that a processor is not vulnarable to
Spectre v2 attacks. Use this field in favour of adding specific processors
to a whitelist. Continue to whitelist the few processors that are known
not to be vulnerable but don't set the appropriate value in the CSV2 field.

ok jsg@

Add the needed ICC_PMR_EL1 register bit defines for the previous
commit to unbreak the build.

from kettenis@
ok drahn@

Add code to print the characteristics of the caches that can be discovered
through the CLIDR_EL1 register.

ok patrick@

Implement single-stepping. Based on an earlier diff from drahn@.
Disable userland debug communication access while there.

ok patrick@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

Adjust OpenBSD/arm64 files with FreeBSD origin to show the upstream
revision. While there, update a few of those files.

Prompted by mikeb@.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Fix unwanted sign-extension of ID register masks. Sign-extension of the
GPI feature mask caused misdetection of the GPI feature when some other
feature was present that was advertised in the upper 32 bits of the same
ID register. Resulting in a crash as soon as the pmap code tried to set
the PAC keys.

Fix suggested by Marc Zyngier who found and debugged the problem.

ok jsg@, deraadt@

Implement support for pointer authentication (PAC) in userland. With PAC
it is possible to "sign" pointers with a hidden key. The signature is
placed in unused bits of the pointer and can be checked later. This can
be used to provide "tail CFI" that is similar to what retguard provides.

Debuggers need to be aware of the fact that pointers can be signed. For
this purpose a new PT_PACMASK ptrace(2) request is introduced that returns
as mask that indicates the bits used for the signature. Separate masks
are provided for code and data pointers even though the masks are identical
in the current implementation. These masks are also written into a special
note section in the core dump.

ok patrick@

Clear BTYPE bits when setting up a signal handler and when handling a
PT_CONTINUE ptrace(2) request. Otherwise we would trap if userland was
interrupted at a point where it is doing an indirect branch that has set
the bits but before it has executed the BTI instruction at the branch
target.

The PT_SETREGS request may need similar treatment, at least when the
PC is changed. But Linux doesn't do this and debuggers might want full
control over the BTYPE bits. So leave this alone for now.

ok guenther@

Implement branch target protection using the branch target identification
feature introduced in Armv8.5. This provides "head-CFI" to complement
the "tail-CFI" provided by retguard. Unfortunately most arm64 machines
don't support this feature yet. But Apple M2 does support it and it
seems to work there.

ok deraadt@

Add detection of EPAN feature bit.

ok deraadt@

Add detection for Spectre-BHB related CLRBHB, ECBHB and CSV2_3/HCXT
feature bits.

ok kettenis@

Document a few more ID register bits. This should add most of the stuff
in ARMv8.5 as far as the ISAR and PFR registers are concerned.

ok deraadt@

Move definitions for CNTV_CTL_EL0 to armreg.h.

ok mpi@, jsg@, phessler@, patrick@

Add a few more PSTATE bits.

ok deraadt@

use ansi volatile keyword, not __volatile
ok miod@ guenther@

Add DIT CPU feature.

ok beck@

slignment -> alignment

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Fix the TCR_TG0_xxx definitions and add TCR_TG0_4K to the initial setting
of TCR_EL1 in locore to make clear we use 4K pages for both userland and
the kernel.

ok patrick@

Make sure that all CPUs end up with the same bits set in SCTLR_EL1.
Do this by clearing all the bits marked RES0 and set all the bits
marked RES1 for the ARMv8.0.

Any optional features introduced in later revisions of the architecture
(such as PAN) will be enabled after SCTLR_EL1 is initialized.

ok patrick@

Add ARMv8.5 instruction set related CPU features.

ok patrick@

spelling

Add code to print CPU features.

ok naddy@

Enable PAN (Privileged Access Never) on CPUs that support it. This means
that user-space access from the kernel is not allowed for "normal"
load/store instructions. Only the special "unprivileged" load/store
instructions are allowed. We already use those in copyin(9) and copyout(9).

ok patrick@, drahn@, jsg@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Cache flush operations on arm64 were being incorrectly treated as write
operations, however they should be treated as read per the design.
Switch to using bit defines, correct said defines.
Fixes cache flushing causing Firefox to abort.
ok kettenis@ kurt@

Newer ARMv8 processors now include a new CSV2 field in their processor
feature register that can indicate that a processor is not vulnarable to
Spectre v2 attacks. Use this field in favour of adding specific processors
to a whitelist. Continue to whitelist the few processors that are known
not to be vulnerable but don't set the appropriate value in the CSV2 field.

ok jsg@

Add the needed ICC_PMR_EL1 register bit defines for the previous
commit to unbreak the build.

from kettenis@
ok drahn@

Add code to print the characteristics of the caches that can be discovered
through the CLIDR_EL1 register.

ok patrick@

Implement single-stepping. Based on an earlier diff from drahn@.
Disable userland debug communication access while there.

ok patrick@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

Adjust OpenBSD/arm64 files with FreeBSD origin to show the upstream
revision. While there, update a few of those files.

Prompted by mikeb@.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Implement support for pointer authentication (PAC) in userland. With PAC
it is possible to "sign" pointers with a hidden key. The signature is
placed in unused bits of the pointer and can be checked later. This can
be used to provide "tail CFI" that is similar to what retguard provides.

Debuggers need to be aware of the fact that pointers can be signed. For
this purpose a new PT_PACMASK ptrace(2) request is introduced that returns
as mask that indicates the bits used for the signature. Separate masks
are provided for code and data pointers even though the masks are identical
in the current implementation. These masks are also written into a special
note section in the core dump.

ok patrick@

Clear BTYPE bits when setting up a signal handler and when handling a
PT_CONTINUE ptrace(2) request. Otherwise we would trap if userland was
interrupted at a point where it is doing an indirect branch that has set
the bits but before it has executed the BTI instruction at the branch
target.

The PT_SETREGS request may need similar treatment, at least when the
PC is changed. But Linux doesn't do this and debuggers might want full
control over the BTYPE bits. So leave this alone for now.

ok guenther@

Implement branch target protection using the branch target identification
feature introduced in Armv8.5. This provides "head-CFI" to complement
the "tail-CFI" provided by retguard. Unfortunately most arm64 machines
don't support this feature yet. But Apple M2 does support it and it
seems to work there.

ok deraadt@

Add detection of EPAN feature bit.

ok deraadt@

Add detection for Spectre-BHB related CLRBHB, ECBHB and CSV2_3/HCXT
feature bits.

ok kettenis@

Document a few more ID register bits. This should add most of the stuff
in ARMv8.5 as far as the ISAR and PFR registers are concerned.

ok deraadt@

Move definitions for CNTV_CTL_EL0 to armreg.h.

ok mpi@, jsg@, phessler@, patrick@

Add a few more PSTATE bits.

ok deraadt@

use ansi volatile keyword, not __volatile
ok miod@ guenther@

Add DIT CPU feature.

ok beck@

slignment -> alignment

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Fix the TCR_TG0_xxx definitions and add TCR_TG0_4K to the initial setting
of TCR_EL1 in locore to make clear we use 4K pages for both userland and
the kernel.

ok patrick@

Make sure that all CPUs end up with the same bits set in SCTLR_EL1.
Do this by clearing all the bits marked RES0 and set all the bits
marked RES1 for the ARMv8.0.

Any optional features introduced in later revisions of the architecture
(such as PAN) will be enabled after SCTLR_EL1 is initialized.

ok patrick@

Add ARMv8.5 instruction set related CPU features.

ok patrick@

spelling

Add code to print CPU features.

ok naddy@

Enable PAN (Privileged Access Never) on CPUs that support it. This means
that user-space access from the kernel is not allowed for "normal"
load/store instructions. Only the special "unprivileged" load/store
instructions are allowed. We already use those in copyin(9) and copyout(9).

ok patrick@, drahn@, jsg@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Cache flush operations on arm64 were being incorrectly treated as write
operations, however they should be treated as read per the design.
Switch to using bit defines, correct said defines.
Fixes cache flushing causing Firefox to abort.
ok kettenis@ kurt@

Newer ARMv8 processors now include a new CSV2 field in their processor
feature register that can indicate that a processor is not vulnarable to
Spectre v2 attacks. Use this field in favour of adding specific processors
to a whitelist. Continue to whitelist the few processors that are known
not to be vulnerable but don't set the appropriate value in the CSV2 field.

ok jsg@

Add the needed ICC_PMR_EL1 register bit defines for the previous
commit to unbreak the build.

from kettenis@
ok drahn@

Add code to print the characteristics of the caches that can be discovered
through the CLIDR_EL1 register.

ok patrick@

Implement single-stepping. Based on an earlier diff from drahn@.
Disable userland debug communication access while there.

ok patrick@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

Adjust OpenBSD/arm64 files with FreeBSD origin to show the upstream
revision. While there, update a few of those files.

Prompted by mikeb@.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Clear BTYPE bits when setting up a signal handler and when handling a
PT_CONTINUE ptrace(2) request. Otherwise we would trap if userland was
interrupted at a point where it is doing an indirect branch that has set
the bits but before it has executed the BTI instruction at the branch
target.

The PT_SETREGS request may need similar treatment, at least when the
PC is changed. But Linux doesn't do this and debuggers might want full
control over the BTYPE bits. So leave this alone for now.

ok guenther@

Implement branch target protection using the branch target identification
feature introduced in Armv8.5. This provides "head-CFI" to complement
the "tail-CFI" provided by retguard. Unfortunately most arm64 machines
don't support this feature yet. But Apple M2 does support it and it
seems to work there.

ok deraadt@

Add detection of EPAN feature bit.

ok deraadt@

Add detection for Spectre-BHB related CLRBHB, ECBHB and CSV2_3/HCXT
feature bits.

ok kettenis@

Document a few more ID register bits. This should add most of the stuff
in ARMv8.5 as far as the ISAR and PFR registers are concerned.

ok deraadt@

Move definitions for CNTV_CTL_EL0 to armreg.h.

ok mpi@, jsg@, phessler@, patrick@

Add a few more PSTATE bits.

ok deraadt@

use ansi volatile keyword, not __volatile
ok miod@ guenther@

Add DIT CPU feature.

ok beck@

slignment -> alignment

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Fix the TCR_TG0_xxx definitions and add TCR_TG0_4K to the initial setting
of TCR_EL1 in locore to make clear we use 4K pages for both userland and
the kernel.

ok patrick@

Make sure that all CPUs end up with the same bits set in SCTLR_EL1.
Do this by clearing all the bits marked RES0 and set all the bits
marked RES1 for the ARMv8.0.

Any optional features introduced in later revisions of the architecture
(such as PAN) will be enabled after SCTLR_EL1 is initialized.

ok patrick@

Add ARMv8.5 instruction set related CPU features.

ok patrick@

spelling

Add code to print CPU features.

ok naddy@

Enable PAN (Privileged Access Never) on CPUs that support it. This means
that user-space access from the kernel is not allowed for "normal"
load/store instructions. Only the special "unprivileged" load/store
instructions are allowed. We already use those in copyin(9) and copyout(9).

ok patrick@, drahn@, jsg@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Cache flush operations on arm64 were being incorrectly treated as write
operations, however they should be treated as read per the design.
Switch to using bit defines, correct said defines.
Fixes cache flushing causing Firefox to abort.
ok kettenis@ kurt@

Newer ARMv8 processors now include a new CSV2 field in their processor
feature register that can indicate that a processor is not vulnarable to
Spectre v2 attacks. Use this field in favour of adding specific processors
to a whitelist. Continue to whitelist the few processors that are known
not to be vulnerable but don't set the appropriate value in the CSV2 field.

ok jsg@

Add the needed ICC_PMR_EL1 register bit defines for the previous
commit to unbreak the build.

from kettenis@
ok drahn@

Add code to print the characteristics of the caches that can be discovered
through the CLIDR_EL1 register.

ok patrick@

Implement single-stepping. Based on an earlier diff from drahn@.
Disable userland debug communication access while there.

ok patrick@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

Adjust OpenBSD/arm64 files with FreeBSD origin to show the upstream
revision. While there, update a few of those files.

Prompted by mikeb@.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Add detection of EPAN feature bit.

ok deraadt@

Add detection for Spectre-BHB related CLRBHB, ECBHB and CSV2_3/HCXT
feature bits.

ok kettenis@

Document a few more ID register bits. This should add most of the stuff
in ARMv8.5 as far as the ISAR and PFR registers are concerned.

ok deraadt@

Move definitions for CNTV_CTL_EL0 to armreg.h.

ok mpi@, jsg@, phessler@, patrick@

Add a few more PSTATE bits.

ok deraadt@

use ansi volatile keyword, not __volatile
ok miod@ guenther@

Add DIT CPU feature.

ok beck@

slignment -> alignment

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Fix the TCR_TG0_xxx definitions and add TCR_TG0_4K to the initial setting
of TCR_EL1 in locore to make clear we use 4K pages for both userland and
the kernel.

ok patrick@

Make sure that all CPUs end up with the same bits set in SCTLR_EL1.
Do this by clearing all the bits marked RES0 and set all the bits
marked RES1 for the ARMv8.0.

Any optional features introduced in later revisions of the architecture
(such as PAN) will be enabled after SCTLR_EL1 is initialized.

ok patrick@

Add ARMv8.5 instruction set related CPU features.

ok patrick@

spelling

Add code to print CPU features.

ok naddy@

Enable PAN (Privileged Access Never) on CPUs that support it. This means
that user-space access from the kernel is not allowed for "normal"
load/store instructions. Only the special "unprivileged" load/store
instructions are allowed. We already use those in copyin(9) and copyout(9).

ok patrick@, drahn@, jsg@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Cache flush operations on arm64 were being incorrectly treated as write
operations, however they should be treated as read per the design.
Switch to using bit defines, correct said defines.
Fixes cache flushing causing Firefox to abort.
ok kettenis@ kurt@

Newer ARMv8 processors now include a new CSV2 field in their processor
feature register that can indicate that a processor is not vulnarable to
Spectre v2 attacks. Use this field in favour of adding specific processors
to a whitelist. Continue to whitelist the few processors that are known
not to be vulnerable but don't set the appropriate value in the CSV2 field.

ok jsg@

Add the needed ICC_PMR_EL1 register bit defines for the previous
commit to unbreak the build.

from kettenis@
ok drahn@

Add code to print the characteristics of the caches that can be discovered
through the CLIDR_EL1 register.

ok patrick@

Implement single-stepping. Based on an earlier diff from drahn@.
Disable userland debug communication access while there.

ok patrick@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

Adjust OpenBSD/arm64 files with FreeBSD origin to show the upstream
revision. While there, update a few of those files.

Prompted by mikeb@.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Add detection for Spectre-BHB related CLRBHB, ECBHB and CSV2_3/HCXT
feature bits.

ok kettenis@

Document a few more ID register bits. This should add most of the stuff
in ARMv8.5 as far as the ISAR and PFR registers are concerned.

ok deraadt@

Move definitions for CNTV_CTL_EL0 to armreg.h.

ok mpi@, jsg@, phessler@, patrick@

Add a few more PSTATE bits.

ok deraadt@

use ansi volatile keyword, not __volatile
ok miod@ guenther@

Add DIT CPU feature.

ok beck@

slignment -> alignment

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Fix the TCR_TG0_xxx definitions and add TCR_TG0_4K to the initial setting
of TCR_EL1 in locore to make clear we use 4K pages for both userland and
the kernel.

ok patrick@

Make sure that all CPUs end up with the same bits set in SCTLR_EL1.
Do this by clearing all the bits marked RES0 and set all the bits
marked RES1 for the ARMv8.0.

Any optional features introduced in later revisions of the architecture
(such as PAN) will be enabled after SCTLR_EL1 is initialized.

ok patrick@

Add ARMv8.5 instruction set related CPU features.

ok patrick@

spelling

Add code to print CPU features.

ok naddy@

Enable PAN (Privileged Access Never) on CPUs that support it. This means
that user-space access from the kernel is not allowed for "normal"
load/store instructions. Only the special "unprivileged" load/store
instructions are allowed. We already use those in copyin(9) and copyout(9).

ok patrick@, drahn@, jsg@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Cache flush operations on arm64 were being incorrectly treated as write
operations, however they should be treated as read per the design.
Switch to using bit defines, correct said defines.
Fixes cache flushing causing Firefox to abort.
ok kettenis@ kurt@

Newer ARMv8 processors now include a new CSV2 field in their processor
feature register that can indicate that a processor is not vulnarable to
Spectre v2 attacks. Use this field in favour of adding specific processors
to a whitelist. Continue to whitelist the few processors that are known
not to be vulnerable but don't set the appropriate value in the CSV2 field.

ok jsg@

Add the needed ICC_PMR_EL1 register bit defines for the previous
commit to unbreak the build.

from kettenis@
ok drahn@

Add code to print the characteristics of the caches that can be discovered
through the CLIDR_EL1 register.

ok patrick@

Implement single-stepping. Based on an earlier diff from drahn@.
Disable userland debug communication access while there.

ok patrick@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

Adjust OpenBSD/arm64 files with FreeBSD origin to show the upstream
revision. While there, update a few of those files.

Prompted by mikeb@.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Document a few more ID register bits. This should add most of the stuff
in ARMv8.5 as far as the ISAR and PFR registers are concerned.

ok deraadt@

Move definitions for CNTV_CTL_EL0 to armreg.h.

ok mpi@, jsg@, phessler@, patrick@

Add a few more PSTATE bits.

ok deraadt@

use ansi volatile keyword, not __volatile
ok miod@ guenther@

Add DIT CPU feature.

ok beck@

slignment -> alignment

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Fix the TCR_TG0_xxx definitions and add TCR_TG0_4K to the initial setting
of TCR_EL1 in locore to make clear we use 4K pages for both userland and
the kernel.

ok patrick@

Make sure that all CPUs end up with the same bits set in SCTLR_EL1.
Do this by clearing all the bits marked RES0 and set all the bits
marked RES1 for the ARMv8.0.

Any optional features introduced in later revisions of the architecture
(such as PAN) will be enabled after SCTLR_EL1 is initialized.

ok patrick@

Add ARMv8.5 instruction set related CPU features.

ok patrick@

spelling

Add code to print CPU features.

ok naddy@

Enable PAN (Privileged Access Never) on CPUs that support it. This means
that user-space access from the kernel is not allowed for "normal"
load/store instructions. Only the special "unprivileged" load/store
instructions are allowed. We already use those in copyin(9) and copyout(9).

ok patrick@, drahn@, jsg@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Cache flush operations on arm64 were being incorrectly treated as write
operations, however they should be treated as read per the design.
Switch to using bit defines, correct said defines.
Fixes cache flushing causing Firefox to abort.
ok kettenis@ kurt@

Newer ARMv8 processors now include a new CSV2 field in their processor
feature register that can indicate that a processor is not vulnarable to
Spectre v2 attacks. Use this field in favour of adding specific processors
to a whitelist. Continue to whitelist the few processors that are known
not to be vulnerable but don't set the appropriate value in the CSV2 field.

ok jsg@

Add the needed ICC_PMR_EL1 register bit defines for the previous
commit to unbreak the build.

from kettenis@
ok drahn@

Add code to print the characteristics of the caches that can be discovered
through the CLIDR_EL1 register.

ok patrick@

Implement single-stepping. Based on an earlier diff from drahn@.
Disable userland debug communication access while there.

ok patrick@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

Adjust OpenBSD/arm64 files with FreeBSD origin to show the upstream
revision. While there, update a few of those files.

Prompted by mikeb@.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Move definitions for CNTV_CTL_EL0 to armreg.h.

ok mpi@, jsg@, phessler@, patrick@

Add a few more PSTATE bits.

ok deraadt@

use ansi volatile keyword, not __volatile
ok miod@ guenther@

Add DIT CPU feature.

ok beck@

slignment -> alignment

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Fix the TCR_TG0_xxx definitions and add TCR_TG0_4K to the initial setting
of TCR_EL1 in locore to make clear we use 4K pages for both userland and
the kernel.

ok patrick@

Make sure that all CPUs end up with the same bits set in SCTLR_EL1.
Do this by clearing all the bits marked RES0 and set all the bits
marked RES1 for the ARMv8.0.

Any optional features introduced in later revisions of the architecture
(such as PAN) will be enabled after SCTLR_EL1 is initialized.

ok patrick@

Add ARMv8.5 instruction set related CPU features.

ok patrick@

spelling

Add code to print CPU features.

ok naddy@

Enable PAN (Privileged Access Never) on CPUs that support it. This means
that user-space access from the kernel is not allowed for "normal"
load/store instructions. Only the special "unprivileged" load/store
instructions are allowed. We already use those in copyin(9) and copyout(9).

ok patrick@, drahn@, jsg@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Cache flush operations on arm64 were being incorrectly treated as write
operations, however they should be treated as read per the design.
Switch to using bit defines, correct said defines.
Fixes cache flushing causing Firefox to abort.
ok kettenis@ kurt@

Newer ARMv8 processors now include a new CSV2 field in their processor
feature register that can indicate that a processor is not vulnarable to
Spectre v2 attacks. Use this field in favour of adding specific processors
to a whitelist. Continue to whitelist the few processors that are known
not to be vulnerable but don't set the appropriate value in the CSV2 field.

ok jsg@

Add the needed ICC_PMR_EL1 register bit defines for the previous
commit to unbreak the build.

from kettenis@
ok drahn@

Add code to print the characteristics of the caches that can be discovered
through the CLIDR_EL1 register.

ok patrick@

Implement single-stepping. Based on an earlier diff from drahn@.
Disable userland debug communication access while there.

ok patrick@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

Adjust OpenBSD/arm64 files with FreeBSD origin to show the upstream
revision. While there, update a few of those files.

Prompted by mikeb@.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Add a few more PSTATE bits.

ok deraadt@

use ansi volatile keyword, not __volatile
ok miod@ guenther@

Add DIT CPU feature.

ok beck@

slignment -> alignment

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Fix the TCR_TG0_xxx definitions and add TCR_TG0_4K to the initial setting
of TCR_EL1 in locore to make clear we use 4K pages for both userland and
the kernel.

ok patrick@

Make sure that all CPUs end up with the same bits set in SCTLR_EL1.
Do this by clearing all the bits marked RES0 and set all the bits
marked RES1 for the ARMv8.0.

Any optional features introduced in later revisions of the architecture
(such as PAN) will be enabled after SCTLR_EL1 is initialized.

ok patrick@

Add ARMv8.5 instruction set related CPU features.

ok patrick@

spelling

Add code to print CPU features.

ok naddy@

Enable PAN (Privileged Access Never) on CPUs that support it. This means
that user-space access from the kernel is not allowed for "normal"
load/store instructions. Only the special "unprivileged" load/store
instructions are allowed. We already use those in copyin(9) and copyout(9).

ok patrick@, drahn@, jsg@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Cache flush operations on arm64 were being incorrectly treated as write
operations, however they should be treated as read per the design.
Switch to using bit defines, correct said defines.
Fixes cache flushing causing Firefox to abort.
ok kettenis@ kurt@

Newer ARMv8 processors now include a new CSV2 field in their processor
feature register that can indicate that a processor is not vulnarable to
Spectre v2 attacks. Use this field in favour of adding specific processors
to a whitelist. Continue to whitelist the few processors that are known
not to be vulnerable but don't set the appropriate value in the CSV2 field.

ok jsg@

Add the needed ICC_PMR_EL1 register bit defines for the previous
commit to unbreak the build.

from kettenis@
ok drahn@

Add code to print the characteristics of the caches that can be discovered
through the CLIDR_EL1 register.

ok patrick@

Implement single-stepping. Based on an earlier diff from drahn@.
Disable userland debug communication access while there.

ok patrick@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

Adjust OpenBSD/arm64 files with FreeBSD origin to show the upstream
revision. While there, update a few of those files.

Prompted by mikeb@.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

use ansi volatile keyword, not __volatile
ok miod@ guenther@

Add DIT CPU feature.

ok beck@

slignment -> alignment

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Fix the TCR_TG0_xxx definitions and add TCR_TG0_4K to the initial setting
of TCR_EL1 in locore to make clear we use 4K pages for both userland and
the kernel.

ok patrick@

Make sure that all CPUs end up with the same bits set in SCTLR_EL1.
Do this by clearing all the bits marked RES0 and set all the bits
marked RES1 for the ARMv8.0.

Any optional features introduced in later revisions of the architecture
(such as PAN) will be enabled after SCTLR_EL1 is initialized.

ok patrick@

Add ARMv8.5 instruction set related CPU features.

ok patrick@

spelling

Add code to print CPU features.

ok naddy@

Enable PAN (Privileged Access Never) on CPUs that support it. This means
that user-space access from the kernel is not allowed for "normal"
load/store instructions. Only the special "unprivileged" load/store
instructions are allowed. We already use those in copyin(9) and copyout(9).

ok patrick@, drahn@, jsg@

Allow userland access to the virtual counter.

ok patrick@, deraadt@

Cache flush operations on arm64 were being incorrectly treated as write
operations, however they should be treated as read per the design.
Switch to using bit defines, correct said defines.
Fixes cache flushing causing Firefox to abort.
ok kettenis@ kurt@

Newer ARMv8 processors now include a new CSV2 field in their processor
feature register that can indicate that a processor is not vulnarable to
Spectre v2 attacks. Use this field in favour of adding specific processors
to a whitelist. Continue to whitelist the few processors that are known
not to be vulnerable but don't set the appropriate value in the CSV2 field.

ok jsg@

Add the needed ICC_PMR_EL1 register bit defines for the previous
commit to unbreak the build.

from kettenis@
ok drahn@

Add code to print the characteristics of the caches that can be discovered
through the CLIDR_EL1 register.

ok patrick@

Implement single-stepping. Based on an earlier diff from drahn@.
Disable userland debug communication access while there.

ok patrick@

Fix TCR definitions to avoid integer overflow. Rename TCR_ASID_16 to TCR_AS
to match the official ARM docs.

ok patrick@, tom@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.