fpu_kern_enter/leave: Disable IPL assertions.

These don't work because mutex_enter/exit on a spin lock may raise an
IPL but not lower it, if another spin lock was already held. For
example,

mutex_enter(some_lock_at_IPL_VM);
printf("foo\n");
fpu_kern_enter();
...
fpu_kern_leave();
mutex_exit(some_lock_at_IPL_VM);

will trigger the panic, because printf takes a lock at IPL_HIGH where
the IPL wil remain until the mutex_exit. (This was a nightmare to
track down before I remembered that detail of spin lock IPL
semantics...)

x86, arm: Allow fpu_kern_enter/leave while cold.

Normally these are forbidden above IPL_VM, so that FPU usage doesn't
block IPL_SCHED or IPL_HIGH interrupts. But while cold, e.g. during
builtin module initialization at boot, all interrupts are blocked
anyway so it's a moot point.

Also initialize x86 cpu_info_primary.ci_kfpu_spl to -1 so we don't
trip over an assertion about it while cold -- the assertion is meant
to detect reentrance into fpu_kern_enter/leave, which is prohibited.

Also initialize cpu0's ci_kfpu_spl.

Rework Arm (32bit and 64bit) AP startup so that cpu_hatch doesn't sleep.

The AP initialisation code in cpu_init_secondary_processor will read and
initialise the required system registers and state for the BP to attach
and report.

Rework the interrupt handler code for this new sequence. Thankfully,
this removes a bunch of code for bcm2836mp.

The VFP detection handler on <= armv7 relies on the global undefined
handler being in place until the BP attaches vfp. That is, after the
APs have been spun up.

gicv3_its.c has a serialisation issue which is protected against in
the gicv3_its_cpu_init, which is called from cpu_hatch, with a spin
lock. The serialisation issue needs addressing more completely.

Tested on RPI3, Apple M1, QEMU, and lx2k

Fixes PR port-arm/56264:
diagnostic assertion "l->l_stat == LSONPROC" failed on RPI3

Trailing whitespace

PR port-arm/55790

Fix KASSERT failure with floating-point exception in userland.

Consider the case in which curlwp owns enabled FPU in vfp_handler().
If FPE is raised, we must skip pcu_load(9) rather than just falling
through. Otherwise, KASSERT fires in vfp_state_load(), since curlwp
already owns enabled FPU.

No regression for ATF is introduced.

PR port-arm/55790

Style fix for clarity, in preparation of main fix.

Replace condition ``curcpu()->ci_pcu_curlwp[PCU_FPU] == curlwp'' with
``curlwp->l_pcu_cpu[PCU_FPU] == curcpu()''. And add KASSERT to check
the two conditions are equivalent, as done for MI pcu code:

https://nxr.netbsd.org/xref/src/sys/kern/subr_pcu.c#323

No functional changes.

branches: 1.72.6;
Retire arm_[di]sb in favour of the isb() and dsb(sy) macro invocations.

Add kthread_fpu_enter/exit support to arm.

Enable ChaCha NEON code on armv7 too.

The 4-blocks-at-a-time assembly helper is disabled for now; adapting
it to armv7 is going to be a little annoying with only 16 128-bit
vector registers.

(Should also do a fifth block in the integer registers for 320 bytes
at a time.)

Split aes_impl declarations out into aes_impl.h.

This will make it less painful to add more operations to struct
aes_impl without having to recompile everything that just uses the
block cipher directly or similar.

Use pcu_save_all_on_cpu, not pcu_save.

We don't care what curlwp is here; we care whose state is in the fpu
registers.

Limit arm32 fpu_kern_enter/leave to IPL_VM or below.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement fpu_kern_enter/leave for arm32.

Explicitly annotate FPU requirements for LLVM MC.

When using GCC, this annotations change the global state, but there is
no push/pop functionality for .fpu to avoid this problem. The state is
local to each inline assembler block with LLVM MC.

Cortex A12 is marketed as A17 but has a distinct part number

observed on Rockchip RK3288

Install the undefined instruction handlers only once, i.e. when attaching
on the BP.

Trailing whitespace

Merge the [pgoyette-compat] branch

Sprinkle #include "opt_cputypes.h"

Add __KERNEL_RCSID

branches: 1.57.2;
Remove the call to vfp_fpscr_handler() from vfp_handler(). It actually never
avoids a full FPU switch, and costs a function call and a few tests.

Discussed on port-arm@ on october 2017:
http://mail-index.netbsd.org/port-arm/2017/10/16/msg004411.html

branches: 1.56.2;
Add more vfp directives for gcc-6

We KASSERT((fregs->vfp_fpexc & VFP_FPEXC_EN) == 0) just before, so
enabled is always false. remove.

In the REENABLE case, make sur the fpexc copy in the pcb also has
VFP_FPEXC_EN set. Otherwise we could trap on every context switch even if
the CPU already has the VFP state.

branches: 1.53.2;
Recognize Cortex-A57 FPU, GIC, and Generic Timer.

in vfp_state_load(), fix backwards logic for fpinst vs. fpinst2.

allow pcu_save() and pcu_discard() to be called on other threads,
ptrace needs to use it that way.

branches: 1.50.2; 1.50.4;
Get the RPI3 working (in aarch32 mode) by recognising Cortex A53 CPUs.
While I'm here add some A57/A72 info as well.

My RPI3 works with FB console - the uart needs some help with its clocks.

change some register dumps from aprint_verbose to aprint_debug

isb after writing cpacr, from Andrew Turner

Fix some inverted return values. Don't return SIGILL if there is an active
FPU exception.

Remove extra )

Not only check to see if we own the VFP but that the VFP is enabled.

Don't try to catch undefined VFP instructions if we own the the FPU.
Let them raise SIGILL.

If we own the FPU, don't take anymore undefined faults. Instead generate
SIGILLs since we obviously don't understand the instruction.

Add VFP IDs for QEMU's emulated Cortex-A15.

branches: 1.41.2; 1.41.4;
fix typo reported in PR/48948

Cleanup a bit of the init logic.

pcu(9):
- Remove PCU_KERNEL (hi matt!) and significantly simplify the code.
This experimental feature was tried on ARM did not meet the expectations.
It may be revived one day, but it should be done in a much simpler way.
- Add a message structure for xcall function, pass the LWP ower and thus
optimise a race condition: if LWP is discarding its state on a remote CPU,
but another LWP already did it - do not cause an unecessary re-faulting.
- Reduce the variety of flags for PCU operations (only PCU_VALID and
PCU_REENABLE are used now), pass them only to the pcu_state_load().
- Rename pcu_used_p() to pcu_valid_p(); hopefully it is less confusing.
- pcu_save_all_on_cpu: SPL ought to be used here.
- Update and improve the pcu(9) man page; it needs wizd(8) though.

propogation -> propagation

branches: 1.37.2;
Various MP changes.

Enable VFP on MV88SV58XX

Add a different version vfp_fpscr_changable if FPU_VFP was not defined.
If no FPU was found, reinit vfp_fpscr_changeable/default to values appropriate
for softfloat.

Query the media and vfp feature registers to determine what our default
mode should be and what bits in the fpscr can be changed.
Print what features are supported:
vfp0 at cpu0: NEON MPE (VFP 3.0+), rounding, NaN propogation, denormals

Improve PCU/VFP handling to the point that the atf tests don't trigger
KASSERTs on the Raspberry PI and its arm1176jzf-s.

XXX Need to emulate bounce instructions to get correct exception codes,
XXX etc.

Be consistent about setting fpscr for Runfast. No functional change.

Fix typo in #define name

Typo in comment

Switch to using FP instructions instead of cp10/11 instructions.

If we can't enable VFP/VFP2 via the CPACCESS register, bail since there
isn't a VFP.

Before checking for an exception, make sure we own the VFP.

Deal with lack of VFP.

Reap LWP_VFPUSED and use PCU internal tracking.
Add bool vfp_used_p(void);

-extend the pcu(9) API by a function which saves all context on the
current CPU, and use it if a CPU is taken offline
-add a bool argument to pcu_discard which tells whether the internal
"LWP has used the coprocessor" flag should be set or reset. The flag
is reported by pcu_used_p(). If set, future accesses should use the
state stored in the PCB. If reset, it should be reset to default.
The former case is useful for setmcontext().
With that, it should not be necessary anymore to manage the "FPU used"
state by an additional MD variable.

approved by matt

Move parts of cpu.h that are not needed by MI code in <arm/locore.h>
Don't include <machine/cpu.h> or <machine/frame.h>, use <arm/locore.h>
Use <arm/asm.h> instead of <machine/arm.h>

Add VFP_FPSCR_RN (even though it's 0) just to be explicit.

Use armreg inlines.
Add exception -> trapsignal code.

branches: 1.20.2;
Add support for the Cortex-A15 Neon/VFP unit

Use the mrc form of the vmrs rX, mvfrX instruction to shut up gas.

Add support for machdep neon_present and id_mvfr sysctls

Add a machdep.fpu_present sysctl for ld.elf_so to use in ld.so.conf to load
libc_vfp.so.

Disable bzero_page_vfp and bcopy_page_vfp since it really isn't any faster
than memcpy.

Always re-enable the VFP when loading for a kernel LWP.

print the PC of the VFP kernel fault in the panic message.

Add support for PCU_KERNEL and vfp_kernel_acquire/vfp_kernel_release.
Add an undefined handler to catch NEON instructions.

Add code to patch pmap_{copy,zero}_page_generic to change calls to
b{copy,zero}_page to b{copy,zero}_page_vfp

move inlines into FPU_VFP

On Cortex, make sure to load/save the upper 16 64-FP registers.
When creating a mcontext_t, make sure _UC_ARM_VFP is set.

For armv7 (cortex), disable access to the upper 16 FP registers (restrict
the register space to 16 64-bit FP registers).

ARMFPE hasn't compiled since NetBSD 4. Remove it.
Complete support for FPU_VFP.
fpregs now contains vfpreg.
XXX vfpreg only has space for 16 64-bit FP registers though VFPv3 and later
have 32 64-bit FP registers.

Only use CPACR register for ARM11 and CORTEX cores.
Add VFP ids for other CORTEX CPUs.

Before testing for VFP, make sure CP10 is enabled. (And CP11 for Neon too).

branches: 1.5.2;
Add include of <arm/pcb.h>

Rework VFP support to use PCU.
Add emulation of instruction which save/restore the VFP FPSCR.
Add a sysarch hook to VFP FPSCR manipulation.

[The emulation will be used by libc to store/fetch exception modes and
rounding mode on a per-thread basis.]

branches: 1.3.12; 1.3.20;
Use lwp_getpcb() on ARM (and acorn26/32), clean from struct user usage.

Ansify function definitions w/o arguments. Generated with sed.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.12; 1.1.20; 1.1.26;
VFP support.

x86, arm: Allow fpu_kern_enter/leave while cold.

Normally these are forbidden above IPL_VM, so that FPU usage doesn't
block IPL_SCHED or IPL_HIGH interrupts. But while cold, e.g. during
builtin module initialization at boot, all interrupts are blocked
anyway so it's a moot point.

Also initialize x86 cpu_info_primary.ci_kfpu_spl to -1 so we don't
trip over an assertion about it while cold -- the assertion is meant
to detect reentrance into fpu_kern_enter/leave, which is prohibited.

Also initialize cpu0's ci_kfpu_spl.

Rework Arm (32bit and 64bit) AP startup so that cpu_hatch doesn't sleep.

The AP initialisation code in cpu_init_secondary_processor will read and
initialise the required system registers and state for the BP to attach
and report.

Rework the interrupt handler code for this new sequence. Thankfully,
this removes a bunch of code for bcm2836mp.

The VFP detection handler on <= armv7 relies on the global undefined
handler being in place until the BP attaches vfp. That is, after the
APs have been spun up.

gicv3_its.c has a serialisation issue which is protected against in
the gicv3_its_cpu_init, which is called from cpu_hatch, with a spin
lock. The serialisation issue needs addressing more completely.

Tested on RPI3, Apple M1, QEMU, and lx2k

Fixes PR port-arm/56264:
diagnostic assertion "l->l_stat == LSONPROC" failed on RPI3

Trailing whitespace

PR port-arm/55790

Fix KASSERT failure with floating-point exception in userland.

Consider the case in which curlwp owns enabled FPU in vfp_handler().
If FPE is raised, we must skip pcu_load(9) rather than just falling
through. Otherwise, KASSERT fires in vfp_state_load(), since curlwp
already owns enabled FPU.

No regression for ATF is introduced.

PR port-arm/55790

Style fix for clarity, in preparation of main fix.

Replace condition ``curcpu()->ci_pcu_curlwp[PCU_FPU] == curlwp'' with
``curlwp->l_pcu_cpu[PCU_FPU] == curcpu()''. And add KASSERT to check
the two conditions are equivalent, as done for MI pcu code:

https://nxr.netbsd.org/xref/src/sys/kern/subr_pcu.c#323

No functional changes.

branches: 1.72.6;
Retire arm_[di]sb in favour of the isb() and dsb(sy) macro invocations.

Add kthread_fpu_enter/exit support to arm.

Enable ChaCha NEON code on armv7 too.

The 4-blocks-at-a-time assembly helper is disabled for now; adapting
it to armv7 is going to be a little annoying with only 16 128-bit
vector registers.

(Should also do a fifth block in the integer registers for 320 bytes
at a time.)

Split aes_impl declarations out into aes_impl.h.

This will make it less painful to add more operations to struct
aes_impl without having to recompile everything that just uses the
block cipher directly or similar.

Use pcu_save_all_on_cpu, not pcu_save.

We don't care what curlwp is here; we care whose state is in the fpu
registers.

Limit arm32 fpu_kern_enter/leave to IPL_VM or below.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement fpu_kern_enter/leave for arm32.

Explicitly annotate FPU requirements for LLVM MC.

When using GCC, this annotations change the global state, but there is
no push/pop functionality for .fpu to avoid this problem. The state is
local to each inline assembler block with LLVM MC.

Cortex A12 is marketed as A17 but has a distinct part number

observed on Rockchip RK3288

Install the undefined instruction handlers only once, i.e. when attaching
on the BP.

Trailing whitespace

Merge the [pgoyette-compat] branch

Sprinkle #include "opt_cputypes.h"

Add __KERNEL_RCSID

branches: 1.57.2;
Remove the call to vfp_fpscr_handler() from vfp_handler(). It actually never
avoids a full FPU switch, and costs a function call and a few tests.

Discussed on port-arm@ on october 2017:
http://mail-index.netbsd.org/port-arm/2017/10/16/msg004411.html

branches: 1.56.2;
Add more vfp directives for gcc-6

We KASSERT((fregs->vfp_fpexc & VFP_FPEXC_EN) == 0) just before, so
enabled is always false. remove.

In the REENABLE case, make sur the fpexc copy in the pcb also has
VFP_FPEXC_EN set. Otherwise we could trap on every context switch even if
the CPU already has the VFP state.

branches: 1.53.2;
Recognize Cortex-A57 FPU, GIC, and Generic Timer.

in vfp_state_load(), fix backwards logic for fpinst vs. fpinst2.

allow pcu_save() and pcu_discard() to be called on other threads,
ptrace needs to use it that way.

branches: 1.50.2; 1.50.4;
Get the RPI3 working (in aarch32 mode) by recognising Cortex A53 CPUs.
While I'm here add some A57/A72 info as well.

My RPI3 works with FB console - the uart needs some help with its clocks.

change some register dumps from aprint_verbose to aprint_debug

isb after writing cpacr, from Andrew Turner

Fix some inverted return values. Don't return SIGILL if there is an active
FPU exception.

Remove extra )

Not only check to see if we own the VFP but that the VFP is enabled.

Don't try to catch undefined VFP instructions if we own the the FPU.
Let them raise SIGILL.

If we own the FPU, don't take anymore undefined faults. Instead generate
SIGILLs since we obviously don't understand the instruction.

Add VFP IDs for QEMU's emulated Cortex-A15.

branches: 1.41.2; 1.41.4;
fix typo reported in PR/48948

Cleanup a bit of the init logic.

pcu(9):
- Remove PCU_KERNEL (hi matt!) and significantly simplify the code.
This experimental feature was tried on ARM did not meet the expectations.
It may be revived one day, but it should be done in a much simpler way.
- Add a message structure for xcall function, pass the LWP ower and thus
optimise a race condition: if LWP is discarding its state on a remote CPU,
but another LWP already did it - do not cause an unecessary re-faulting.
- Reduce the variety of flags for PCU operations (only PCU_VALID and
PCU_REENABLE are used now), pass them only to the pcu_state_load().
- Rename pcu_used_p() to pcu_valid_p(); hopefully it is less confusing.
- pcu_save_all_on_cpu: SPL ought to be used here.
- Update and improve the pcu(9) man page; it needs wizd(8) though.

propogation -> propagation

branches: 1.37.2;
Various MP changes.

Enable VFP on MV88SV58XX

Add a different version vfp_fpscr_changable if FPU_VFP was not defined.
If no FPU was found, reinit vfp_fpscr_changeable/default to values appropriate
for softfloat.

Query the media and vfp feature registers to determine what our default
mode should be and what bits in the fpscr can be changed.
Print what features are supported:
vfp0 at cpu0: NEON MPE (VFP 3.0+), rounding, NaN propogation, denormals

Improve PCU/VFP handling to the point that the atf tests don't trigger
KASSERTs on the Raspberry PI and its arm1176jzf-s.

XXX Need to emulate bounce instructions to get correct exception codes,
XXX etc.

Be consistent about setting fpscr for Runfast. No functional change.

Fix typo in #define name

Typo in comment

Switch to using FP instructions instead of cp10/11 instructions.

If we can't enable VFP/VFP2 via the CPACCESS register, bail since there
isn't a VFP.

Before checking for an exception, make sure we own the VFP.

Deal with lack of VFP.

Reap LWP_VFPUSED and use PCU internal tracking.
Add bool vfp_used_p(void);

-extend the pcu(9) API by a function which saves all context on the
current CPU, and use it if a CPU is taken offline
-add a bool argument to pcu_discard which tells whether the internal
"LWP has used the coprocessor" flag should be set or reset. The flag
is reported by pcu_used_p(). If set, future accesses should use the
state stored in the PCB. If reset, it should be reset to default.
The former case is useful for setmcontext().
With that, it should not be necessary anymore to manage the "FPU used"
state by an additional MD variable.

approved by matt

Move parts of cpu.h that are not needed by MI code in <arm/locore.h>
Don't include <machine/cpu.h> or <machine/frame.h>, use <arm/locore.h>
Use <arm/asm.h> instead of <machine/arm.h>

Add VFP_FPSCR_RN (even though it's 0) just to be explicit.

Use armreg inlines.
Add exception -> trapsignal code.

branches: 1.20.2;
Add support for the Cortex-A15 Neon/VFP unit

Use the mrc form of the vmrs rX, mvfrX instruction to shut up gas.

Add support for machdep neon_present and id_mvfr sysctls

Add a machdep.fpu_present sysctl for ld.elf_so to use in ld.so.conf to load
libc_vfp.so.

Disable bzero_page_vfp and bcopy_page_vfp since it really isn't any faster
than memcpy.

Always re-enable the VFP when loading for a kernel LWP.

print the PC of the VFP kernel fault in the panic message.

Add support for PCU_KERNEL and vfp_kernel_acquire/vfp_kernel_release.
Add an undefined handler to catch NEON instructions.

Add code to patch pmap_{copy,zero}_page_generic to change calls to
b{copy,zero}_page to b{copy,zero}_page_vfp

move inlines into FPU_VFP

On Cortex, make sure to load/save the upper 16 64-FP registers.
When creating a mcontext_t, make sure _UC_ARM_VFP is set.

For armv7 (cortex), disable access to the upper 16 FP registers (restrict
the register space to 16 64-bit FP registers).

ARMFPE hasn't compiled since NetBSD 4. Remove it.
Complete support for FPU_VFP.
fpregs now contains vfpreg.
XXX vfpreg only has space for 16 64-bit FP registers though VFPv3 and later
have 32 64-bit FP registers.

Only use CPACR register for ARM11 and CORTEX cores.
Add VFP ids for other CORTEX CPUs.

Before testing for VFP, make sure CP10 is enabled. (And CP11 for Neon too).

branches: 1.5.2;
Add include of <arm/pcb.h>

Rework VFP support to use PCU.
Add emulation of instruction which save/restore the VFP FPSCR.
Add a sysarch hook to VFP FPSCR manipulation.

[The emulation will be used by libc to store/fetch exception modes and
rounding mode on a per-thread basis.]

branches: 1.3.12; 1.3.20;
Use lwp_getpcb() on ARM (and acorn26/32), clean from struct user usage.

Ansify function definitions w/o arguments. Generated with sed.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.12; 1.1.20; 1.1.26;
VFP support.

Rework Arm (32bit and 64bit) AP startup so that cpu_hatch doesn't sleep.

The AP initialisation code in cpu_init_secondary_processor will read and
initialise the required system registers and state for the BP to attach
and report.

Rework the interrupt handler code for this new sequence. Thankfully,
this removes a bunch of code for bcm2836mp.

The VFP detection handler on <= armv7 relies on the global undefined
handler being in place until the BP attaches vfp. That is, after the
APs have been spun up.

gicv3_its.c has a serialisation issue which is protected against in
the gicv3_its_cpu_init, which is called from cpu_hatch, with a spin
lock. The serialisation issue needs addressing more completely.

Tested on RPI3, Apple M1, QEMU, and lx2k

Fixes PR port-arm/56264:
diagnostic assertion "l->l_stat == LSONPROC" failed on RPI3

Trailing whitespace

PR port-arm/55790

Fix KASSERT failure with floating-point exception in userland.

Consider the case in which curlwp owns enabled FPU in vfp_handler().
If FPE is raised, we must skip pcu_load(9) rather than just falling
through. Otherwise, KASSERT fires in vfp_state_load(), since curlwp
already owns enabled FPU.

No regression for ATF is introduced.

PR port-arm/55790

Style fix for clarity, in preparation of main fix.

Replace condition ``curcpu()->ci_pcu_curlwp[PCU_FPU] == curlwp'' with
``curlwp->l_pcu_cpu[PCU_FPU] == curcpu()''. And add KASSERT to check
the two conditions are equivalent, as done for MI pcu code:

https://nxr.netbsd.org/xref/src/sys/kern/subr_pcu.c#323

No functional changes.

branches: 1.72.6;
Retire arm_[di]sb in favour of the isb() and dsb(sy) macro invocations.

Add kthread_fpu_enter/exit support to arm.

Enable ChaCha NEON code on armv7 too.

The 4-blocks-at-a-time assembly helper is disabled for now; adapting
it to armv7 is going to be a little annoying with only 16 128-bit
vector registers.

(Should also do a fifth block in the integer registers for 320 bytes
at a time.)

Split aes_impl declarations out into aes_impl.h.

This will make it less painful to add more operations to struct
aes_impl without having to recompile everything that just uses the
block cipher directly or similar.

Use pcu_save_all_on_cpu, not pcu_save.

We don't care what curlwp is here; we care whose state is in the fpu
registers.

Limit arm32 fpu_kern_enter/leave to IPL_VM or below.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement fpu_kern_enter/leave for arm32.

Explicitly annotate FPU requirements for LLVM MC.

When using GCC, this annotations change the global state, but there is
no push/pop functionality for .fpu to avoid this problem. The state is
local to each inline assembler block with LLVM MC.

Cortex A12 is marketed as A17 but has a distinct part number

observed on Rockchip RK3288

Install the undefined instruction handlers only once, i.e. when attaching
on the BP.

Trailing whitespace

Merge the [pgoyette-compat] branch

Sprinkle #include "opt_cputypes.h"

Add __KERNEL_RCSID

branches: 1.57.2;
Remove the call to vfp_fpscr_handler() from vfp_handler(). It actually never
avoids a full FPU switch, and costs a function call and a few tests.

Discussed on port-arm@ on october 2017:
http://mail-index.netbsd.org/port-arm/2017/10/16/msg004411.html

branches: 1.56.2;
Add more vfp directives for gcc-6

We KASSERT((fregs->vfp_fpexc & VFP_FPEXC_EN) == 0) just before, so
enabled is always false. remove.

In the REENABLE case, make sur the fpexc copy in the pcb also has
VFP_FPEXC_EN set. Otherwise we could trap on every context switch even if
the CPU already has the VFP state.

branches: 1.53.2;
Recognize Cortex-A57 FPU, GIC, and Generic Timer.

in vfp_state_load(), fix backwards logic for fpinst vs. fpinst2.

allow pcu_save() and pcu_discard() to be called on other threads,
ptrace needs to use it that way.

branches: 1.50.2; 1.50.4;
Get the RPI3 working (in aarch32 mode) by recognising Cortex A53 CPUs.
While I'm here add some A57/A72 info as well.

My RPI3 works with FB console - the uart needs some help with its clocks.

change some register dumps from aprint_verbose to aprint_debug

isb after writing cpacr, from Andrew Turner

Fix some inverted return values. Don't return SIGILL if there is an active
FPU exception.

Remove extra )

Not only check to see if we own the VFP but that the VFP is enabled.

Don't try to catch undefined VFP instructions if we own the the FPU.
Let them raise SIGILL.

If we own the FPU, don't take anymore undefined faults. Instead generate
SIGILLs since we obviously don't understand the instruction.

Add VFP IDs for QEMU's emulated Cortex-A15.

branches: 1.41.2; 1.41.4;
fix typo reported in PR/48948

Cleanup a bit of the init logic.

pcu(9):
- Remove PCU_KERNEL (hi matt!) and significantly simplify the code.
This experimental feature was tried on ARM did not meet the expectations.
It may be revived one day, but it should be done in a much simpler way.
- Add a message structure for xcall function, pass the LWP ower and thus
optimise a race condition: if LWP is discarding its state on a remote CPU,
but another LWP already did it - do not cause an unecessary re-faulting.
- Reduce the variety of flags for PCU operations (only PCU_VALID and
PCU_REENABLE are used now), pass them only to the pcu_state_load().
- Rename pcu_used_p() to pcu_valid_p(); hopefully it is less confusing.
- pcu_save_all_on_cpu: SPL ought to be used here.
- Update and improve the pcu(9) man page; it needs wizd(8) though.

propogation -> propagation

branches: 1.37.2;
Various MP changes.

Enable VFP on MV88SV58XX

Add a different version vfp_fpscr_changable if FPU_VFP was not defined.
If no FPU was found, reinit vfp_fpscr_changeable/default to values appropriate
for softfloat.

Query the media and vfp feature registers to determine what our default
mode should be and what bits in the fpscr can be changed.
Print what features are supported:
vfp0 at cpu0: NEON MPE (VFP 3.0+), rounding, NaN propogation, denormals

Improve PCU/VFP handling to the point that the atf tests don't trigger
KASSERTs on the Raspberry PI and its arm1176jzf-s.

XXX Need to emulate bounce instructions to get correct exception codes,
XXX etc.

Be consistent about setting fpscr for Runfast. No functional change.

Fix typo in #define name

Typo in comment

Switch to using FP instructions instead of cp10/11 instructions.

If we can't enable VFP/VFP2 via the CPACCESS register, bail since there
isn't a VFP.

Before checking for an exception, make sure we own the VFP.

Deal with lack of VFP.

Reap LWP_VFPUSED and use PCU internal tracking.
Add bool vfp_used_p(void);

-extend the pcu(9) API by a function which saves all context on the
current CPU, and use it if a CPU is taken offline
-add a bool argument to pcu_discard which tells whether the internal
"LWP has used the coprocessor" flag should be set or reset. The flag
is reported by pcu_used_p(). If set, future accesses should use the
state stored in the PCB. If reset, it should be reset to default.
The former case is useful for setmcontext().
With that, it should not be necessary anymore to manage the "FPU used"
state by an additional MD variable.

approved by matt

Move parts of cpu.h that are not needed by MI code in <arm/locore.h>
Don't include <machine/cpu.h> or <machine/frame.h>, use <arm/locore.h>
Use <arm/asm.h> instead of <machine/arm.h>

Add VFP_FPSCR_RN (even though it's 0) just to be explicit.

Use armreg inlines.
Add exception -> trapsignal code.

branches: 1.20.2;
Add support for the Cortex-A15 Neon/VFP unit

Use the mrc form of the vmrs rX, mvfrX instruction to shut up gas.

Add support for machdep neon_present and id_mvfr sysctls

Add a machdep.fpu_present sysctl for ld.elf_so to use in ld.so.conf to load
libc_vfp.so.

Disable bzero_page_vfp and bcopy_page_vfp since it really isn't any faster
than memcpy.

Always re-enable the VFP when loading for a kernel LWP.

print the PC of the VFP kernel fault in the panic message.

Add support for PCU_KERNEL and vfp_kernel_acquire/vfp_kernel_release.
Add an undefined handler to catch NEON instructions.

Add code to patch pmap_{copy,zero}_page_generic to change calls to
b{copy,zero}_page to b{copy,zero}_page_vfp

move inlines into FPU_VFP

On Cortex, make sure to load/save the upper 16 64-FP registers.
When creating a mcontext_t, make sure _UC_ARM_VFP is set.

For armv7 (cortex), disable access to the upper 16 FP registers (restrict
the register space to 16 64-bit FP registers).

ARMFPE hasn't compiled since NetBSD 4. Remove it.
Complete support for FPU_VFP.
fpregs now contains vfpreg.
XXX vfpreg only has space for 16 64-bit FP registers though VFPv3 and later
have 32 64-bit FP registers.

Only use CPACR register for ARM11 and CORTEX cores.
Add VFP ids for other CORTEX CPUs.

Before testing for VFP, make sure CP10 is enabled. (And CP11 for Neon too).

branches: 1.5.2;
Add include of <arm/pcb.h>

Rework VFP support to use PCU.
Add emulation of instruction which save/restore the VFP FPSCR.
Add a sysarch hook to VFP FPSCR manipulation.

[The emulation will be used by libc to store/fetch exception modes and
rounding mode on a per-thread basis.]

branches: 1.3.12; 1.3.20;
Use lwp_getpcb() on ARM (and acorn26/32), clean from struct user usage.

Ansify function definitions w/o arguments. Generated with sed.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.12; 1.1.20; 1.1.26;
VFP support.

Trailing whitespace

PR port-arm/55790

Fix KASSERT failure with floating-point exception in userland.

Consider the case in which curlwp owns enabled FPU in vfp_handler().
If FPE is raised, we must skip pcu_load(9) rather than just falling
through. Otherwise, KASSERT fires in vfp_state_load(), since curlwp
already owns enabled FPU.

No regression for ATF is introduced.

PR port-arm/55790

Style fix for clarity, in preparation of main fix.

Replace condition ``curcpu()->ci_pcu_curlwp[PCU_FPU] == curlwp'' with
``curlwp->l_pcu_cpu[PCU_FPU] == curcpu()''. And add KASSERT to check
the two conditions are equivalent, as done for MI pcu code:

https://nxr.netbsd.org/xref/src/sys/kern/subr_pcu.c#323

No functional changes.

branches: 1.72.6;
Retire arm_[di]sb in favour of the isb() and dsb(sy) macro invocations.

Add kthread_fpu_enter/exit support to arm.

Enable ChaCha NEON code on armv7 too.

The 4-blocks-at-a-time assembly helper is disabled for now; adapting
it to armv7 is going to be a little annoying with only 16 128-bit
vector registers.

(Should also do a fifth block in the integer registers for 320 bytes
at a time.)

Split aes_impl declarations out into aes_impl.h.

This will make it less painful to add more operations to struct
aes_impl without having to recompile everything that just uses the
block cipher directly or similar.

Use pcu_save_all_on_cpu, not pcu_save.

We don't care what curlwp is here; we care whose state is in the fpu
registers.

Limit arm32 fpu_kern_enter/leave to IPL_VM or below.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement fpu_kern_enter/leave for arm32.

Explicitly annotate FPU requirements for LLVM MC.

When using GCC, this annotations change the global state, but there is
no push/pop functionality for .fpu to avoid this problem. The state is
local to each inline assembler block with LLVM MC.

Cortex A12 is marketed as A17 but has a distinct part number

observed on Rockchip RK3288

Install the undefined instruction handlers only once, i.e. when attaching
on the BP.

Trailing whitespace

Merge the [pgoyette-compat] branch

Sprinkle #include "opt_cputypes.h"

Add __KERNEL_RCSID

branches: 1.57.2;
Remove the call to vfp_fpscr_handler() from vfp_handler(). It actually never
avoids a full FPU switch, and costs a function call and a few tests.

Discussed on port-arm@ on october 2017:
http://mail-index.netbsd.org/port-arm/2017/10/16/msg004411.html

branches: 1.56.2;
Add more vfp directives for gcc-6

We KASSERT((fregs->vfp_fpexc & VFP_FPEXC_EN) == 0) just before, so
enabled is always false. remove.

In the REENABLE case, make sur the fpexc copy in the pcb also has
VFP_FPEXC_EN set. Otherwise we could trap on every context switch even if
the CPU already has the VFP state.

branches: 1.53.2;
Recognize Cortex-A57 FPU, GIC, and Generic Timer.

in vfp_state_load(), fix backwards logic for fpinst vs. fpinst2.

allow pcu_save() and pcu_discard() to be called on other threads,
ptrace needs to use it that way.

branches: 1.50.2; 1.50.4;
Get the RPI3 working (in aarch32 mode) by recognising Cortex A53 CPUs.
While I'm here add some A57/A72 info as well.

My RPI3 works with FB console - the uart needs some help with its clocks.

change some register dumps from aprint_verbose to aprint_debug

isb after writing cpacr, from Andrew Turner

Fix some inverted return values. Don't return SIGILL if there is an active
FPU exception.

Remove extra )

Not only check to see if we own the VFP but that the VFP is enabled.

Don't try to catch undefined VFP instructions if we own the the FPU.
Let them raise SIGILL.

If we own the FPU, don't take anymore undefined faults. Instead generate
SIGILLs since we obviously don't understand the instruction.

Add VFP IDs for QEMU's emulated Cortex-A15.

branches: 1.41.2; 1.41.4;
fix typo reported in PR/48948

Cleanup a bit of the init logic.

pcu(9):
- Remove PCU_KERNEL (hi matt!) and significantly simplify the code.
This experimental feature was tried on ARM did not meet the expectations.
It may be revived one day, but it should be done in a much simpler way.
- Add a message structure for xcall function, pass the LWP ower and thus
optimise a race condition: if LWP is discarding its state on a remote CPU,
but another LWP already did it - do not cause an unecessary re-faulting.
- Reduce the variety of flags for PCU operations (only PCU_VALID and
PCU_REENABLE are used now), pass them only to the pcu_state_load().
- Rename pcu_used_p() to pcu_valid_p(); hopefully it is less confusing.
- pcu_save_all_on_cpu: SPL ought to be used here.
- Update and improve the pcu(9) man page; it needs wizd(8) though.

propogation -> propagation

branches: 1.37.2;
Various MP changes.

Enable VFP on MV88SV58XX

Add a different version vfp_fpscr_changable if FPU_VFP was not defined.
If no FPU was found, reinit vfp_fpscr_changeable/default to values appropriate
for softfloat.

Query the media and vfp feature registers to determine what our default
mode should be and what bits in the fpscr can be changed.
Print what features are supported:
vfp0 at cpu0: NEON MPE (VFP 3.0+), rounding, NaN propogation, denormals

Improve PCU/VFP handling to the point that the atf tests don't trigger
KASSERTs on the Raspberry PI and its arm1176jzf-s.

XXX Need to emulate bounce instructions to get correct exception codes,
XXX etc.

Be consistent about setting fpscr for Runfast. No functional change.

Fix typo in #define name

Typo in comment

Switch to using FP instructions instead of cp10/11 instructions.

If we can't enable VFP/VFP2 via the CPACCESS register, bail since there
isn't a VFP.

Before checking for an exception, make sure we own the VFP.

Deal with lack of VFP.

Reap LWP_VFPUSED and use PCU internal tracking.
Add bool vfp_used_p(void);

-extend the pcu(9) API by a function which saves all context on the
current CPU, and use it if a CPU is taken offline
-add a bool argument to pcu_discard which tells whether the internal
"LWP has used the coprocessor" flag should be set or reset. The flag
is reported by pcu_used_p(). If set, future accesses should use the
state stored in the PCB. If reset, it should be reset to default.
The former case is useful for setmcontext().
With that, it should not be necessary anymore to manage the "FPU used"
state by an additional MD variable.

approved by matt

Move parts of cpu.h that are not needed by MI code in <arm/locore.h>
Don't include <machine/cpu.h> or <machine/frame.h>, use <arm/locore.h>
Use <arm/asm.h> instead of <machine/arm.h>

Add VFP_FPSCR_RN (even though it's 0) just to be explicit.

Use armreg inlines.
Add exception -> trapsignal code.

branches: 1.20.2;
Add support for the Cortex-A15 Neon/VFP unit

Use the mrc form of the vmrs rX, mvfrX instruction to shut up gas.

Add support for machdep neon_present and id_mvfr sysctls

Add a machdep.fpu_present sysctl for ld.elf_so to use in ld.so.conf to load
libc_vfp.so.

Disable bzero_page_vfp and bcopy_page_vfp since it really isn't any faster
than memcpy.

Always re-enable the VFP when loading for a kernel LWP.

print the PC of the VFP kernel fault in the panic message.

Add support for PCU_KERNEL and vfp_kernel_acquire/vfp_kernel_release.
Add an undefined handler to catch NEON instructions.

Add code to patch pmap_{copy,zero}_page_generic to change calls to
b{copy,zero}_page to b{copy,zero}_page_vfp

move inlines into FPU_VFP

On Cortex, make sure to load/save the upper 16 64-FP registers.
When creating a mcontext_t, make sure _UC_ARM_VFP is set.

For armv7 (cortex), disable access to the upper 16 FP registers (restrict
the register space to 16 64-bit FP registers).

ARMFPE hasn't compiled since NetBSD 4. Remove it.
Complete support for FPU_VFP.
fpregs now contains vfpreg.
XXX vfpreg only has space for 16 64-bit FP registers though VFPv3 and later
have 32 64-bit FP registers.

Only use CPACR register for ARM11 and CORTEX cores.
Add VFP ids for other CORTEX CPUs.

Before testing for VFP, make sure CP10 is enabled. (And CP11 for Neon too).

branches: 1.5.2;
Add include of <arm/pcb.h>

Rework VFP support to use PCU.
Add emulation of instruction which save/restore the VFP FPSCR.
Add a sysarch hook to VFP FPSCR manipulation.

[The emulation will be used by libc to store/fetch exception modes and
rounding mode on a per-thread basis.]

branches: 1.3.12; 1.3.20;
Use lwp_getpcb() on ARM (and acorn26/32), clean from struct user usage.

Ansify function definitions w/o arguments. Generated with sed.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.12; 1.1.20; 1.1.26;
VFP support.

PR port-arm/55790

Fix KASSERT failure with floating-point exception in userland.

Consider the case in which curlwp owns enabled FPU in vfp_handler().
If FPE is raised, we must skip pcu_load(9) rather than just falling
through. Otherwise, KASSERT fires in vfp_state_load(), since curlwp
already owns enabled FPU.

No regression for ATF is introduced.

PR port-arm/55790

Style fix for clarity, in preparation of main fix.

Replace condition ``curcpu()->ci_pcu_curlwp[PCU_FPU] == curlwp'' with
``curlwp->l_pcu_cpu[PCU_FPU] == curcpu()''. And add KASSERT to check
the two conditions are equivalent, as done for MI pcu code:

https://nxr.netbsd.org/xref/src/sys/kern/subr_pcu.c#323

No functional changes.

Retire arm_[di]sb in favour of the isb() and dsb(sy) macro invocations.

Add kthread_fpu_enter/exit support to arm.

Enable ChaCha NEON code on armv7 too.

The 4-blocks-at-a-time assembly helper is disabled for now; adapting
it to armv7 is going to be a little annoying with only 16 128-bit
vector registers.

(Should also do a fifth block in the integer registers for 320 bytes
at a time.)

Split aes_impl declarations out into aes_impl.h.

This will make it less painful to add more operations to struct
aes_impl without having to recompile everything that just uses the
block cipher directly or similar.

Use pcu_save_all_on_cpu, not pcu_save.

We don't care what curlwp is here; we care whose state is in the fpu
registers.

Limit arm32 fpu_kern_enter/leave to IPL_VM or below.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement fpu_kern_enter/leave for arm32.

Explicitly annotate FPU requirements for LLVM MC.

When using GCC, this annotations change the global state, but there is
no push/pop functionality for .fpu to avoid this problem. The state is
local to each inline assembler block with LLVM MC.

Cortex A12 is marketed as A17 but has a distinct part number

observed on Rockchip RK3288

Install the undefined instruction handlers only once, i.e. when attaching
on the BP.

Trailing whitespace

Merge the [pgoyette-compat] branch

Sprinkle #include "opt_cputypes.h"

Add __KERNEL_RCSID

branches: 1.57.2;
Remove the call to vfp_fpscr_handler() from vfp_handler(). It actually never
avoids a full FPU switch, and costs a function call and a few tests.

Discussed on port-arm@ on october 2017:
http://mail-index.netbsd.org/port-arm/2017/10/16/msg004411.html

branches: 1.56.2;
Add more vfp directives for gcc-6

We KASSERT((fregs->vfp_fpexc & VFP_FPEXC_EN) == 0) just before, so
enabled is always false. remove.

In the REENABLE case, make sur the fpexc copy in the pcb also has
VFP_FPEXC_EN set. Otherwise we could trap on every context switch even if
the CPU already has the VFP state.

branches: 1.53.2;
Recognize Cortex-A57 FPU, GIC, and Generic Timer.

in vfp_state_load(), fix backwards logic for fpinst vs. fpinst2.

allow pcu_save() and pcu_discard() to be called on other threads,
ptrace needs to use it that way.

branches: 1.50.2; 1.50.4;
Get the RPI3 working (in aarch32 mode) by recognising Cortex A53 CPUs.
While I'm here add some A57/A72 info as well.

My RPI3 works with FB console - the uart needs some help with its clocks.

change some register dumps from aprint_verbose to aprint_debug

isb after writing cpacr, from Andrew Turner

Fix some inverted return values. Don't return SIGILL if there is an active
FPU exception.

Remove extra )

Not only check to see if we own the VFP but that the VFP is enabled.

Don't try to catch undefined VFP instructions if we own the the FPU.
Let them raise SIGILL.

If we own the FPU, don't take anymore undefined faults. Instead generate
SIGILLs since we obviously don't understand the instruction.

Add VFP IDs for QEMU's emulated Cortex-A15.

branches: 1.41.2; 1.41.4;
fix typo reported in PR/48948

Cleanup a bit of the init logic.

pcu(9):
- Remove PCU_KERNEL (hi matt!) and significantly simplify the code.
This experimental feature was tried on ARM did not meet the expectations.
It may be revived one day, but it should be done in a much simpler way.
- Add a message structure for xcall function, pass the LWP ower and thus
optimise a race condition: if LWP is discarding its state on a remote CPU,
but another LWP already did it - do not cause an unecessary re-faulting.
- Reduce the variety of flags for PCU operations (only PCU_VALID and
PCU_REENABLE are used now), pass them only to the pcu_state_load().
- Rename pcu_used_p() to pcu_valid_p(); hopefully it is less confusing.
- pcu_save_all_on_cpu: SPL ought to be used here.
- Update and improve the pcu(9) man page; it needs wizd(8) though.

propogation -> propagation

branches: 1.37.2;
Various MP changes.

Enable VFP on MV88SV58XX

Add a different version vfp_fpscr_changable if FPU_VFP was not defined.
If no FPU was found, reinit vfp_fpscr_changeable/default to values appropriate
for softfloat.

Query the media and vfp feature registers to determine what our default
mode should be and what bits in the fpscr can be changed.
Print what features are supported:
vfp0 at cpu0: NEON MPE (VFP 3.0+), rounding, NaN propogation, denormals

Improve PCU/VFP handling to the point that the atf tests don't trigger
KASSERTs on the Raspberry PI and its arm1176jzf-s.

XXX Need to emulate bounce instructions to get correct exception codes,
XXX etc.

Be consistent about setting fpscr for Runfast. No functional change.

Fix typo in #define name

Typo in comment

Switch to using FP instructions instead of cp10/11 instructions.

If we can't enable VFP/VFP2 via the CPACCESS register, bail since there
isn't a VFP.

Before checking for an exception, make sure we own the VFP.

Deal with lack of VFP.

Reap LWP_VFPUSED and use PCU internal tracking.
Add bool vfp_used_p(void);

-extend the pcu(9) API by a function which saves all context on the
current CPU, and use it if a CPU is taken offline
-add a bool argument to pcu_discard which tells whether the internal
"LWP has used the coprocessor" flag should be set or reset. The flag
is reported by pcu_used_p(). If set, future accesses should use the
state stored in the PCB. If reset, it should be reset to default.
The former case is useful for setmcontext().
With that, it should not be necessary anymore to manage the "FPU used"
state by an additional MD variable.

approved by matt

Move parts of cpu.h that are not needed by MI code in <arm/locore.h>
Don't include <machine/cpu.h> or <machine/frame.h>, use <arm/locore.h>
Use <arm/asm.h> instead of <machine/arm.h>

Add VFP_FPSCR_RN (even though it's 0) just to be explicit.

Use armreg inlines.
Add exception -> trapsignal code.

branches: 1.20.2;
Add support for the Cortex-A15 Neon/VFP unit

Use the mrc form of the vmrs rX, mvfrX instruction to shut up gas.

Add support for machdep neon_present and id_mvfr sysctls

Add a machdep.fpu_present sysctl for ld.elf_so to use in ld.so.conf to load
libc_vfp.so.

Disable bzero_page_vfp and bcopy_page_vfp since it really isn't any faster
than memcpy.

Always re-enable the VFP when loading for a kernel LWP.

print the PC of the VFP kernel fault in the panic message.

Add support for PCU_KERNEL and vfp_kernel_acquire/vfp_kernel_release.
Add an undefined handler to catch NEON instructions.

Add code to patch pmap_{copy,zero}_page_generic to change calls to
b{copy,zero}_page to b{copy,zero}_page_vfp

move inlines into FPU_VFP

On Cortex, make sure to load/save the upper 16 64-FP registers.
When creating a mcontext_t, make sure _UC_ARM_VFP is set.

For armv7 (cortex), disable access to the upper 16 FP registers (restrict
the register space to 16 64-bit FP registers).

ARMFPE hasn't compiled since NetBSD 4. Remove it.
Complete support for FPU_VFP.
fpregs now contains vfpreg.
XXX vfpreg only has space for 16 64-bit FP registers though VFPv3 and later
have 32 64-bit FP registers.

Only use CPACR register for ARM11 and CORTEX cores.
Add VFP ids for other CORTEX CPUs.

Before testing for VFP, make sure CP10 is enabled. (And CP11 for Neon too).

branches: 1.5.2;
Add include of <arm/pcb.h>

Rework VFP support to use PCU.
Add emulation of instruction which save/restore the VFP FPSCR.
Add a sysarch hook to VFP FPSCR manipulation.

[The emulation will be used by libc to store/fetch exception modes and
rounding mode on a per-thread basis.]

branches: 1.3.12; 1.3.20;
Use lwp_getpcb() on ARM (and acorn26/32), clean from struct user usage.

Ansify function definitions w/o arguments. Generated with sed.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.12; 1.1.20; 1.1.26;
VFP support.

Retire arm_[di]sb in favour of the isb() and dsb(sy) macro invocations.

Add kthread_fpu_enter/exit support to arm.

Enable ChaCha NEON code on armv7 too.

The 4-blocks-at-a-time assembly helper is disabled for now; adapting
it to armv7 is going to be a little annoying with only 16 128-bit
vector registers.

(Should also do a fifth block in the integer registers for 320 bytes
at a time.)

Split aes_impl declarations out into aes_impl.h.

This will make it less painful to add more operations to struct
aes_impl without having to recompile everything that just uses the
block cipher directly or similar.

Use pcu_save_all_on_cpu, not pcu_save.

We don't care what curlwp is here; we care whose state is in the fpu
registers.

Limit arm32 fpu_kern_enter/leave to IPL_VM or below.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement fpu_kern_enter/leave for arm32.

Explicitly annotate FPU requirements for LLVM MC.

When using GCC, this annotations change the global state, but there is
no push/pop functionality for .fpu to avoid this problem. The state is
local to each inline assembler block with LLVM MC.

Cortex A12 is marketed as A17 but has a distinct part number

observed on Rockchip RK3288

Install the undefined instruction handlers only once, i.e. when attaching
on the BP.

Trailing whitespace

Merge the [pgoyette-compat] branch

Sprinkle #include "opt_cputypes.h"

Add __KERNEL_RCSID

branches: 1.57.2;
Remove the call to vfp_fpscr_handler() from vfp_handler(). It actually never
avoids a full FPU switch, and costs a function call and a few tests.

Discussed on port-arm@ on october 2017:
http://mail-index.netbsd.org/port-arm/2017/10/16/msg004411.html

branches: 1.56.2;
Add more vfp directives for gcc-6

We KASSERT((fregs->vfp_fpexc & VFP_FPEXC_EN) == 0) just before, so
enabled is always false. remove.

In the REENABLE case, make sur the fpexc copy in the pcb also has
VFP_FPEXC_EN set. Otherwise we could trap on every context switch even if
the CPU already has the VFP state.

branches: 1.53.2;
Recognize Cortex-A57 FPU, GIC, and Generic Timer.

in vfp_state_load(), fix backwards logic for fpinst vs. fpinst2.

allow pcu_save() and pcu_discard() to be called on other threads,
ptrace needs to use it that way.

branches: 1.50.2; 1.50.4;
Get the RPI3 working (in aarch32 mode) by recognising Cortex A53 CPUs.
While I'm here add some A57/A72 info as well.

My RPI3 works with FB console - the uart needs some help with its clocks.

change some register dumps from aprint_verbose to aprint_debug

isb after writing cpacr, from Andrew Turner

Fix some inverted return values. Don't return SIGILL if there is an active
FPU exception.

Remove extra )

Not only check to see if we own the VFP but that the VFP is enabled.

Don't try to catch undefined VFP instructions if we own the the FPU.
Let them raise SIGILL.

If we own the FPU, don't take anymore undefined faults. Instead generate
SIGILLs since we obviously don't understand the instruction.

Add VFP IDs for QEMU's emulated Cortex-A15.

branches: 1.41.2; 1.41.4;
fix typo reported in PR/48948

Cleanup a bit of the init logic.

pcu(9):
- Remove PCU_KERNEL (hi matt!) and significantly simplify the code.
This experimental feature was tried on ARM did not meet the expectations.
It may be revived one day, but it should be done in a much simpler way.
- Add a message structure for xcall function, pass the LWP ower and thus
optimise a race condition: if LWP is discarding its state on a remote CPU,
but another LWP already did it - do not cause an unecessary re-faulting.
- Reduce the variety of flags for PCU operations (only PCU_VALID and
PCU_REENABLE are used now), pass them only to the pcu_state_load().
- Rename pcu_used_p() to pcu_valid_p(); hopefully it is less confusing.
- pcu_save_all_on_cpu: SPL ought to be used here.
- Update and improve the pcu(9) man page; it needs wizd(8) though.

propogation -> propagation

branches: 1.37.2;
Various MP changes.

Enable VFP on MV88SV58XX

Add a different version vfp_fpscr_changable if FPU_VFP was not defined.
If no FPU was found, reinit vfp_fpscr_changeable/default to values appropriate
for softfloat.

Query the media and vfp feature registers to determine what our default
mode should be and what bits in the fpscr can be changed.
Print what features are supported:
vfp0 at cpu0: NEON MPE (VFP 3.0+), rounding, NaN propogation, denormals

Improve PCU/VFP handling to the point that the atf tests don't trigger
KASSERTs on the Raspberry PI and its arm1176jzf-s.

XXX Need to emulate bounce instructions to get correct exception codes,
XXX etc.

Be consistent about setting fpscr for Runfast. No functional change.

Fix typo in #define name

Typo in comment

Switch to using FP instructions instead of cp10/11 instructions.

If we can't enable VFP/VFP2 via the CPACCESS register, bail since there
isn't a VFP.

Before checking for an exception, make sure we own the VFP.

Deal with lack of VFP.

Reap LWP_VFPUSED and use PCU internal tracking.
Add bool vfp_used_p(void);

-extend the pcu(9) API by a function which saves all context on the
current CPU, and use it if a CPU is taken offline
-add a bool argument to pcu_discard which tells whether the internal
"LWP has used the coprocessor" flag should be set or reset. The flag
is reported by pcu_used_p(). If set, future accesses should use the
state stored in the PCB. If reset, it should be reset to default.
The former case is useful for setmcontext().
With that, it should not be necessary anymore to manage the "FPU used"
state by an additional MD variable.

approved by matt

Move parts of cpu.h that are not needed by MI code in <arm/locore.h>
Don't include <machine/cpu.h> or <machine/frame.h>, use <arm/locore.h>
Use <arm/asm.h> instead of <machine/arm.h>

Add VFP_FPSCR_RN (even though it's 0) just to be explicit.

Use armreg inlines.
Add exception -> trapsignal code.

branches: 1.20.2;
Add support for the Cortex-A15 Neon/VFP unit

Use the mrc form of the vmrs rX, mvfrX instruction to shut up gas.

Add support for machdep neon_present and id_mvfr sysctls