port-arm/58194: Resurrect vmt(4) from bitrot

On this architecture vmt(4) used to search for a node "/hypervisor" in the
FDT and probed the VMware hypervisor call only when the node was
found. However, things appear to have changed and VMware no longer provides
the FDT node.

Since vmt(4) doesn't actually need to read anything from FDT, and the
hypervisor call logically resides in virtual CPUs themselves, it would be
better to attach it directly to cpu, just like how it's probed on x86.

Change KDB to KGDB, including "sys/kgdb.h", which were likely meant to be defined.

Also comment out kgdb_machdep.c in files.aarch64, it doesn't exist yet.

Provide a shared pmap_devmap implementation and convert all pmap_devmap
arrays to use DEVMAP_ENTRY{,_END}

G/C

Provide MI PMAP support on AARCH64

MI PMAP EFI_RUNTIME support

Use "non-posted" instead of "strongly ordered" to describe nGnRnE mappings

Rename the following defines:
- _ARM_BUS_SPACE_MAP_STRONGLY_ORDERED to BUS_SPACE_MAP_NONPOSTED
- PMAP_DEV_SO to PMAP_DEV_NP
- LX_BLKPAG_ATTR_DEVICE_MEM_SO to LX_BLKPAG_ATTR_DEVICE_MEM_NP
Rename the following option:
- AARCH64_DEVICE_MEM_STRONGLY_ORDERED to AARCH64_DEVICE_MEM_NONPOSTED

Remove GIC_SPLFUNCS.

add support Hardware updates to Access flag and Dirty state (FEAT_HAFDBS)

- The DBM bit of the PTE is now used to determine if it is writable, and
the AF bit is treated entirely as a reference bit. A valid PTE is always
treated as readable. There can be no valid PTE that is not readable.
- LX_BLKPAG_OS_{READ,WRITE} are used only for debugging purposes,
and has been superseded by LX_BLKPAG_AF and LX_BLKPAG_DBM.
- Improve comment

The need for reference/modify emulation has been eliminated,
and access/permission faults have been reduced, however,
there has been little change in overall performance.

add support COMPAT_LINUX32 for aarch64

Implement gic_splraise and the gic_splx fast path in asm (armv8).

Use sys/uvm/pmap/pmap_tlb.c on Aarch64 in the same way that some Arm, MIPS,
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.

Additionally the following changes have been made

- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.

Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.

before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w

after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w

>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>

add support COMPAT_LINUX for aarch64

Arm: Add support for SMC Calling Convention

Arm DEN0028 defines a calling mechanism used with Secure Monitor Call (SMC)
and Hypervisor Call (HVC) instructions. To discover SMCCC, we must:

1) Find the PSCI conduit (either via ACPI FADT, or Device Tree)
2) Use PSCI_VERSION to determine whether PSCI_FEATURES is supported
3) Call PSCI_FEATURES with SMCCC_VERSION to determine the implementation
version.

aarch64: Remove empty source file and references to it.

branches: 1.30.6;
make process_machdep.c included always since it provides register i/o used by
sys_process_getlwpstatus.c which is always included.

harmonize process_machdep.c inclusion.

Collapse all CPU_CORTEXA<n> options into CPU_CORTEX and do runtime
detection instead of ifdefs where required.

Part III of ad's performance improvements for aarch64

- Assembly language stubs for mutex_enter() and mutex_exit().

Implement ChaCha with NEON on ARM.

XXX Needs performance measurement.
XXX Needs adaptation to arm32 neon which has half the registers.

Add options PMAPBOOT_DEBUG to dump TTBR when pmapboot_enter().
Formerly DEBUG_MMU in locore.S, but there was a bit of confusion.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement AES in kernel using ARMv8.0-AES on aarch64.

PMAP_DEBUG has been deleted on arm

Add KASAN-DMA support on aarch64, same as amd64. Discussed with skrll@.

branches: 1.20.4;
Various updates and improvements to cpu start up on arm/aarch64

- start sharing more code around the AP startup messaging.
- call arm_cpu_topology_set early so that ci_core_id is available for
drivers, e.g. bcm2835_intr.c
- both arm and aarch64 now have
- a static cpu_info_store array
- the same arm_cpu_{hatched,mbox}

add support pmap_pv(9)

Patch originally from jmcneill@. thanks

Small re-org. NFCI.

port the arm64 cpu topology setup for big.little to arm.

rename arm64 cpu_do_topology() to arm_cpu_do_topology() and
call it from both arm cpu_attach().

replace both aarch64_set_topology() inline code in arm
cpu_attach() with new arm_cpu_do_topology(), which is called
by the arm64 locore as well (possibly not needed, which would
allow it to become static.)

not yet tested on a real big.little armv7 system. tested
on rockpro64 and pinebook pro.

branches: 1.16.2;
Do not use Early Write Acknowledge for PCIe I/O and config space.

Enable early write acknowledge for device memory mappings.

Move all non-emulation-specific coredump code into the coredump module,
and remove all #ifdef COREDUMP conditional compilation. Now, the
coredump module is completely separated from the emulation modules, and
they can all be independently loaded and unloaded.

Welcome to 9.99.18 !

branches: 1.13.4;
Merge the [pgoyette-compat] branch

Add needs-flag to tprof_armv8.c

Split armv7/armv8 tprof backend config logic from the fdt bus glue.

Add CPU_THUNDERX which sets COHERENCY_UNIT and CACHE_LINE_SIZE to 128

Add support for EFI runtime services on aarch64.

Provide generic start code that assumes the MMU is off and caches are
disabled as per the linux booting protocol for ARMv6 and ARMv7 boards.
u-boot image type should be changed to 'linux' for correct behaviour.

The new start code builds a minimal "bootstrap" L1PT with cached access
disabled and uses the same table for all processors. AP startup is
performed in less steps and more code is written in C.

The bootstrap tables and stack are placed into an (orphaned) section
"_init_memory" which is given to uvm when it is no longer used.

Various kernels have been converted to use this code and tested. Some
boards were provided by TNF. Thanks!

The GENERIC kernel now boots on boards using the TEGRA, SUNXI and EXYNOS
kernels. The GENERIC kernel will also work on RPI2 using u-boot.

Thanks to martin@ and aymeric@ for testing on parallella and nanosoc
respectively

add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.

Whitespace

cleanup locore, and changed the way to map memories during boot.
- add functions bootpage_enter() and bootpage_alloc() to adapt various layout
of physical memory map. especially for 64bit physical memory layout.
pmapboot_alloc() allocates pagetable pages from _end[].
- changed to map only the required amount for PA=VA identity mapping
(kernel image, UART device, and FDT blob) with L2_BLOCK(2Mbyte).
- changing page permission for kernel image, and making KSEG mapping are done
at cpu_kernel_vm_init() instead of at locore.
- optimize PTE entries with PTE Contiguous bit. it is enabled on devmap only for now.

reviewed by skrll@, thanks.

catch up to files.arm's recent "opt_console.h" changes

branches: 1.3.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)

branches: 1.2.2;
retire copyinout.S and fusu.S

branches: 1.1.4; 1.1.6;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.

Change KDB to KGDB, including "sys/kgdb.h", which were likely meant to be defined.

Also comment out kgdb_machdep.c in files.aarch64, it doesn't exist yet.

Provide a shared pmap_devmap implementation and convert all pmap_devmap
arrays to use DEVMAP_ENTRY{,_END}

G/C

Provide MI PMAP support on AARCH64

MI PMAP EFI_RUNTIME support

Use "non-posted" instead of "strongly ordered" to describe nGnRnE mappings

Rename the following defines:
- _ARM_BUS_SPACE_MAP_STRONGLY_ORDERED to BUS_SPACE_MAP_NONPOSTED
- PMAP_DEV_SO to PMAP_DEV_NP
- LX_BLKPAG_ATTR_DEVICE_MEM_SO to LX_BLKPAG_ATTR_DEVICE_MEM_NP
Rename the following option:
- AARCH64_DEVICE_MEM_STRONGLY_ORDERED to AARCH64_DEVICE_MEM_NONPOSTED

Remove GIC_SPLFUNCS.

add support Hardware updates to Access flag and Dirty state (FEAT_HAFDBS)

- The DBM bit of the PTE is now used to determine if it is writable, and
the AF bit is treated entirely as a reference bit. A valid PTE is always
treated as readable. There can be no valid PTE that is not readable.
- LX_BLKPAG_OS_{READ,WRITE} are used only for debugging purposes,
and has been superseded by LX_BLKPAG_AF and LX_BLKPAG_DBM.
- Improve comment

The need for reference/modify emulation has been eliminated,
and access/permission faults have been reduced, however,
there has been little change in overall performance.

add support COMPAT_LINUX32 for aarch64

Implement gic_splraise and the gic_splx fast path in asm (armv8).

Use sys/uvm/pmap/pmap_tlb.c on Aarch64 in the same way that some Arm, MIPS,
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.

Additionally the following changes have been made

- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.

Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.

before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w

after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w

>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>

add support COMPAT_LINUX for aarch64

Arm: Add support for SMC Calling Convention

Arm DEN0028 defines a calling mechanism used with Secure Monitor Call (SMC)
and Hypervisor Call (HVC) instructions. To discover SMCCC, we must:

1) Find the PSCI conduit (either via ACPI FADT, or Device Tree)
2) Use PSCI_VERSION to determine whether PSCI_FEATURES is supported
3) Call PSCI_FEATURES with SMCCC_VERSION to determine the implementation
version.

aarch64: Remove empty source file and references to it.

branches: 1.30.6;
make process_machdep.c included always since it provides register i/o used by
sys_process_getlwpstatus.c which is always included.

harmonize process_machdep.c inclusion.

Collapse all CPU_CORTEXA<n> options into CPU_CORTEX and do runtime
detection instead of ifdefs where required.

Part III of ad's performance improvements for aarch64

- Assembly language stubs for mutex_enter() and mutex_exit().

Implement ChaCha with NEON on ARM.

XXX Needs performance measurement.
XXX Needs adaptation to arm32 neon which has half the registers.

Add options PMAPBOOT_DEBUG to dump TTBR when pmapboot_enter().
Formerly DEBUG_MMU in locore.S, but there was a bit of confusion.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement AES in kernel using ARMv8.0-AES on aarch64.

PMAP_DEBUG has been deleted on arm

Add KASAN-DMA support on aarch64, same as amd64. Discussed with skrll@.

branches: 1.20.4;
Various updates and improvements to cpu start up on arm/aarch64

- start sharing more code around the AP startup messaging.
- call arm_cpu_topology_set early so that ci_core_id is available for
drivers, e.g. bcm2835_intr.c
- both arm and aarch64 now have
- a static cpu_info_store array
- the same arm_cpu_{hatched,mbox}

add support pmap_pv(9)

Patch originally from jmcneill@. thanks

Small re-org. NFCI.

port the arm64 cpu topology setup for big.little to arm.

rename arm64 cpu_do_topology() to arm_cpu_do_topology() and
call it from both arm cpu_attach().

replace both aarch64_set_topology() inline code in arm
cpu_attach() with new arm_cpu_do_topology(), which is called
by the arm64 locore as well (possibly not needed, which would
allow it to become static.)

not yet tested on a real big.little armv7 system. tested
on rockpro64 and pinebook pro.

branches: 1.16.2;
Do not use Early Write Acknowledge for PCIe I/O and config space.

Enable early write acknowledge for device memory mappings.

Move all non-emulation-specific coredump code into the coredump module,
and remove all #ifdef COREDUMP conditional compilation. Now, the
coredump module is completely separated from the emulation modules, and
they can all be independently loaded and unloaded.

Welcome to 9.99.18 !

branches: 1.13.4;
Merge the [pgoyette-compat] branch

Add needs-flag to tprof_armv8.c

Split armv7/armv8 tprof backend config logic from the fdt bus glue.

Add CPU_THUNDERX which sets COHERENCY_UNIT and CACHE_LINE_SIZE to 128

Add support for EFI runtime services on aarch64.

Provide generic start code that assumes the MMU is off and caches are
disabled as per the linux booting protocol for ARMv6 and ARMv7 boards.
u-boot image type should be changed to 'linux' for correct behaviour.

The new start code builds a minimal "bootstrap" L1PT with cached access
disabled and uses the same table for all processors. AP startup is
performed in less steps and more code is written in C.

The bootstrap tables and stack are placed into an (orphaned) section
"_init_memory" which is given to uvm when it is no longer used.

Various kernels have been converted to use this code and tested. Some
boards were provided by TNF. Thanks!

The GENERIC kernel now boots on boards using the TEGRA, SUNXI and EXYNOS
kernels. The GENERIC kernel will also work on RPI2 using u-boot.

Thanks to martin@ and aymeric@ for testing on parallella and nanosoc
respectively

add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.

Whitespace

cleanup locore, and changed the way to map memories during boot.
- add functions bootpage_enter() and bootpage_alloc() to adapt various layout
of physical memory map. especially for 64bit physical memory layout.
pmapboot_alloc() allocates pagetable pages from _end[].
- changed to map only the required amount for PA=VA identity mapping
(kernel image, UART device, and FDT blob) with L2_BLOCK(2Mbyte).
- changing page permission for kernel image, and making KSEG mapping are done
at cpu_kernel_vm_init() instead of at locore.
- optimize PTE entries with PTE Contiguous bit. it is enabled on devmap only for now.

reviewed by skrll@, thanks.

catch up to files.arm's recent "opt_console.h" changes

branches: 1.3.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)

branches: 1.2.2;
retire copyinout.S and fusu.S

branches: 1.1.4; 1.1.6;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.

Provide a shared pmap_devmap implementation and convert all pmap_devmap
arrays to use DEVMAP_ENTRY{,_END}

G/C

Provide MI PMAP support on AARCH64

MI PMAP EFI_RUNTIME support

Use "non-posted" instead of "strongly ordered" to describe nGnRnE mappings

Rename the following defines:
- _ARM_BUS_SPACE_MAP_STRONGLY_ORDERED to BUS_SPACE_MAP_NONPOSTED
- PMAP_DEV_SO to PMAP_DEV_NP
- LX_BLKPAG_ATTR_DEVICE_MEM_SO to LX_BLKPAG_ATTR_DEVICE_MEM_NP
Rename the following option:
- AARCH64_DEVICE_MEM_STRONGLY_ORDERED to AARCH64_DEVICE_MEM_NONPOSTED

Remove GIC_SPLFUNCS.

add support Hardware updates to Access flag and Dirty state (FEAT_HAFDBS)

- The DBM bit of the PTE is now used to determine if it is writable, and
the AF bit is treated entirely as a reference bit. A valid PTE is always
treated as readable. There can be no valid PTE that is not readable.
- LX_BLKPAG_OS_{READ,WRITE} are used only for debugging purposes,
and has been superseded by LX_BLKPAG_AF and LX_BLKPAG_DBM.
- Improve comment

The need for reference/modify emulation has been eliminated,
and access/permission faults have been reduced, however,
there has been little change in overall performance.

add support COMPAT_LINUX32 for aarch64

Implement gic_splraise and the gic_splx fast path in asm (armv8).

Use sys/uvm/pmap/pmap_tlb.c on Aarch64 in the same way that some Arm, MIPS,
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.

Additionally the following changes have been made

- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.

Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.

before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w

after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w

>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>

add support COMPAT_LINUX for aarch64

Arm: Add support for SMC Calling Convention

Arm DEN0028 defines a calling mechanism used with Secure Monitor Call (SMC)
and Hypervisor Call (HVC) instructions. To discover SMCCC, we must:

1) Find the PSCI conduit (either via ACPI FADT, or Device Tree)
2) Use PSCI_VERSION to determine whether PSCI_FEATURES is supported
3) Call PSCI_FEATURES with SMCCC_VERSION to determine the implementation
version.

aarch64: Remove empty source file and references to it.

branches: 1.30.6;
make process_machdep.c included always since it provides register i/o used by
sys_process_getlwpstatus.c which is always included.

harmonize process_machdep.c inclusion.

Collapse all CPU_CORTEXA<n> options into CPU_CORTEX and do runtime
detection instead of ifdefs where required.

Part III of ad's performance improvements for aarch64

- Assembly language stubs for mutex_enter() and mutex_exit().

Implement ChaCha with NEON on ARM.

XXX Needs performance measurement.
XXX Needs adaptation to arm32 neon which has half the registers.

Add options PMAPBOOT_DEBUG to dump TTBR when pmapboot_enter().
Formerly DEBUG_MMU in locore.S, but there was a bit of confusion.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement AES in kernel using ARMv8.0-AES on aarch64.

PMAP_DEBUG has been deleted on arm

Add KASAN-DMA support on aarch64, same as amd64. Discussed with skrll@.

branches: 1.20.4;
Various updates and improvements to cpu start up on arm/aarch64

- start sharing more code around the AP startup messaging.
- call arm_cpu_topology_set early so that ci_core_id is available for
drivers, e.g. bcm2835_intr.c
- both arm and aarch64 now have
- a static cpu_info_store array
- the same arm_cpu_{hatched,mbox}

add support pmap_pv(9)

Patch originally from jmcneill@. thanks

Small re-org. NFCI.

port the arm64 cpu topology setup for big.little to arm.

rename arm64 cpu_do_topology() to arm_cpu_do_topology() and
call it from both arm cpu_attach().

replace both aarch64_set_topology() inline code in arm
cpu_attach() with new arm_cpu_do_topology(), which is called
by the arm64 locore as well (possibly not needed, which would
allow it to become static.)

not yet tested on a real big.little armv7 system. tested
on rockpro64 and pinebook pro.

branches: 1.16.2;
Do not use Early Write Acknowledge for PCIe I/O and config space.

Enable early write acknowledge for device memory mappings.

Move all non-emulation-specific coredump code into the coredump module,
and remove all #ifdef COREDUMP conditional compilation. Now, the
coredump module is completely separated from the emulation modules, and
they can all be independently loaded and unloaded.

Welcome to 9.99.18 !

branches: 1.13.4;
Merge the [pgoyette-compat] branch

Add needs-flag to tprof_armv8.c

Split armv7/armv8 tprof backend config logic from the fdt bus glue.

Add CPU_THUNDERX which sets COHERENCY_UNIT and CACHE_LINE_SIZE to 128

Add support for EFI runtime services on aarch64.

Provide generic start code that assumes the MMU is off and caches are
disabled as per the linux booting protocol for ARMv6 and ARMv7 boards.
u-boot image type should be changed to 'linux' for correct behaviour.

The new start code builds a minimal "bootstrap" L1PT with cached access
disabled and uses the same table for all processors. AP startup is
performed in less steps and more code is written in C.

The bootstrap tables and stack are placed into an (orphaned) section
"_init_memory" which is given to uvm when it is no longer used.

Various kernels have been converted to use this code and tested. Some
boards were provided by TNF. Thanks!

The GENERIC kernel now boots on boards using the TEGRA, SUNXI and EXYNOS
kernels. The GENERIC kernel will also work on RPI2 using u-boot.

Thanks to martin@ and aymeric@ for testing on parallella and nanosoc
respectively

add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.

Whitespace

cleanup locore, and changed the way to map memories during boot.
- add functions bootpage_enter() and bootpage_alloc() to adapt various layout
of physical memory map. especially for 64bit physical memory layout.
pmapboot_alloc() allocates pagetable pages from _end[].
- changed to map only the required amount for PA=VA identity mapping
(kernel image, UART device, and FDT blob) with L2_BLOCK(2Mbyte).
- changing page permission for kernel image, and making KSEG mapping are done
at cpu_kernel_vm_init() instead of at locore.
- optimize PTE entries with PTE Contiguous bit. it is enabled on devmap only for now.

reviewed by skrll@, thanks.

catch up to files.arm's recent "opt_console.h" changes

branches: 1.3.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)

branches: 1.2.2;
retire copyinout.S and fusu.S

branches: 1.1.4; 1.1.6;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.

G/C

Provide MI PMAP support on AARCH64

MI PMAP EFI_RUNTIME support

Use "non-posted" instead of "strongly ordered" to describe nGnRnE mappings

Rename the following defines:
- _ARM_BUS_SPACE_MAP_STRONGLY_ORDERED to BUS_SPACE_MAP_NONPOSTED
- PMAP_DEV_SO to PMAP_DEV_NP
- LX_BLKPAG_ATTR_DEVICE_MEM_SO to LX_BLKPAG_ATTR_DEVICE_MEM_NP
Rename the following option:
- AARCH64_DEVICE_MEM_STRONGLY_ORDERED to AARCH64_DEVICE_MEM_NONPOSTED

Remove GIC_SPLFUNCS.

add support Hardware updates to Access flag and Dirty state (FEAT_HAFDBS)

- The DBM bit of the PTE is now used to determine if it is writable, and
the AF bit is treated entirely as a reference bit. A valid PTE is always
treated as readable. There can be no valid PTE that is not readable.
- LX_BLKPAG_OS_{READ,WRITE} are used only for debugging purposes,
and has been superseded by LX_BLKPAG_AF and LX_BLKPAG_DBM.
- Improve comment

The need for reference/modify emulation has been eliminated,
and access/permission faults have been reduced, however,
there has been little change in overall performance.

add support COMPAT_LINUX32 for aarch64

Implement gic_splraise and the gic_splx fast path in asm (armv8).

Use sys/uvm/pmap/pmap_tlb.c on Aarch64 in the same way that some Arm, MIPS,
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.

Additionally the following changes have been made

- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.

Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.

before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w

after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w

>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>

add support COMPAT_LINUX for aarch64

Arm: Add support for SMC Calling Convention

Arm DEN0028 defines a calling mechanism used with Secure Monitor Call (SMC)
and Hypervisor Call (HVC) instructions. To discover SMCCC, we must:

1) Find the PSCI conduit (either via ACPI FADT, or Device Tree)
2) Use PSCI_VERSION to determine whether PSCI_FEATURES is supported
3) Call PSCI_FEATURES with SMCCC_VERSION to determine the implementation
version.

aarch64: Remove empty source file and references to it.

branches: 1.30.6;
make process_machdep.c included always since it provides register i/o used by
sys_process_getlwpstatus.c which is always included.

harmonize process_machdep.c inclusion.

Collapse all CPU_CORTEXA<n> options into CPU_CORTEX and do runtime
detection instead of ifdefs where required.

Part III of ad's performance improvements for aarch64

- Assembly language stubs for mutex_enter() and mutex_exit().

Implement ChaCha with NEON on ARM.

XXX Needs performance measurement.
XXX Needs adaptation to arm32 neon which has half the registers.

Add options PMAPBOOT_DEBUG to dump TTBR when pmapboot_enter().
Formerly DEBUG_MMU in locore.S, but there was a bit of confusion.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement AES in kernel using ARMv8.0-AES on aarch64.

PMAP_DEBUG has been deleted on arm

Add KASAN-DMA support on aarch64, same as amd64. Discussed with skrll@.

branches: 1.20.4;
Various updates and improvements to cpu start up on arm/aarch64

- start sharing more code around the AP startup messaging.
- call arm_cpu_topology_set early so that ci_core_id is available for
drivers, e.g. bcm2835_intr.c
- both arm and aarch64 now have
- a static cpu_info_store array
- the same arm_cpu_{hatched,mbox}

add support pmap_pv(9)

Patch originally from jmcneill@. thanks

Small re-org. NFCI.

port the arm64 cpu topology setup for big.little to arm.

rename arm64 cpu_do_topology() to arm_cpu_do_topology() and
call it from both arm cpu_attach().

replace both aarch64_set_topology() inline code in arm
cpu_attach() with new arm_cpu_do_topology(), which is called
by the arm64 locore as well (possibly not needed, which would
allow it to become static.)

not yet tested on a real big.little armv7 system. tested
on rockpro64 and pinebook pro.

branches: 1.16.2;
Do not use Early Write Acknowledge for PCIe I/O and config space.

Enable early write acknowledge for device memory mappings.

Move all non-emulation-specific coredump code into the coredump module,
and remove all #ifdef COREDUMP conditional compilation. Now, the
coredump module is completely separated from the emulation modules, and
they can all be independently loaded and unloaded.

Welcome to 9.99.18 !

branches: 1.13.4;
Merge the [pgoyette-compat] branch

Add needs-flag to tprof_armv8.c

Split armv7/armv8 tprof backend config logic from the fdt bus glue.

Add CPU_THUNDERX which sets COHERENCY_UNIT and CACHE_LINE_SIZE to 128

Add support for EFI runtime services on aarch64.

Provide generic start code that assumes the MMU is off and caches are
disabled as per the linux booting protocol for ARMv6 and ARMv7 boards.
u-boot image type should be changed to 'linux' for correct behaviour.

The new start code builds a minimal "bootstrap" L1PT with cached access
disabled and uses the same table for all processors. AP startup is
performed in less steps and more code is written in C.

The bootstrap tables and stack are placed into an (orphaned) section
"_init_memory" which is given to uvm when it is no longer used.

Various kernels have been converted to use this code and tested. Some
boards were provided by TNF. Thanks!

The GENERIC kernel now boots on boards using the TEGRA, SUNXI and EXYNOS
kernels. The GENERIC kernel will also work on RPI2 using u-boot.

Thanks to martin@ and aymeric@ for testing on parallella and nanosoc
respectively

add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.

Whitespace

cleanup locore, and changed the way to map memories during boot.
- add functions bootpage_enter() and bootpage_alloc() to adapt various layout
of physical memory map. especially for 64bit physical memory layout.
pmapboot_alloc() allocates pagetable pages from _end[].
- changed to map only the required amount for PA=VA identity mapping
(kernel image, UART device, and FDT blob) with L2_BLOCK(2Mbyte).
- changing page permission for kernel image, and making KSEG mapping are done
at cpu_kernel_vm_init() instead of at locore.
- optimize PTE entries with PTE Contiguous bit. it is enabled on devmap only for now.

reviewed by skrll@, thanks.

catch up to files.arm's recent "opt_console.h" changes

branches: 1.3.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)

branches: 1.2.2;
retire copyinout.S and fusu.S

branches: 1.1.4; 1.1.6;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.

Provide MI PMAP support on AARCH64

MI PMAP EFI_RUNTIME support

Use "non-posted" instead of "strongly ordered" to describe nGnRnE mappings

Rename the following defines:
- _ARM_BUS_SPACE_MAP_STRONGLY_ORDERED to BUS_SPACE_MAP_NONPOSTED
- PMAP_DEV_SO to PMAP_DEV_NP
- LX_BLKPAG_ATTR_DEVICE_MEM_SO to LX_BLKPAG_ATTR_DEVICE_MEM_NP
Rename the following option:
- AARCH64_DEVICE_MEM_STRONGLY_ORDERED to AARCH64_DEVICE_MEM_NONPOSTED

Remove GIC_SPLFUNCS.

add support Hardware updates to Access flag and Dirty state (FEAT_HAFDBS)

- The DBM bit of the PTE is now used to determine if it is writable, and
the AF bit is treated entirely as a reference bit. A valid PTE is always
treated as readable. There can be no valid PTE that is not readable.
- LX_BLKPAG_OS_{READ,WRITE} are used only for debugging purposes,
and has been superseded by LX_BLKPAG_AF and LX_BLKPAG_DBM.
- Improve comment

The need for reference/modify emulation has been eliminated,
and access/permission faults have been reduced, however,
there has been little change in overall performance.

add support COMPAT_LINUX32 for aarch64

Implement gic_splraise and the gic_splx fast path in asm (armv8).

Use sys/uvm/pmap/pmap_tlb.c on Aarch64 in the same way that some Arm, MIPS,
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.

Additionally the following changes have been made

- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.

Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.

before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w

after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w

>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>

add support COMPAT_LINUX for aarch64

Arm: Add support for SMC Calling Convention

Arm DEN0028 defines a calling mechanism used with Secure Monitor Call (SMC)
and Hypervisor Call (HVC) instructions. To discover SMCCC, we must:

1) Find the PSCI conduit (either via ACPI FADT, or Device Tree)
2) Use PSCI_VERSION to determine whether PSCI_FEATURES is supported
3) Call PSCI_FEATURES with SMCCC_VERSION to determine the implementation
version.

aarch64: Remove empty source file and references to it.

branches: 1.30.6;
make process_machdep.c included always since it provides register i/o used by
sys_process_getlwpstatus.c which is always included.

harmonize process_machdep.c inclusion.

Collapse all CPU_CORTEXA<n> options into CPU_CORTEX and do runtime
detection instead of ifdefs where required.

Part III of ad's performance improvements for aarch64

- Assembly language stubs for mutex_enter() and mutex_exit().

Implement ChaCha with NEON on ARM.

XXX Needs performance measurement.
XXX Needs adaptation to arm32 neon which has half the registers.

Add options PMAPBOOT_DEBUG to dump TTBR when pmapboot_enter().
Formerly DEBUG_MMU in locore.S, but there was a bit of confusion.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement AES in kernel using ARMv8.0-AES on aarch64.

PMAP_DEBUG has been deleted on arm

Add KASAN-DMA support on aarch64, same as amd64. Discussed with skrll@.

branches: 1.20.4;
Various updates and improvements to cpu start up on arm/aarch64

- start sharing more code around the AP startup messaging.
- call arm_cpu_topology_set early so that ci_core_id is available for
drivers, e.g. bcm2835_intr.c
- both arm and aarch64 now have
- a static cpu_info_store array
- the same arm_cpu_{hatched,mbox}

add support pmap_pv(9)

Patch originally from jmcneill@. thanks

Small re-org. NFCI.

port the arm64 cpu topology setup for big.little to arm.

rename arm64 cpu_do_topology() to arm_cpu_do_topology() and
call it from both arm cpu_attach().

replace both aarch64_set_topology() inline code in arm
cpu_attach() with new arm_cpu_do_topology(), which is called
by the arm64 locore as well (possibly not needed, which would
allow it to become static.)

not yet tested on a real big.little armv7 system. tested
on rockpro64 and pinebook pro.

branches: 1.16.2;
Do not use Early Write Acknowledge for PCIe I/O and config space.

Enable early write acknowledge for device memory mappings.

Move all non-emulation-specific coredump code into the coredump module,
and remove all #ifdef COREDUMP conditional compilation. Now, the
coredump module is completely separated from the emulation modules, and
they can all be independently loaded and unloaded.

Welcome to 9.99.18 !

branches: 1.13.4;
Merge the [pgoyette-compat] branch

Add needs-flag to tprof_armv8.c

Split armv7/armv8 tprof backend config logic from the fdt bus glue.

Add CPU_THUNDERX which sets COHERENCY_UNIT and CACHE_LINE_SIZE to 128

Add support for EFI runtime services on aarch64.

Provide generic start code that assumes the MMU is off and caches are
disabled as per the linux booting protocol for ARMv6 and ARMv7 boards.
u-boot image type should be changed to 'linux' for correct behaviour.

The new start code builds a minimal "bootstrap" L1PT with cached access
disabled and uses the same table for all processors. AP startup is
performed in less steps and more code is written in C.

The bootstrap tables and stack are placed into an (orphaned) section
"_init_memory" which is given to uvm when it is no longer used.

Various kernels have been converted to use this code and tested. Some
boards were provided by TNF. Thanks!

The GENERIC kernel now boots on boards using the TEGRA, SUNXI and EXYNOS
kernels. The GENERIC kernel will also work on RPI2 using u-boot.

Thanks to martin@ and aymeric@ for testing on parallella and nanosoc
respectively

add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.

Whitespace

cleanup locore, and changed the way to map memories during boot.
- add functions bootpage_enter() and bootpage_alloc() to adapt various layout
of physical memory map. especially for 64bit physical memory layout.
pmapboot_alloc() allocates pagetable pages from _end[].
- changed to map only the required amount for PA=VA identity mapping
(kernel image, UART device, and FDT blob) with L2_BLOCK(2Mbyte).
- changing page permission for kernel image, and making KSEG mapping are done
at cpu_kernel_vm_init() instead of at locore.
- optimize PTE entries with PTE Contiguous bit. it is enabled on devmap only for now.

reviewed by skrll@, thanks.

catch up to files.arm's recent "opt_console.h" changes

branches: 1.3.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)

branches: 1.2.2;
retire copyinout.S and fusu.S

branches: 1.1.4; 1.1.6;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.

MI PMAP EFI_RUNTIME support

Use "non-posted" instead of "strongly ordered" to describe nGnRnE mappings

Rename the following defines:
- _ARM_BUS_SPACE_MAP_STRONGLY_ORDERED to BUS_SPACE_MAP_NONPOSTED
- PMAP_DEV_SO to PMAP_DEV_NP
- LX_BLKPAG_ATTR_DEVICE_MEM_SO to LX_BLKPAG_ATTR_DEVICE_MEM_NP
Rename the following option:
- AARCH64_DEVICE_MEM_STRONGLY_ORDERED to AARCH64_DEVICE_MEM_NONPOSTED

Remove GIC_SPLFUNCS.

add support Hardware updates to Access flag and Dirty state (FEAT_HAFDBS)

- The DBM bit of the PTE is now used to determine if it is writable, and
the AF bit is treated entirely as a reference bit. A valid PTE is always
treated as readable. There can be no valid PTE that is not readable.
- LX_BLKPAG_OS_{READ,WRITE} are used only for debugging purposes,
and has been superseded by LX_BLKPAG_AF and LX_BLKPAG_DBM.
- Improve comment

The need for reference/modify emulation has been eliminated,
and access/permission faults have been reduced, however,
there has been little change in overall performance.

add support COMPAT_LINUX32 for aarch64

Implement gic_splraise and the gic_splx fast path in asm (armv8).

Use sys/uvm/pmap/pmap_tlb.c on Aarch64 in the same way that some Arm, MIPS,
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.

Additionally the following changes have been made

- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.

Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.

before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w

after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w

>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>

add support COMPAT_LINUX for aarch64

Arm: Add support for SMC Calling Convention

Arm DEN0028 defines a calling mechanism used with Secure Monitor Call (SMC)
and Hypervisor Call (HVC) instructions. To discover SMCCC, we must:

1) Find the PSCI conduit (either via ACPI FADT, or Device Tree)
2) Use PSCI_VERSION to determine whether PSCI_FEATURES is supported
3) Call PSCI_FEATURES with SMCCC_VERSION to determine the implementation
version.

aarch64: Remove empty source file and references to it.

branches: 1.30.6;
make process_machdep.c included always since it provides register i/o used by
sys_process_getlwpstatus.c which is always included.

harmonize process_machdep.c inclusion.

Collapse all CPU_CORTEXA<n> options into CPU_CORTEX and do runtime
detection instead of ifdefs where required.

Part III of ad's performance improvements for aarch64

- Assembly language stubs for mutex_enter() and mutex_exit().

Implement ChaCha with NEON on ARM.

XXX Needs performance measurement.
XXX Needs adaptation to arm32 neon which has half the registers.

Add options PMAPBOOT_DEBUG to dump TTBR when pmapboot_enter().
Formerly DEBUG_MMU in locore.S, but there was a bit of confusion.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement AES in kernel using ARMv8.0-AES on aarch64.

PMAP_DEBUG has been deleted on arm

Add KASAN-DMA support on aarch64, same as amd64. Discussed with skrll@.

branches: 1.20.4;
Various updates and improvements to cpu start up on arm/aarch64

- start sharing more code around the AP startup messaging.
- call arm_cpu_topology_set early so that ci_core_id is available for
drivers, e.g. bcm2835_intr.c
- both arm and aarch64 now have
- a static cpu_info_store array
- the same arm_cpu_{hatched,mbox}

add support pmap_pv(9)

Patch originally from jmcneill@. thanks

Small re-org. NFCI.

port the arm64 cpu topology setup for big.little to arm.

rename arm64 cpu_do_topology() to arm_cpu_do_topology() and
call it from both arm cpu_attach().

replace both aarch64_set_topology() inline code in arm
cpu_attach() with new arm_cpu_do_topology(), which is called
by the arm64 locore as well (possibly not needed, which would
allow it to become static.)

not yet tested on a real big.little armv7 system. tested
on rockpro64 and pinebook pro.

branches: 1.16.2;
Do not use Early Write Acknowledge for PCIe I/O and config space.

Enable early write acknowledge for device memory mappings.

Move all non-emulation-specific coredump code into the coredump module,
and remove all #ifdef COREDUMP conditional compilation. Now, the
coredump module is completely separated from the emulation modules, and
they can all be independently loaded and unloaded.

Welcome to 9.99.18 !

branches: 1.13.4;
Merge the [pgoyette-compat] branch

Add needs-flag to tprof_armv8.c

Split armv7/armv8 tprof backend config logic from the fdt bus glue.

Add CPU_THUNDERX which sets COHERENCY_UNIT and CACHE_LINE_SIZE to 128

Add support for EFI runtime services on aarch64.

Provide generic start code that assumes the MMU is off and caches are
disabled as per the linux booting protocol for ARMv6 and ARMv7 boards.
u-boot image type should be changed to 'linux' for correct behaviour.

The new start code builds a minimal "bootstrap" L1PT with cached access
disabled and uses the same table for all processors. AP startup is
performed in less steps and more code is written in C.

The bootstrap tables and stack are placed into an (orphaned) section
"_init_memory" which is given to uvm when it is no longer used.

Various kernels have been converted to use this code and tested. Some
boards were provided by TNF. Thanks!

The GENERIC kernel now boots on boards using the TEGRA, SUNXI and EXYNOS
kernels. The GENERIC kernel will also work on RPI2 using u-boot.

Thanks to martin@ and aymeric@ for testing on parallella and nanosoc
respectively

add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.

Whitespace

cleanup locore, and changed the way to map memories during boot.
- add functions bootpage_enter() and bootpage_alloc() to adapt various layout
of physical memory map. especially for 64bit physical memory layout.
pmapboot_alloc() allocates pagetable pages from _end[].
- changed to map only the required amount for PA=VA identity mapping
(kernel image, UART device, and FDT blob) with L2_BLOCK(2Mbyte).
- changing page permission for kernel image, and making KSEG mapping are done
at cpu_kernel_vm_init() instead of at locore.
- optimize PTE entries with PTE Contiguous bit. it is enabled on devmap only for now.

reviewed by skrll@, thanks.

catch up to files.arm's recent "opt_console.h" changes

branches: 1.3.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)

branches: 1.2.2;
retire copyinout.S and fusu.S

branches: 1.1.4; 1.1.6;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.

Use "non-posted" instead of "strongly ordered" to describe nGnRnE mappings

Rename the following defines:
- _ARM_BUS_SPACE_MAP_STRONGLY_ORDERED to BUS_SPACE_MAP_NONPOSTED
- PMAP_DEV_SO to PMAP_DEV_NP
- LX_BLKPAG_ATTR_DEVICE_MEM_SO to LX_BLKPAG_ATTR_DEVICE_MEM_NP
Rename the following option:
- AARCH64_DEVICE_MEM_STRONGLY_ORDERED to AARCH64_DEVICE_MEM_NONPOSTED

Remove GIC_SPLFUNCS.

add support Hardware updates to Access flag and Dirty state (FEAT_HAFDBS)

- The DBM bit of the PTE is now used to determine if it is writable, and
the AF bit is treated entirely as a reference bit. A valid PTE is always
treated as readable. There can be no valid PTE that is not readable.
- LX_BLKPAG_OS_{READ,WRITE} are used only for debugging purposes,
and has been superseded by LX_BLKPAG_AF and LX_BLKPAG_DBM.
- Improve comment

The need for reference/modify emulation has been eliminated,
and access/permission faults have been reduced, however,
there has been little change in overall performance.

add support COMPAT_LINUX32 for aarch64

Implement gic_splraise and the gic_splx fast path in asm (armv8).

Use sys/uvm/pmap/pmap_tlb.c on Aarch64 in the same way that some Arm, MIPS,
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.

Additionally the following changes have been made

- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.

Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.

before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w

after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w

>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>

add support COMPAT_LINUX for aarch64

Arm: Add support for SMC Calling Convention

Arm DEN0028 defines a calling mechanism used with Secure Monitor Call (SMC)
and Hypervisor Call (HVC) instructions. To discover SMCCC, we must:

1) Find the PSCI conduit (either via ACPI FADT, or Device Tree)
2) Use PSCI_VERSION to determine whether PSCI_FEATURES is supported
3) Call PSCI_FEATURES with SMCCC_VERSION to determine the implementation
version.

aarch64: Remove empty source file and references to it.

branches: 1.30.6;
make process_machdep.c included always since it provides register i/o used by
sys_process_getlwpstatus.c which is always included.

harmonize process_machdep.c inclusion.

Collapse all CPU_CORTEXA<n> options into CPU_CORTEX and do runtime
detection instead of ifdefs where required.

Part III of ad's performance improvements for aarch64

- Assembly language stubs for mutex_enter() and mutex_exit().

Implement ChaCha with NEON on ARM.

XXX Needs performance measurement.
XXX Needs adaptation to arm32 neon which has half the registers.

Add options PMAPBOOT_DEBUG to dump TTBR when pmapboot_enter().
Formerly DEBUG_MMU in locore.S, but there was a bit of confusion.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement AES in kernel using ARMv8.0-AES on aarch64.

PMAP_DEBUG has been deleted on arm

Add KASAN-DMA support on aarch64, same as amd64. Discussed with skrll@.

branches: 1.20.4;
Various updates and improvements to cpu start up on arm/aarch64

- start sharing more code around the AP startup messaging.
- call arm_cpu_topology_set early so that ci_core_id is available for
drivers, e.g. bcm2835_intr.c
- both arm and aarch64 now have
- a static cpu_info_store array
- the same arm_cpu_{hatched,mbox}

add support pmap_pv(9)

Patch originally from jmcneill@. thanks

Small re-org. NFCI.

port the arm64 cpu topology setup for big.little to arm.

rename arm64 cpu_do_topology() to arm_cpu_do_topology() and
call it from both arm cpu_attach().

replace both aarch64_set_topology() inline code in arm
cpu_attach() with new arm_cpu_do_topology(), which is called
by the arm64 locore as well (possibly not needed, which would
allow it to become static.)

not yet tested on a real big.little armv7 system. tested
on rockpro64 and pinebook pro.

branches: 1.16.2;
Do not use Early Write Acknowledge for PCIe I/O and config space.

Enable early write acknowledge for device memory mappings.

Move all non-emulation-specific coredump code into the coredump module,
and remove all #ifdef COREDUMP conditional compilation. Now, the
coredump module is completely separated from the emulation modules, and
they can all be independently loaded and unloaded.

Welcome to 9.99.18 !

branches: 1.13.4;
Merge the [pgoyette-compat] branch

Add needs-flag to tprof_armv8.c

Split armv7/armv8 tprof backend config logic from the fdt bus glue.

Add CPU_THUNDERX which sets COHERENCY_UNIT and CACHE_LINE_SIZE to 128

Add support for EFI runtime services on aarch64.

Provide generic start code that assumes the MMU is off and caches are
disabled as per the linux booting protocol for ARMv6 and ARMv7 boards.
u-boot image type should be changed to 'linux' for correct behaviour.

The new start code builds a minimal "bootstrap" L1PT with cached access
disabled and uses the same table for all processors. AP startup is
performed in less steps and more code is written in C.

The bootstrap tables and stack are placed into an (orphaned) section
"_init_memory" which is given to uvm when it is no longer used.

Various kernels have been converted to use this code and tested. Some
boards were provided by TNF. Thanks!

The GENERIC kernel now boots on boards using the TEGRA, SUNXI and EXYNOS
kernels. The GENERIC kernel will also work on RPI2 using u-boot.

Thanks to martin@ and aymeric@ for testing on parallella and nanosoc
respectively

add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.

Whitespace

cleanup locore, and changed the way to map memories during boot.
- add functions bootpage_enter() and bootpage_alloc() to adapt various layout
of physical memory map. especially for 64bit physical memory layout.
pmapboot_alloc() allocates pagetable pages from _end[].
- changed to map only the required amount for PA=VA identity mapping
(kernel image, UART device, and FDT blob) with L2_BLOCK(2Mbyte).
- changing page permission for kernel image, and making KSEG mapping are done
at cpu_kernel_vm_init() instead of at locore.
- optimize PTE entries with PTE Contiguous bit. it is enabled on devmap only for now.

reviewed by skrll@, thanks.

catch up to files.arm's recent "opt_console.h" changes

branches: 1.3.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)

branches: 1.2.2;
retire copyinout.S and fusu.S

branches: 1.1.4; 1.1.6;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.

Remove GIC_SPLFUNCS.

add support Hardware updates to Access flag and Dirty state (FEAT_HAFDBS)

- The DBM bit of the PTE is now used to determine if it is writable, and
the AF bit is treated entirely as a reference bit. A valid PTE is always
treated as readable. There can be no valid PTE that is not readable.
- LX_BLKPAG_OS_{READ,WRITE} are used only for debugging purposes,
and has been superseded by LX_BLKPAG_AF and LX_BLKPAG_DBM.
- Improve comment

The need for reference/modify emulation has been eliminated,
and access/permission faults have been reduced, however,
there has been little change in overall performance.

add support COMPAT_LINUX32 for aarch64

Implement gic_splraise and the gic_splx fast path in asm (armv8).

Use sys/uvm/pmap/pmap_tlb.c on Aarch64 in the same way that some Arm, MIPS,
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.

Additionally the following changes have been made

- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.

Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.

before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w

after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w

>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>

add support COMPAT_LINUX for aarch64

Arm: Add support for SMC Calling Convention

Arm DEN0028 defines a calling mechanism used with Secure Monitor Call (SMC)
and Hypervisor Call (HVC) instructions. To discover SMCCC, we must:

1) Find the PSCI conduit (either via ACPI FADT, or Device Tree)
2) Use PSCI_VERSION to determine whether PSCI_FEATURES is supported
3) Call PSCI_FEATURES with SMCCC_VERSION to determine the implementation
version.

aarch64: Remove empty source file and references to it.

branches: 1.30.6;
make process_machdep.c included always since it provides register i/o used by
sys_process_getlwpstatus.c which is always included.

harmonize process_machdep.c inclusion.

Collapse all CPU_CORTEXA<n> options into CPU_CORTEX and do runtime
detection instead of ifdefs where required.

Part III of ad's performance improvements for aarch64

- Assembly language stubs for mutex_enter() and mutex_exit().

Implement ChaCha with NEON on ARM.

XXX Needs performance measurement.
XXX Needs adaptation to arm32 neon which has half the registers.

Add options PMAPBOOT_DEBUG to dump TTBR when pmapboot_enter().
Formerly DEBUG_MMU in locore.S, but there was a bit of confusion.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement AES in kernel using ARMv8.0-AES on aarch64.

PMAP_DEBUG has been deleted on arm

Add KASAN-DMA support on aarch64, same as amd64. Discussed with skrll@.

branches: 1.20.4;
Various updates and improvements to cpu start up on arm/aarch64

- start sharing more code around the AP startup messaging.
- call arm_cpu_topology_set early so that ci_core_id is available for
drivers, e.g. bcm2835_intr.c
- both arm and aarch64 now have
- a static cpu_info_store array
- the same arm_cpu_{hatched,mbox}

add support pmap_pv(9)

Patch originally from jmcneill@. thanks

Small re-org. NFCI.

port the arm64 cpu topology setup for big.little to arm.

rename arm64 cpu_do_topology() to arm_cpu_do_topology() and
call it from both arm cpu_attach().

replace both aarch64_set_topology() inline code in arm
cpu_attach() with new arm_cpu_do_topology(), which is called
by the arm64 locore as well (possibly not needed, which would
allow it to become static.)

not yet tested on a real big.little armv7 system. tested
on rockpro64 and pinebook pro.

branches: 1.16.2;
Do not use Early Write Acknowledge for PCIe I/O and config space.

Enable early write acknowledge for device memory mappings.

Move all non-emulation-specific coredump code into the coredump module,
and remove all #ifdef COREDUMP conditional compilation. Now, the
coredump module is completely separated from the emulation modules, and
they can all be independently loaded and unloaded.

Welcome to 9.99.18 !

branches: 1.13.4;
Merge the [pgoyette-compat] branch

Add needs-flag to tprof_armv8.c

Split armv7/armv8 tprof backend config logic from the fdt bus glue.

Add CPU_THUNDERX which sets COHERENCY_UNIT and CACHE_LINE_SIZE to 128

Add support for EFI runtime services on aarch64.

Provide generic start code that assumes the MMU is off and caches are
disabled as per the linux booting protocol for ARMv6 and ARMv7 boards.
u-boot image type should be changed to 'linux' for correct behaviour.

The new start code builds a minimal "bootstrap" L1PT with cached access
disabled and uses the same table for all processors. AP startup is
performed in less steps and more code is written in C.

The bootstrap tables and stack are placed into an (orphaned) section
"_init_memory" which is given to uvm when it is no longer used.

Various kernels have been converted to use this code and tested. Some
boards were provided by TNF. Thanks!

The GENERIC kernel now boots on boards using the TEGRA, SUNXI and EXYNOS
kernels. The GENERIC kernel will also work on RPI2 using u-boot.

Thanks to martin@ and aymeric@ for testing on parallella and nanosoc
respectively

add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.

Whitespace

cleanup locore, and changed the way to map memories during boot.
- add functions bootpage_enter() and bootpage_alloc() to adapt various layout
of physical memory map. especially for 64bit physical memory layout.
pmapboot_alloc() allocates pagetable pages from _end[].
- changed to map only the required amount for PA=VA identity mapping
(kernel image, UART device, and FDT blob) with L2_BLOCK(2Mbyte).
- changing page permission for kernel image, and making KSEG mapping are done
at cpu_kernel_vm_init() instead of at locore.
- optimize PTE entries with PTE Contiguous bit. it is enabled on devmap only for now.

reviewed by skrll@, thanks.

catch up to files.arm's recent "opt_console.h" changes

branches: 1.3.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)

branches: 1.2.2;
retire copyinout.S and fusu.S

branches: 1.1.4; 1.1.6;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.

add support Hardware updates to Access flag and Dirty state (FEAT_HAFDBS)

- The DBM bit of the PTE is now used to determine if it is writable, and
the AF bit is treated entirely as a reference bit. A valid PTE is always
treated as readable. There can be no valid PTE that is not readable.
- LX_BLKPAG_OS_{READ,WRITE} are used only for debugging purposes,
and has been superseded by LX_BLKPAG_AF and LX_BLKPAG_DBM.
- Improve comment

The need for reference/modify emulation has been eliminated,
and access/permission faults have been reduced, however,
there has been little change in overall performance.

add support COMPAT_LINUX32 for aarch64

Implement gic_splraise and the gic_splx fast path in asm (armv8).

Use sys/uvm/pmap/pmap_tlb.c on Aarch64 in the same way that some Arm, MIPS,
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.

Additionally the following changes have been made

- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.

Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.

before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w

after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w

>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>

add support COMPAT_LINUX for aarch64

Arm: Add support for SMC Calling Convention

Arm DEN0028 defines a calling mechanism used with Secure Monitor Call (SMC)
and Hypervisor Call (HVC) instructions. To discover SMCCC, we must:

1) Find the PSCI conduit (either via ACPI FADT, or Device Tree)
2) Use PSCI_VERSION to determine whether PSCI_FEATURES is supported
3) Call PSCI_FEATURES with SMCCC_VERSION to determine the implementation
version.

aarch64: Remove empty source file and references to it.

branches: 1.30.6;
make process_machdep.c included always since it provides register i/o used by
sys_process_getlwpstatus.c which is always included.

harmonize process_machdep.c inclusion.

Collapse all CPU_CORTEXA<n> options into CPU_CORTEX and do runtime
detection instead of ifdefs where required.

Part III of ad's performance improvements for aarch64

- Assembly language stubs for mutex_enter() and mutex_exit().

Implement ChaCha with NEON on ARM.

XXX Needs performance measurement.
XXX Needs adaptation to arm32 neon which has half the registers.

Add options PMAPBOOT_DEBUG to dump TTBR when pmapboot_enter().
Formerly DEBUG_MMU in locore.S, but there was a bit of confusion.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement AES in kernel using ARMv8.0-AES on aarch64.

PMAP_DEBUG has been deleted on arm

Add KASAN-DMA support on aarch64, same as amd64. Discussed with skrll@.

branches: 1.20.4;
Various updates and improvements to cpu start up on arm/aarch64

- start sharing more code around the AP startup messaging.
- call arm_cpu_topology_set early so that ci_core_id is available for
drivers, e.g. bcm2835_intr.c
- both arm and aarch64 now have
- a static cpu_info_store array
- the same arm_cpu_{hatched,mbox}

add support pmap_pv(9)

Patch originally from jmcneill@. thanks

Small re-org. NFCI.

port the arm64 cpu topology setup for big.little to arm.

rename arm64 cpu_do_topology() to arm_cpu_do_topology() and
call it from both arm cpu_attach().

replace both aarch64_set_topology() inline code in arm
cpu_attach() with new arm_cpu_do_topology(), which is called
by the arm64 locore as well (possibly not needed, which would
allow it to become static.)

not yet tested on a real big.little armv7 system. tested
on rockpro64 and pinebook pro.

branches: 1.16.2;
Do not use Early Write Acknowledge for PCIe I/O and config space.

Enable early write acknowledge for device memory mappings.

Move all non-emulation-specific coredump code into the coredump module,
and remove all #ifdef COREDUMP conditional compilation. Now, the
coredump module is completely separated from the emulation modules, and
they can all be independently loaded and unloaded.

Welcome to 9.99.18 !

branches: 1.13.4;
Merge the [pgoyette-compat] branch

Add needs-flag to tprof_armv8.c

Split armv7/armv8 tprof backend config logic from the fdt bus glue.

Add CPU_THUNDERX which sets COHERENCY_UNIT and CACHE_LINE_SIZE to 128

Add support for EFI runtime services on aarch64.

Provide generic start code that assumes the MMU is off and caches are
disabled as per the linux booting protocol for ARMv6 and ARMv7 boards.
u-boot image type should be changed to 'linux' for correct behaviour.

The new start code builds a minimal "bootstrap" L1PT with cached access
disabled and uses the same table for all processors. AP startup is
performed in less steps and more code is written in C.

The bootstrap tables and stack are placed into an (orphaned) section
"_init_memory" which is given to uvm when it is no longer used.

Various kernels have been converted to use this code and tested. Some
boards were provided by TNF. Thanks!

The GENERIC kernel now boots on boards using the TEGRA, SUNXI and EXYNOS
kernels. The GENERIC kernel will also work on RPI2 using u-boot.

Thanks to martin@ and aymeric@ for testing on parallella and nanosoc
respectively

add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.

Whitespace

cleanup locore, and changed the way to map memories during boot.
- add functions bootpage_enter() and bootpage_alloc() to adapt various layout
of physical memory map. especially for 64bit physical memory layout.
pmapboot_alloc() allocates pagetable pages from _end[].
- changed to map only the required amount for PA=VA identity mapping
(kernel image, UART device, and FDT blob) with L2_BLOCK(2Mbyte).
- changing page permission for kernel image, and making KSEG mapping are done
at cpu_kernel_vm_init() instead of at locore.
- optimize PTE entries with PTE Contiguous bit. it is enabled on devmap only for now.

reviewed by skrll@, thanks.

catch up to files.arm's recent "opt_console.h" changes

branches: 1.3.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)

branches: 1.2.2;
retire copyinout.S and fusu.S

branches: 1.1.4; 1.1.6;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.

add support COMPAT_LINUX32 for aarch64

Implement gic_splraise and the gic_splx fast path in asm (armv8).

Use sys/uvm/pmap/pmap_tlb.c on Aarch64 in the same way that some Arm, MIPS,
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.

Additionally the following changes have been made

- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.

Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.

before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w

after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w

>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>

add support COMPAT_LINUX for aarch64

Arm: Add support for SMC Calling Convention

Arm DEN0028 defines a calling mechanism used with Secure Monitor Call (SMC)
and Hypervisor Call (HVC) instructions. To discover SMCCC, we must:

1) Find the PSCI conduit (either via ACPI FADT, or Device Tree)
2) Use PSCI_VERSION to determine whether PSCI_FEATURES is supported
3) Call PSCI_FEATURES with SMCCC_VERSION to determine the implementation
version.

aarch64: Remove empty source file and references to it.

branches: 1.30.6;
make process_machdep.c included always since it provides register i/o used by
sys_process_getlwpstatus.c which is always included.

harmonize process_machdep.c inclusion.

Collapse all CPU_CORTEXA<n> options into CPU_CORTEX and do runtime
detection instead of ifdefs where required.

Part III of ad's performance improvements for aarch64

- Assembly language stubs for mutex_enter() and mutex_exit().

Implement ChaCha with NEON on ARM.

XXX Needs performance measurement.
XXX Needs adaptation to arm32 neon which has half the registers.

Add options PMAPBOOT_DEBUG to dump TTBR when pmapboot_enter().
Formerly DEBUG_MMU in locore.S, but there was a bit of confusion.

New permutation-based AES implementation using ARM NEON.

Also derived from Mike Hamburg's public-domain vpaes code.

Implement AES in kernel using ARMv8.0-AES on aarch64.

PMAP_DEBUG has been deleted on arm

Add KASAN-DMA support on aarch64, same as amd64. Discussed with skrll@.

branches: 1.20.4;
Various updates and improvements to cpu start up on arm/aarch64

- start sharing more code around the AP startup messaging.
- call arm_cpu_topology_set early so that ci_core_id is available for
drivers, e.g. bcm2835_intr.c
- both arm and aarch64 now have
- a static cpu_info_store array
- the same arm_cpu_{hatched,mbox}

add support pmap_pv(9)

Patch originally from jmcneill@. thanks

Small re-org. NFCI.

port the arm64 cpu topology setup for big.little to arm.

rename arm64 cpu_do_topology() to arm_cpu_do_topology() and
call it from both arm cpu_attach().

replace both aarch64_set_topology() inline code in arm
cpu_attach() with new arm_cpu_do_topology(), which is called
by the arm64 locore as well (possibly not needed, which would
allow it to become static.)

not yet tested on a real big.little armv7 system. tested
on rockpro64 and pinebook pro.

branches: 1.16.2;
Do not use Early Write Acknowledge for PCIe I/O and config space.

Enable early write acknowledge for device memory mappings.

Move all non-emulation-specific coredump code into the coredump module,
and remove all #ifdef COREDUMP conditional compilation. Now, the
coredump module is completely separated from the emulation modules, and
they can all be independently loaded and unloaded.

Welcome to 9.99.18 !

branches: 1.13.4;
Merge the [pgoyette-compat] branch

Add needs-flag to tprof_armv8.c

Split armv7/armv8 tprof backend config logic from the fdt bus glue.

Add CPU_THUNDERX which sets COHERENCY_UNIT and CACHE_LINE_SIZE to 128

Add support for EFI runtime services on aarch64.

Provide generic start code that assumes the MMU is off and caches are
disabled as per the linux booting protocol for ARMv6 and ARMv7 boards.
u-boot image type should be changed to 'linux' for correct behaviour.

The new start code builds a minimal "bootstrap" L1PT with cached access
disabled and uses the same table for all processors. AP startup is
performed in less steps and more code is written in C.

The bootstrap tables and stack are placed into an (orphaned) section
"_init_memory" which is given to uvm when it is no longer used.

Various kernels have been converted to use this code and tested. Some
boards were provided by TNF. Thanks!

The GENERIC kernel now boots on boards using the TEGRA, SUNXI and EXYNOS
kernels. The GENERIC kernel will also work on RPI2 using u-boot.

Thanks to martin@ and aymeric@ for testing on parallella and nanosoc
respectively

add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.

Whitespace

cleanup locore, and changed the way to map memories during boot.
- add functions bootpage_enter() and bootpage_alloc() to adapt various layout
of physical memory map. especially for 64bit physical memory layout.
pmapboot_alloc() allocates pagetable pages from _end[].
- changed to map only the required amount for PA=VA identity mapping
(kernel image, UART device, and FDT blob) with L2_BLOCK(2Mbyte).
- changing page permission for kernel image, and making KSEG mapping are done
at cpu_kernel_vm_init() instead of at locore.
- optimize PTE entries with PTE Contiguous bit. it is enabled on devmap only for now.

reviewed by skrll@, thanks.