Import the kernel parts of bhyve/arm64

To support virtual machines on arm64 add the vmm code. This is based on
earlier work by Mihai Carabas and Alexandru Elisei at University
Politehnica of Bucharest, with further work by myself and Mark Johnston.

All AArch64 CPUs should work, however only the GICv3 interrupt
controller is supported. There is initial support to allow the GICv2
to be supported in the future. Only pure Armv8.0 virtualisation is
supported, the Virtualization Host Extensions are not currently used.

With a separate userspace patch and U-Boot port FreeBSD guests are able
to boot to multiuser mode, and the hypervisor can be tested with the
kvm unit tests. Linux partially boots, but hangs before entering
userspace. Other operating systems are untested.

Sponsored by: Arm Ltd
Sponsored by: Innovate UK
Sponsored by: The FreeBSD Foundation
Sponsored by: University Politehnica of Bucharest
Differential Revision: https://reviews.freebsd.org/D37428

arm64: start defining a std.apple configuration

The M1 uses FDT, and has bge to start with. Add a SOC_* option for
the first SoC we'll be supporting.

IOMMU is added commented out because it does have it, but IOMMU is not
well-tested on aarch64. An initial version of the DART driver will be
upstreamed that just puts the DARTs that support bypass mode into bypass
mode -- we'll be missing some functionality, but we at least still end
up with some USB ports.

Reviewed by: karels, manu
Input from: jrtc27 (IOMMU)
Differential Revision: https://reviews.freebsd.org/D39823

sys: Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

arm64: emulate swp/swpb instructions

Add another undefined instruction handler for compat32 and watch out for
SWP/SWPB instructions.

SWP/SWPB were deprecated in ARMv6 and declared obsolete in ARMv7, but
this implementation is motivated by some proprietary software that still
uses SWP/SWPB. Because it's deprecated, emulation is pushed back behind
a sysctl that defaults to OFF in GENERIC so that it doesn't potentially
adversely affect package builds; it's unknown whether software may test
for a functional swp/swpb instruction with the desire of using it later,
so we err on the side of caution to ensure we don't end up with swp/swpb
use in freebsd/arm packages (which are built on aarch64).

The EMUL_SWP config option may be used to enable emulation by default in
environments where emulation is desired and won't really be turned off.

Reviewed by: andrew, mmel (both earlier version)
Sponsored by: Stormshield
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D39667

Add Rockchip RK3568 SoC

Attach its clock drivers and rk817 pmic driver to the build.

conf: Document why we have ARM64 and RISCV options

These are needed for the 'cpu ARM64' and 'cpu RISCV' options in these
architecture's config files. cpu lines are non-optional in config(8), so
we must define them here. There's no other use for them in the tree.

Sponsored by: Netflix

Per-thread stack canary on arm64

With the update to llvm 13 we are able to tell the compiler it can find
the SSP canary relative to the register that holds the userspace stack
pointer. As this is unused in most of the kernel it can be used here
to point to a per-thread SSP canary.

As the kernel could be built with an old toolchain, e.g. when upgrading
from 13, add a warning that the options was enabled but the compiler
doesn't support it to both the build and kernel boot.

Discussed with: emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33079

Use L2 blocks when in the identity map

This reduces the memory mapped to be closer to the minimal memory
needed to enable the MMU.

Reviewed by: mmel
Sponsored by: Innovate UK
Differential Revision:://reviews.freebsd.org/D27765

arm64: Add a SOC_BRCM_NS2 option

Only compile files needed for this platform if the option is enabled in the
kernel config file.
Add the option to GENERIC.

MFC after: 3 days

ARM64: Port FreeBSD to Nvidia Jetson TX1 and Nano.

Add support for the Tergra210 SoC and its companion PMIC MAX77620.

Add i.MX 8M Quad support

- Add CCM driver and clocks implementations for i.MX 8M
- Add GPC driver for iMX8
- Add clock tree for i.MX 8M Quad
- Add clocks support and new compat strings (where required) for existing i.MX 6 UART, I2C, and GPIO drivers
- Enable aarch64-compatible drivers form i.MX 6 in arm64 GENERIC kernel config
- Add dtb/imx8 kernel module with DTBs for Nitrogen8M and iMX8MQ EVK

With this patch both Nitrogen8M and iMX8MQ EVK boot with NFS root up to multiuser login prompt

Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D25274

Add LS1046A clockgen driver.

Driver provides probe and attach functions for LS1046A clockgen and passes
configuration information to QorIQ clockgen class. It may be used as
a reference implementation for different QorIQ clockgen devices.

Submitted by: Dawid Gorecki <dgr@semihalf.com>
Reviewed by: mmel, manu
Obtained from: Semihalf
Sponsored by: Alstom Group
Differential Revision: https://reviews.freebsd.org/D24352

Dynamically select LSE-based atomic(9)s on arm64.

Once all CPUs are online, determine if they all support LSE atomics and
set lse_supported to indicate this. For now the atomic(9)
implementations are still always inlined, though it would be preferable
to create out-of-line functions to avoid text bloat. This was not done
here since big.little systems exist in which some CPUs implement LSE
while others do not, and ifunc resolution must occur well before this
scenario can be detected. It does seem unlikely that FreeBSD will
ever run on such platforms, however, so converting atomic(9) to use
ifuncs is probably a good next step.

Add a LSE_ATOMICS arm64 kernel configuration option to unconditionally
select LSE-based atomic(9) implementations when the target system is
known.

Reviewed by: andrew, kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation, Amazon (hardware)
Differential Revision: https://reviews.freebsd.org/D23325

Add support for booting kernel directly from U-Boot using booti command.

In some cases, like is locked bootstrap or device's inability to boot from
removable media, we cannot use standard boot sequence and is necessary to
boot kernel directly from U-Boot.

Discussed with: jhibbits
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D13861

revert r354714 "Boot arm64 kernel using booti command from U-boot."

After discussing with mmel@, it was clear this is insufficient to address
all the needs. mmel@ will commit his original patch, from
https://reviews.freebsd.org/D13861, and the additions needed from r354714
will be made afterward.

Requested by: mmel
Sponsored by: Juniper Networks, Inc.

Boot arm64 kernel using booti command from U-boot.

Summary:
Boot arm64 kernel using booti command from U-boot. booti can relocate initrd
image into higher ram addresses, therefore align the initrd load address to 1GiB
and create VA = PA map for it. Create L2 pagetable entries to copy the initrd
image into KVA.
(parts of the code in https://reviews.freebsd.org/D13861 was referred and used
as appropriate)

Submitted by: Siddharth Tuli <siddharthtuli_gmail.com>
Reviewed by: manu
Sponsored by: Juniper Networks, Inc
Differential Revision: https://reviews.freebsd.org/D22255

arm64: add SOC_BRCM_BCM2838, build it in GENERIC

BCM2838/BCM2711 is the Raspberry Pi 4, which we will soon be able to boot
on once some ports bits are worked out.

arm64: allwinner: Add Allwinner H6 Support

This adds support for H6 SoC.
Add a CCU driver for H6 that support all PLLs and most of the clocks
that we are intersted in for now (i2c, mmc, usb, etc ...)

MFC after: 1 month

Add support for Intel Stratix 10 platform.

Intel Stratix 10 SoC includes a quad-core arm64 cluster and FPGA fabric.

This adds support for reconfiguring FPGA.

Accessing FPGA core of this SoC require the level of privilege EL3,
while kernel runs in EL1 (lower) level of privilege.

This provides an Intel service layer interface that uses SMCCC to pass
queries to the secure-monitor (EL3).

Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D21454

arm64: Add new SoC type MARVELL_8K

Sponsored by: Rubicon Communications, LLC ("Netgate")

arm64/rockchip: add RK3399 support

Add CRU (Clock and Reset Unit) driver for RK3399.
Add support in rk_pinctrl driver.

Submitted by: Greg V <greg@unrelenting.technology> (Original version)
Differential Revision: https://reviews.freebsd.org/D16732

MFC after: 1 month

Enable UART support for Xilinx Ultrascale+ SoCs

Xilinx Ultrascale+ are based on Cortex-A53 and use existing
UART driver (uart_dev_cdnc). Enable it in arm64 GENERIC config.

Submitted by: Michal Stanek <mst@semihalf.com>
Obtained from: Semihalf

Remove the psci option from arm64. It is now a standard option as it is
required to boot correctly.

Sponsored by: DARPA, AFRL

Move most of the contents of opt_compat.h to opt_global.h.

opt_compat.h is mentioned in nearly 180 files. In-progress network
driver compabibility improvements may add over 100 more so this is
closer to "just about everywhere" than "only some files" per the
guidance in sys/conf/options.

Keep COMPAT_LINUX32 in opt_compat.h as it is confined to a subset of
sys/compat/linux/*.c. A fake _COMPAT_LINUX option ensure opt_compat.h
is created on all architectures.

Move COMPAT_LINUXKPI to opt_dontuse.h as it is only used to control the
set of compiled files.

Reviewed by: kib, cem, jhb, jtl
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14941

rk3328: Add support for this SoC

* rk_cru is a cru driver that needs to be subclassed by
the real CRU driver
* rk_clk_pll handle the pll type clock on RockChip SoC, it's only read
only for now.
* rk_clk_composite handle the different composite clock types (with gate,
with mux etc ...)
* rk_clk_gate handle the RockChip gates
* rk_clk_mux handle the RockChip muxes (unused for now)
* Only clocks for supported devices are supported for now, the rest will be
added when driver support comes
* The assigned-clock* property are not handled for now so we rely a lot on the
bootloader to setup some initial values for some clocks.

Add rudimentary support for building FreeBSD/arm64 with COMPAT_FREEBSD32.

Right now I'm using two Raspberry Pi's (2 and 3) to test CloudABI
support for armv6, armv7 and aarch64. It would be nice if I could
restrict this to just a single instance when testing smaller changes.
This is why I'd like to get COMPAT_CLOUDABI32 to work on arm64.

As COMPAT_CLOUDABI32 depends on COMPAT_FREEBSD32, at least for the ELF
loading, this change adds all of the bits necessary to at least build a
kernel with COMPAT_FREEBSD32. All of the machine dependent system calls
are still stubbed out, for the reason that implementations for these are
only useful if actual support for running FreeBSD binaries is added.
This is outside the scope of this work.

Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D13144

Support the EFI Runtime Services on arm64. As with amd64 we use the 1:1
mapping. This uses the new common code shared with amd64.

The RTC should only be accessed via EFI. There is no locking around it as
the spec only has this as a requirement for the PC-AT CMOS device.

Reviewed by: kib, imp
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12595

arm64: Add Allwinner H5 SoC

Allwinner H5 is an H3 (arm32) with Cortex A53 cores.
Add support for it and enable it in GENERIC kernel config

Tested on: OrangePi PC2

Call the PSCI reset from cpu_reset on arm64. When rebooting from DDB the
kernel calls this directly so the event handler is not called, meaning
the computer fails to reboot.

Tested by: cognet
MFC after: 1 week
Sponsored by: DARPA, AFRL

Make BRCM2837 port conform FreeBSD/ARM64 guidelines

- Rename SOC_BCM2837 to SOC_BRCM_BCM2837, put it to opt_soc.h
- do not use files.XXX files, just move required sources to
conf/files.arm64 and make them depend on soc_brcm_bcm2837

Suggested by: andrew

Add initial Raspberry Pi 3 support

RPI3 kernel config builds kernel compatible with latest upstream device
tree and firmware: https://github.com/raspberrypi/firmware/tree/master/boot
As of today it's 597c662a613df1144a6bc43e5f4505d83bd748ca

Default console is PL01x, so pi3-disable-bt dt overlay should be configured
in config.txt and stock U-Boot should be patched to use proper serial port.

Yet unsupported: SMP, VCHIQ, RNG driver. RNG requires some work due to
upstream device tree incompatibility.

Multiple people contributed to this work over time: db@, loos@, manu@

Add SOC_ALLWINNER_A64 option for Allwinner A64 (sun50i) SoCs.

Add support for intrng to arm64. As the GICv3 drivers will need to be
updated, and until further testing can be done, this is disabled for now.

It is expected arm64 will switch to this interface, and the old interface
will be removed before 11.0 is released.

Obtained from: ABT Systems Ltd
Relnotes: yes
Sponsored by: The FreeBSD Foundation

Add the SOC_HISI_HI6220 option. This will be used to enable drivers for
the HiSilicon hi6220 SoC used in the HiKey 96boards board.

Sponsored by: ABT Systems Ltd

Add an option to select which SoCs we are building for. It is intended to
be used with any SoC specific drivers, for example a ThunderX nic driver
would use something like the following in files.arm64:

arm64/cavium/thunder_nic.c optional soc_cavm_thunderx thndr_nic

Reviewed by: imp
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D3479

Apply erratum for mrs ICC_IAR1_EL1 speculative execution on ThunderX

ERRATUM: 22978, 23154
PASS (rev.): 1.0/1.1

Reviewed by: imp
Obtained from: Semihalf
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3184

Allow the arm64 config file to define SOCDEV_PA and SOCDEV_VA that may be
used in locore.S to map an extra range when enabling the MMU.

Bring in the start of the arm64 kernel.

This is only the minimum set of files needed to boot in qemu. As such it is
missing a few things.

The bus_dma code is currently only stub functions with a full implementation
from the development tree to follow.

The gic driver has been copied as the interrupt framework is different. It
is expected the two drivers will be merged by the arm intrng project,
however this will need to be imported into the tree and support for arm64
would need to be added.

This includes code developed by myself, SemiHalf, Ed Maste, and Robin
Randhawa from ARM. This has been funded by the FreeBSD Foundation, with
early development by myself in my spare time with assistance from Robin.

Differential Revision: https://reviews.freebsd.org/D2199
Reviewed by: emaste, imp
Relnotes: yes
Sponsored by: The FreeBSD Foundation