The traditional LL/SC atomics perform poorly on modern arm64 systems with
many CPU cores. With the recent conversion of the sched lock to a mutex
some systems appear to hang if the sched lock is contended. ARMv8.1
introduced an LSE feature that provides atomic instructions such as CAS
that perform much better. Unfortunately these can't be used on older
ARMv8.0 systems. Use -moutline-atomics to make the compiler generate
function calls for atomic operations and provide an implementation for
the functions we use in the kernel that use LSE when available and fall
back on LL/SC.

Fixes regressions seen on Ampere Altra and Apple M2 Pro/Max/Ultra since
the conversion of the sched lock to a mutex.

tested by claudio@, phessler@, mpi@
ok patrick@

Add rpigpio(4), a driver for the RP1 GPIO controller on the Raspberry Pi 5.

With this, GPIOs can be correctly configured and engaged. Complete pinctrl
as well as IRQ functionality is yet to be implemented.

ok kettenis@

Move fdt attachment into sys/conf/files.conf instead of duplicating it on
an MD basis.

ok patrick@

Add ampchwm(4) a driver to access the Ampere Altra Fine-Grained Power Telemetry

This adds per core energy sensors (in Joules) and one per SoC temparature
sensor.
OK kettenis@ deraadt@

Implement access to EFI variables and ESRT through an ioctl(2) interface
that is compatible with what FreeBSD and NetBSD have. Setting EFI
variables is only allowed at securelevel 0 and below.

Heavily based on work done by Sergii Dmytruk.

ok yasuoka@

Add aplefuse(4), a driver for the eFuses on Apple Silicon SoCs.

ok patrick@

Add aplpwm(4), a driver for the PWM controller found on Apple silicon.

ok tobhe@

Tedu spmi bus definition that moved over to the fdt files.

Make EFI runtime calls more robust on arm64 as well. While I have not seen
any broken implementations yet, Linux developers claim that arm64 machines
intended to run Windows are not much better than x86 machines. And I
totally believe that.

ok patrick@

Reorganize the EFI code a bit. Move the efi.h header from dev/acpi to
dev/efi and rename the arm64 efi.c to efi_machdep.c, preparing the way
for MI EFI code and an amd64 implementation of EFI runtime support.

ok deraadt@, mlarkin@

Add apldcms(4), a driver for the touchpad on M2 laptops. This driver
needs firmware that is provided on the ESP by the Asahi installer and
copied into /etc/firmware/apple by the OpenBSD installer.

ok tobhe@

Add apldc(4), apldchidev(4), apldckdb(4) and aplrtkit(4). Together these
drivers implement support for the Dockchannel-base keyboard found on
Apple M2 laptops.

ok mlarkin@, patrick@

Add aplaudio(4) and aplmca(4). The aplmca(4) driver controls the hardware
block that takes data from apldma(4), serializes it and sends it out on
the i2s ports. The aplaudio(4) driver ties together aplmca(4) and various
codecs to present an audio(4) interface to the system.

This is still WIP, but good enough to play back audio on the speaker in the
M1 mini.

ok patrick@

The NVMe firmware distributed with the macOS 13 developer beta does not
like some of the shortcuts we've taken in the code that talks to RTKit.
In particular it does not like our NAK of the request for ioreport buffers.
So allocate the requested buffers and ACK the request instead.

However, the hardware implements a address filter between the coprocessor
that the firmware runs on and main memory. So we have to add a mapping
into this address filter such that the firmware can access these buffers.
Support for this address filter is implemented in a new aplsart(4) driver.

ok dlg@

Add aplcpu(4), a driver to control the CPU performance levels on Apple SoCs.

ok gnezdo@

Add apldma(4), a driver for the DMA controller found on Apple SoCs.
This driver implements an interface for using DMA for audio output.

ok ratchov@

Add aplnco(4), a driver for the Numerically-controlled oscillator (NCO)
clock that drives the audio clocks on Apple silicon.

ok patrick@

Add aplsmc(4), a driver for the SMC found on Apple M1 SoCs.

The SMC implements a lot of functionality. For now the driver only
implements a bunch of sensors. This is a small subset of the sensors
that are made available by the SMC as we don't know what measurments
are provided for most sensors.

ok patrick@

Remove apldwusb(4). This driver is now unused.

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Add apliic(4), a driver for the I2C controller found on various Apple SoCs.

ok kettenis@

Add an implementation for the protocol to communicate with coprocessors
running firmware based Apple's RTKit OS. Use this code to bring up the
storage controller that implements Apple's flavour of NVMe.

ok dlg@, patrick@

Enable aplmbox(4).

Add aplpmgr(4), a driver for the power management controller found on
various Apple SoCs.

ok patrick@

Add support for the keyboard/touchpad on Apple M1 laptops.

ok patrick@

Add aplspi(4), a driver for the SPI controller found on the Apple M1 SoC.

ok patrick@

Add aplpinctrl(4), a driver for the Apple GPIO controller found on M1 SoCs.

ok patrick@

start working on support for Apple NVME Storage as found in apple M1 devs

the Apple NVME Storage (aka ans) controller is almost but not quite
a vanilla nvme controller. one difference is that it doesnt attach
to a pci bus, so it needs this custom bus glue. this custom bus
glue also provides us with a nice way to provide a different set
of functions to handle other things that ans does differently to
vanilla nvme controllers.

this is different to how linux deals with ans. the linux support
fakes a pci controller for ans to attach to. i assumed that at some
point a vendor would include nvme in an soc directly and made it a
bus independent driver from day 1. turns out i was right, but i
would never have guessed that the vendor would be apple.

some of the other differences between vanilla nvme and ans are
around command submission and completion. ans nvme command submission
is done via an array of command slots where the host picks a slot
and then posts every slot number it fills in to a doorbell. this
is different to vanilla nvme controllers which use a ring, and post
the producer index in that ring to a doorbell. ans also includes
some weird iommu, the handling of which we wrap up into the command
submission and completion functions.

this code is not yet enabled because it is incomplete. i'm getting
what i've done in so people with actual hardware can start poking
it more seriously.

this implementation is based on information figured out by the guys
porting linux to apple hardware.

Add aplspmi(4), a driver for the Apple SPMI controller, and aplpmu(4)
a driver for the Apple "sera" SPMI power management unit that contains
the RTC on Apple M1 systems.

ok patrick@

Add apldwusb(4), a glue driver for the Synpsys DesignWare USB 3 controllers
found on the Apple M1 SoC.

Move the pciecam stanza back into files.arm64 since armv7 has its own
pciecam(4) implementation hidden away in arch/armv7/vexpress.

Unbreaks armv7 kernel builds.

Move pciecam.c to dev/fdt/.

ok deraadt@

acpiiort(4) and smmu(4)

Add apldart(4), a driver for the IOMMU on Apple M1 SoCs.

ok patrick@

Add aplcpie(4), a (minimal) driver for the PCIe host bridge on Apple M1 SoCs.

ok patrick@

Add aplintc(4), a driver for the interrupt controller found on
Apple M1 SoCs.

ok patrick@

Enable apldog(4).

Add cryptox(4), a driver for armv8 cryptographic extensions.
The driver currently only supports AES-CBC mode but can easily
be extended to other algorithms and modes.
The aesv8-armx.S file was generated from the CRYPTOGAMS project.

Asked to commit by and ok patrick@

Port over NetBSD's arm64 disassembler for DDB.

ok kettenis@, patrick@

The dwctwo(4) FDT glue is now identical on armv7 and arm64. Move it to
/dev/fdt.

ok patrick@, visa@

Implement smbios support on arm64.

ok deraadt@, jsg@

permit maxusers to grow up to 128

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Add the apm(4) subsystem to arm64.

NB: acpi(4) is not hooked up to this yet.

previous version OK deraadt@ patrick@
OK jsg@

Add support for the GIC v3 ITS and use it to implement MSI support for
rkpcie(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

Hook up acpi(4) on arm64. Various bits of driver glue are still missing
but this is enough to boot multi-user on the mcbin with suitable firmware.

ok mlarkin@

Move pluart(4) to dev/fdt.

ok jsg@

Move com_fdt.c to dev/fdt since we can share it between armv7 and arm64 now.

ok patrick@

Add a common rules file for ofw sources to help keep the configurations
of fdt-enabled platforms in sync.

OK deraadt@

Implement a power domain framework to turn on/off so-called power
domains. This mechanism is used by the newer i.MX8M SoCs so that
drivers can call into ATF to supply power to e.g. a USB port.

ok kettenis@

use the arm64 openprom.c on arm64
ok patrick@

Move common mutex implementations to a MI place.

Archs not yet converted can to the jump by defining __USE_MI_MUTEX.

ok visa@

Implement FUCKWIT for arm64; unmap the kernel almost entirely while userland
is running. This provides protection against meltown on cores that are
vilnerable (just Cortex-A75 so far) but also seems to be an essential to
protect against spectre-like attacks against the kernel.

This implementation only exposes a single treampoline page that does not
contain any kernel virtual addresses and also hides the real virtual address
of the exception vectors, which helps on cores vulnerable to "variant 3a"
(Cortex-A57, Cortex-A72). The implementation is inspired by the work done
by Will Deacon for Linux, but there are no knobs to turn it off. The
overhead is fairly limited: around 3-4% slowdown on Cortex-A57.

ok patrick@, deraadt@

Implement support for calling EFI runtime services and use it to implement
a time-of-day clock device based on the GetTime() and SetTime() services.
The virtual memory mappings for the runtime services calls are implemented
through a separate pmap that is only activated when we make a runtime services
call.

ok tom@, visa@
tested by naddy@

Split early startup code out of locore.S into locore0.S. Adjust link
run so that this locore0.o is always at the start of the executable.
But randomize the link order of all other .o files in the kernel, so
that their exec/rodata/data/bss segments land all over the place.
Late during kernel boot, unmap the early startup code.

As a result, the internal layout of every newly build bsd kernel is
different from past kernels. Internal relative offsets are not known
to an outside attacker. The only known offsets are in the startup code,
which has been unmapped.

Ramdisk kernels cannot be compiled like this, because they are gzip'd.
When the internal pointer references change, the compression dictionary
bloats and results in poorer compression.

With guidance and ok deraadt@

Add rkgrf(4), a driver that makes the "generic register file" of the Rockchip
RK3399 available to other drivers through the regmap interface.

Add agintc(4), a driver for interrupt controllers conforming to ARM's
generic interrupt controller architecture specification v3/4.

The hard work was done by drahn@, I just cleaned it up a bit and fixed
a couple of bugs.

ok patrick@, drahn@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Implement ampintcmsi(4) in ampintc(4) to support MSI. The GICv2M is an
extension to the GIC controller, which is represented as subnode in the
device tree. There can be multiple GICv2Ms, so it makes sense to attach
those to ampintc(4) as some kind of simplebus. The GICv2M is simply an
interrupt generator that can be used by PCIe devices to ring the door
bell. There is no need for further configuration, we only need to find
out which SPIs we are allowed to use for MSI and to register an edge
triggered interrupt on a (randomly) allocated SPI.

ok kettenis@

Hook up pciecam(4), but don't compile it yet.

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

The only difference between armv7 and arm64 fdt virtio attachments is an
uneeded include. Remove the include and move to MI fdt directory.

Make the RAMDISK kernel actually try to boot the ramdisk.

openprom(4)

Add drivers for the Raspberry Pi 3.

Move virtio config from files.pci to files.pci

virtio itself is not PCI, it is a paravirtual bus on top of either PCI
or MMIO (arm). This is the first step of moving the virtio files to
dev/pv. No functional change.

arm kernel changes tested by jsg@ and patrick@
no objections from sf@ mlarkin@

Add Allwinner A64 devices to OpenBSD/arm64. This means we can run
on the Pine64. While there add a few more usb and pseudo devices.

Consolidate bus space tags into generic bus space code. Especially
considering we don't have a separate iobus and the a4x iobus wasn't
even compiled it does not make sense to declare those in iobus.
Improves code readability as well.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Add rpigpio(4), a driver for the RP1 GPIO controller on the Raspberry Pi 5.

With this, GPIOs can be correctly configured and engaged. Complete pinctrl
as well as IRQ functionality is yet to be implemented.

ok kettenis@

Move fdt attachment into sys/conf/files.conf instead of duplicating it on
an MD basis.

ok patrick@

Add ampchwm(4) a driver to access the Ampere Altra Fine-Grained Power Telemetry

This adds per core energy sensors (in Joules) and one per SoC temparature
sensor.
OK kettenis@ deraadt@

Implement access to EFI variables and ESRT through an ioctl(2) interface
that is compatible with what FreeBSD and NetBSD have. Setting EFI
variables is only allowed at securelevel 0 and below.

Heavily based on work done by Sergii Dmytruk.

ok yasuoka@

Add aplefuse(4), a driver for the eFuses on Apple Silicon SoCs.

ok patrick@

Add aplpwm(4), a driver for the PWM controller found on Apple silicon.

ok tobhe@

Tedu spmi bus definition that moved over to the fdt files.

Make EFI runtime calls more robust on arm64 as well. While I have not seen
any broken implementations yet, Linux developers claim that arm64 machines
intended to run Windows are not much better than x86 machines. And I
totally believe that.

ok patrick@

Reorganize the EFI code a bit. Move the efi.h header from dev/acpi to
dev/efi and rename the arm64 efi.c to efi_machdep.c, preparing the way
for MI EFI code and an amd64 implementation of EFI runtime support.

ok deraadt@, mlarkin@

Add apldcms(4), a driver for the touchpad on M2 laptops. This driver
needs firmware that is provided on the ESP by the Asahi installer and
copied into /etc/firmware/apple by the OpenBSD installer.

ok tobhe@

Add apldc(4), apldchidev(4), apldckdb(4) and aplrtkit(4). Together these
drivers implement support for the Dockchannel-base keyboard found on
Apple M2 laptops.

ok mlarkin@, patrick@

Add aplaudio(4) and aplmca(4). The aplmca(4) driver controls the hardware
block that takes data from apldma(4), serializes it and sends it out on
the i2s ports. The aplaudio(4) driver ties together aplmca(4) and various
codecs to present an audio(4) interface to the system.

This is still WIP, but good enough to play back audio on the speaker in the
M1 mini.

ok patrick@

The NVMe firmware distributed with the macOS 13 developer beta does not
like some of the shortcuts we've taken in the code that talks to RTKit.
In particular it does not like our NAK of the request for ioreport buffers.
So allocate the requested buffers and ACK the request instead.

However, the hardware implements a address filter between the coprocessor
that the firmware runs on and main memory. So we have to add a mapping
into this address filter such that the firmware can access these buffers.
Support for this address filter is implemented in a new aplsart(4) driver.

ok dlg@

Add aplcpu(4), a driver to control the CPU performance levels on Apple SoCs.

ok gnezdo@

Add apldma(4), a driver for the DMA controller found on Apple SoCs.
This driver implements an interface for using DMA for audio output.

ok ratchov@

Add aplnco(4), a driver for the Numerically-controlled oscillator (NCO)
clock that drives the audio clocks on Apple silicon.

ok patrick@

Add aplsmc(4), a driver for the SMC found on Apple M1 SoCs.

The SMC implements a lot of functionality. For now the driver only
implements a bunch of sensors. This is a small subset of the sensors
that are made available by the SMC as we don't know what measurments
are provided for most sensors.

ok patrick@

Remove apldwusb(4). This driver is now unused.

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Add apliic(4), a driver for the I2C controller found on various Apple SoCs.

ok kettenis@

Add an implementation for the protocol to communicate with coprocessors
running firmware based Apple's RTKit OS. Use this code to bring up the
storage controller that implements Apple's flavour of NVMe.

ok dlg@, patrick@

Enable aplmbox(4).

Add aplpmgr(4), a driver for the power management controller found on
various Apple SoCs.

ok patrick@

Add support for the keyboard/touchpad on Apple M1 laptops.

ok patrick@

Add aplspi(4), a driver for the SPI controller found on the Apple M1 SoC.

ok patrick@

Add aplpinctrl(4), a driver for the Apple GPIO controller found on M1 SoCs.

ok patrick@

start working on support for Apple NVME Storage as found in apple M1 devs

the Apple NVME Storage (aka ans) controller is almost but not quite
a vanilla nvme controller. one difference is that it doesnt attach
to a pci bus, so it needs this custom bus glue. this custom bus
glue also provides us with a nice way to provide a different set
of functions to handle other things that ans does differently to
vanilla nvme controllers.

this is different to how linux deals with ans. the linux support
fakes a pci controller for ans to attach to. i assumed that at some
point a vendor would include nvme in an soc directly and made it a
bus independent driver from day 1. turns out i was right, but i
would never have guessed that the vendor would be apple.

some of the other differences between vanilla nvme and ans are
around command submission and completion. ans nvme command submission
is done via an array of command slots where the host picks a slot
and then posts every slot number it fills in to a doorbell. this
is different to vanilla nvme controllers which use a ring, and post
the producer index in that ring to a doorbell. ans also includes
some weird iommu, the handling of which we wrap up into the command
submission and completion functions.

this code is not yet enabled because it is incomplete. i'm getting
what i've done in so people with actual hardware can start poking
it more seriously.

this implementation is based on information figured out by the guys
porting linux to apple hardware.

Add aplspmi(4), a driver for the Apple SPMI controller, and aplpmu(4)
a driver for the Apple "sera" SPMI power management unit that contains
the RTC on Apple M1 systems.

ok patrick@

Add apldwusb(4), a glue driver for the Synpsys DesignWare USB 3 controllers
found on the Apple M1 SoC.

Move the pciecam stanza back into files.arm64 since armv7 has its own
pciecam(4) implementation hidden away in arch/armv7/vexpress.

Unbreaks armv7 kernel builds.

Move pciecam.c to dev/fdt/.

ok deraadt@

acpiiort(4) and smmu(4)

Add apldart(4), a driver for the IOMMU on Apple M1 SoCs.

ok patrick@

Add aplcpie(4), a (minimal) driver for the PCIe host bridge on Apple M1 SoCs.

ok patrick@

Add aplintc(4), a driver for the interrupt controller found on
Apple M1 SoCs.

ok patrick@

Enable apldog(4).

Add cryptox(4), a driver for armv8 cryptographic extensions.
The driver currently only supports AES-CBC mode but can easily
be extended to other algorithms and modes.
The aesv8-armx.S file was generated from the CRYPTOGAMS project.

Asked to commit by and ok patrick@

Port over NetBSD's arm64 disassembler for DDB.

ok kettenis@, patrick@

The dwctwo(4) FDT glue is now identical on armv7 and arm64. Move it to
/dev/fdt.

ok patrick@, visa@

Implement smbios support on arm64.

ok deraadt@, jsg@

permit maxusers to grow up to 128

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Add the apm(4) subsystem to arm64.

NB: acpi(4) is not hooked up to this yet.

previous version OK deraadt@ patrick@
OK jsg@

Add support for the GIC v3 ITS and use it to implement MSI support for
rkpcie(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

Hook up acpi(4) on arm64. Various bits of driver glue are still missing
but this is enough to boot multi-user on the mcbin with suitable firmware.

ok mlarkin@

Move pluart(4) to dev/fdt.

ok jsg@

Move com_fdt.c to dev/fdt since we can share it between armv7 and arm64 now.

ok patrick@

Add a common rules file for ofw sources to help keep the configurations
of fdt-enabled platforms in sync.

OK deraadt@

Implement a power domain framework to turn on/off so-called power
domains. This mechanism is used by the newer i.MX8M SoCs so that
drivers can call into ATF to supply power to e.g. a USB port.

ok kettenis@

use the arm64 openprom.c on arm64
ok patrick@

Move common mutex implementations to a MI place.

Archs not yet converted can to the jump by defining __USE_MI_MUTEX.

ok visa@

Implement FUCKWIT for arm64; unmap the kernel almost entirely while userland
is running. This provides protection against meltown on cores that are
vilnerable (just Cortex-A75 so far) but also seems to be an essential to
protect against spectre-like attacks against the kernel.

This implementation only exposes a single treampoline page that does not
contain any kernel virtual addresses and also hides the real virtual address
of the exception vectors, which helps on cores vulnerable to "variant 3a"
(Cortex-A57, Cortex-A72). The implementation is inspired by the work done
by Will Deacon for Linux, but there are no knobs to turn it off. The
overhead is fairly limited: around 3-4% slowdown on Cortex-A57.

ok patrick@, deraadt@

Implement support for calling EFI runtime services and use it to implement
a time-of-day clock device based on the GetTime() and SetTime() services.
The virtual memory mappings for the runtime services calls are implemented
through a separate pmap that is only activated when we make a runtime services
call.

ok tom@, visa@
tested by naddy@

Split early startup code out of locore.S into locore0.S. Adjust link
run so that this locore0.o is always at the start of the executable.
But randomize the link order of all other .o files in the kernel, so
that their exec/rodata/data/bss segments land all over the place.
Late during kernel boot, unmap the early startup code.

As a result, the internal layout of every newly build bsd kernel is
different from past kernels. Internal relative offsets are not known
to an outside attacker. The only known offsets are in the startup code,
which has been unmapped.

Ramdisk kernels cannot be compiled like this, because they are gzip'd.
When the internal pointer references change, the compression dictionary
bloats and results in poorer compression.

With guidance and ok deraadt@

Add rkgrf(4), a driver that makes the "generic register file" of the Rockchip
RK3399 available to other drivers through the regmap interface.

Add agintc(4), a driver for interrupt controllers conforming to ARM's
generic interrupt controller architecture specification v3/4.

The hard work was done by drahn@, I just cleaned it up a bit and fixed
a couple of bugs.

ok patrick@, drahn@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Implement ampintcmsi(4) in ampintc(4) to support MSI. The GICv2M is an
extension to the GIC controller, which is represented as subnode in the
device tree. There can be multiple GICv2Ms, so it makes sense to attach
those to ampintc(4) as some kind of simplebus. The GICv2M is simply an
interrupt generator that can be used by PCIe devices to ring the door
bell. There is no need for further configuration, we only need to find
out which SPIs we are allowed to use for MSI and to register an edge
triggered interrupt on a (randomly) allocated SPI.

ok kettenis@

Hook up pciecam(4), but don't compile it yet.

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

The only difference between armv7 and arm64 fdt virtio attachments is an
uneeded include. Remove the include and move to MI fdt directory.

Make the RAMDISK kernel actually try to boot the ramdisk.

openprom(4)

Add drivers for the Raspberry Pi 3.

Move virtio config from files.pci to files.pci

virtio itself is not PCI, it is a paravirtual bus on top of either PCI
or MMIO (arm). This is the first step of moving the virtio files to
dev/pv. No functional change.

arm kernel changes tested by jsg@ and patrick@
no objections from sf@ mlarkin@

Add Allwinner A64 devices to OpenBSD/arm64. This means we can run
on the Pine64. While there add a few more usb and pseudo devices.

Consolidate bus space tags into generic bus space code. Especially
considering we don't have a separate iobus and the a4x iobus wasn't
even compiled it does not make sense to declare those in iobus.
Improves code readability as well.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Move fdt attachment into sys/conf/files.conf instead of duplicating it on
an MD basis.

ok patrick@

Add ampchwm(4) a driver to access the Ampere Altra Fine-Grained Power Telemetry

This adds per core energy sensors (in Joules) and one per SoC temparature
sensor.
OK kettenis@ deraadt@

Implement access to EFI variables and ESRT through an ioctl(2) interface
that is compatible with what FreeBSD and NetBSD have. Setting EFI
variables is only allowed at securelevel 0 and below.

Heavily based on work done by Sergii Dmytruk.

ok yasuoka@

Add aplefuse(4), a driver for the eFuses on Apple Silicon SoCs.

ok patrick@

Add aplpwm(4), a driver for the PWM controller found on Apple silicon.

ok tobhe@

Tedu spmi bus definition that moved over to the fdt files.

Make EFI runtime calls more robust on arm64 as well. While I have not seen
any broken implementations yet, Linux developers claim that arm64 machines
intended to run Windows are not much better than x86 machines. And I
totally believe that.

ok patrick@

Reorganize the EFI code a bit. Move the efi.h header from dev/acpi to
dev/efi and rename the arm64 efi.c to efi_machdep.c, preparing the way
for MI EFI code and an amd64 implementation of EFI runtime support.

ok deraadt@, mlarkin@

Add apldcms(4), a driver for the touchpad on M2 laptops. This driver
needs firmware that is provided on the ESP by the Asahi installer and
copied into /etc/firmware/apple by the OpenBSD installer.

ok tobhe@

Add apldc(4), apldchidev(4), apldckdb(4) and aplrtkit(4). Together these
drivers implement support for the Dockchannel-base keyboard found on
Apple M2 laptops.

ok mlarkin@, patrick@

Add aplaudio(4) and aplmca(4). The aplmca(4) driver controls the hardware
block that takes data from apldma(4), serializes it and sends it out on
the i2s ports. The aplaudio(4) driver ties together aplmca(4) and various
codecs to present an audio(4) interface to the system.

This is still WIP, but good enough to play back audio on the speaker in the
M1 mini.

ok patrick@

The NVMe firmware distributed with the macOS 13 developer beta does not
like some of the shortcuts we've taken in the code that talks to RTKit.
In particular it does not like our NAK of the request for ioreport buffers.
So allocate the requested buffers and ACK the request instead.

However, the hardware implements a address filter between the coprocessor
that the firmware runs on and main memory. So we have to add a mapping
into this address filter such that the firmware can access these buffers.
Support for this address filter is implemented in a new aplsart(4) driver.

ok dlg@

Add aplcpu(4), a driver to control the CPU performance levels on Apple SoCs.

ok gnezdo@

Add apldma(4), a driver for the DMA controller found on Apple SoCs.
This driver implements an interface for using DMA for audio output.

ok ratchov@

Add aplnco(4), a driver for the Numerically-controlled oscillator (NCO)
clock that drives the audio clocks on Apple silicon.

ok patrick@

Add aplsmc(4), a driver for the SMC found on Apple M1 SoCs.

The SMC implements a lot of functionality. For now the driver only
implements a bunch of sensors. This is a small subset of the sensors
that are made available by the SMC as we don't know what measurments
are provided for most sensors.

ok patrick@

Remove apldwusb(4). This driver is now unused.

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Add apliic(4), a driver for the I2C controller found on various Apple SoCs.

ok kettenis@

Add an implementation for the protocol to communicate with coprocessors
running firmware based Apple's RTKit OS. Use this code to bring up the
storage controller that implements Apple's flavour of NVMe.

ok dlg@, patrick@

Enable aplmbox(4).

Add aplpmgr(4), a driver for the power management controller found on
various Apple SoCs.

ok patrick@

Add support for the keyboard/touchpad on Apple M1 laptops.

ok patrick@

Add aplspi(4), a driver for the SPI controller found on the Apple M1 SoC.

ok patrick@

Add aplpinctrl(4), a driver for the Apple GPIO controller found on M1 SoCs.

ok patrick@

start working on support for Apple NVME Storage as found in apple M1 devs

the Apple NVME Storage (aka ans) controller is almost but not quite
a vanilla nvme controller. one difference is that it doesnt attach
to a pci bus, so it needs this custom bus glue. this custom bus
glue also provides us with a nice way to provide a different set
of functions to handle other things that ans does differently to
vanilla nvme controllers.

this is different to how linux deals with ans. the linux support
fakes a pci controller for ans to attach to. i assumed that at some
point a vendor would include nvme in an soc directly and made it a
bus independent driver from day 1. turns out i was right, but i
would never have guessed that the vendor would be apple.

some of the other differences between vanilla nvme and ans are
around command submission and completion. ans nvme command submission
is done via an array of command slots where the host picks a slot
and then posts every slot number it fills in to a doorbell. this
is different to vanilla nvme controllers which use a ring, and post
the producer index in that ring to a doorbell. ans also includes
some weird iommu, the handling of which we wrap up into the command
submission and completion functions.

this code is not yet enabled because it is incomplete. i'm getting
what i've done in so people with actual hardware can start poking
it more seriously.

this implementation is based on information figured out by the guys
porting linux to apple hardware.

Add aplspmi(4), a driver for the Apple SPMI controller, and aplpmu(4)
a driver for the Apple "sera" SPMI power management unit that contains
the RTC on Apple M1 systems.

ok patrick@

Add apldwusb(4), a glue driver for the Synpsys DesignWare USB 3 controllers
found on the Apple M1 SoC.

Move the pciecam stanza back into files.arm64 since armv7 has its own
pciecam(4) implementation hidden away in arch/armv7/vexpress.

Unbreaks armv7 kernel builds.

Move pciecam.c to dev/fdt/.

ok deraadt@

acpiiort(4) and smmu(4)

Add apldart(4), a driver for the IOMMU on Apple M1 SoCs.

ok patrick@

Add aplcpie(4), a (minimal) driver for the PCIe host bridge on Apple M1 SoCs.

ok patrick@

Add aplintc(4), a driver for the interrupt controller found on
Apple M1 SoCs.

ok patrick@

Enable apldog(4).

Add cryptox(4), a driver for armv8 cryptographic extensions.
The driver currently only supports AES-CBC mode but can easily
be extended to other algorithms and modes.
The aesv8-armx.S file was generated from the CRYPTOGAMS project.

Asked to commit by and ok patrick@

Port over NetBSD's arm64 disassembler for DDB.

ok kettenis@, patrick@

The dwctwo(4) FDT glue is now identical on armv7 and arm64. Move it to
/dev/fdt.

ok patrick@, visa@

Implement smbios support on arm64.

ok deraadt@, jsg@

permit maxusers to grow up to 128

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Add the apm(4) subsystem to arm64.

NB: acpi(4) is not hooked up to this yet.

previous version OK deraadt@ patrick@
OK jsg@

Add support for the GIC v3 ITS and use it to implement MSI support for
rkpcie(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

Hook up acpi(4) on arm64. Various bits of driver glue are still missing
but this is enough to boot multi-user on the mcbin with suitable firmware.

ok mlarkin@

Move pluart(4) to dev/fdt.

ok jsg@

Move com_fdt.c to dev/fdt since we can share it between armv7 and arm64 now.

ok patrick@

Add a common rules file for ofw sources to help keep the configurations
of fdt-enabled platforms in sync.

OK deraadt@

Implement a power domain framework to turn on/off so-called power
domains. This mechanism is used by the newer i.MX8M SoCs so that
drivers can call into ATF to supply power to e.g. a USB port.

ok kettenis@

use the arm64 openprom.c on arm64
ok patrick@

Move common mutex implementations to a MI place.

Archs not yet converted can to the jump by defining __USE_MI_MUTEX.

ok visa@

Implement FUCKWIT for arm64; unmap the kernel almost entirely while userland
is running. This provides protection against meltown on cores that are
vilnerable (just Cortex-A75 so far) but also seems to be an essential to
protect against spectre-like attacks against the kernel.

This implementation only exposes a single treampoline page that does not
contain any kernel virtual addresses and also hides the real virtual address
of the exception vectors, which helps on cores vulnerable to "variant 3a"
(Cortex-A57, Cortex-A72). The implementation is inspired by the work done
by Will Deacon for Linux, but there are no knobs to turn it off. The
overhead is fairly limited: around 3-4% slowdown on Cortex-A57.

ok patrick@, deraadt@

Implement support for calling EFI runtime services and use it to implement
a time-of-day clock device based on the GetTime() and SetTime() services.
The virtual memory mappings for the runtime services calls are implemented
through a separate pmap that is only activated when we make a runtime services
call.

ok tom@, visa@
tested by naddy@

Split early startup code out of locore.S into locore0.S. Adjust link
run so that this locore0.o is always at the start of the executable.
But randomize the link order of all other .o files in the kernel, so
that their exec/rodata/data/bss segments land all over the place.
Late during kernel boot, unmap the early startup code.

As a result, the internal layout of every newly build bsd kernel is
different from past kernels. Internal relative offsets are not known
to an outside attacker. The only known offsets are in the startup code,
which has been unmapped.

Ramdisk kernels cannot be compiled like this, because they are gzip'd.
When the internal pointer references change, the compression dictionary
bloats and results in poorer compression.

With guidance and ok deraadt@

Add rkgrf(4), a driver that makes the "generic register file" of the Rockchip
RK3399 available to other drivers through the regmap interface.

Add agintc(4), a driver for interrupt controllers conforming to ARM's
generic interrupt controller architecture specification v3/4.

The hard work was done by drahn@, I just cleaned it up a bit and fixed
a couple of bugs.

ok patrick@, drahn@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Implement ampintcmsi(4) in ampintc(4) to support MSI. The GICv2M is an
extension to the GIC controller, which is represented as subnode in the
device tree. There can be multiple GICv2Ms, so it makes sense to attach
those to ampintc(4) as some kind of simplebus. The GICv2M is simply an
interrupt generator that can be used by PCIe devices to ring the door
bell. There is no need for further configuration, we only need to find
out which SPIs we are allowed to use for MSI and to register an edge
triggered interrupt on a (randomly) allocated SPI.

ok kettenis@

Hook up pciecam(4), but don't compile it yet.

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

The only difference between armv7 and arm64 fdt virtio attachments is an
uneeded include. Remove the include and move to MI fdt directory.

Make the RAMDISK kernel actually try to boot the ramdisk.

openprom(4)

Add drivers for the Raspberry Pi 3.

Move virtio config from files.pci to files.pci

virtio itself is not PCI, it is a paravirtual bus on top of either PCI
or MMIO (arm). This is the first step of moving the virtio files to
dev/pv. No functional change.

arm kernel changes tested by jsg@ and patrick@
no objections from sf@ mlarkin@

Add Allwinner A64 devices to OpenBSD/arm64. This means we can run
on the Pine64. While there add a few more usb and pseudo devices.

Consolidate bus space tags into generic bus space code. Especially
considering we don't have a separate iobus and the a4x iobus wasn't
even compiled it does not make sense to declare those in iobus.
Improves code readability as well.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Add ampchwm(4) a driver to access the Ampere Altra Fine-Grained Power Telemetry

This adds per core energy sensors (in Joules) and one per SoC temparature
sensor.
OK kettenis@ deraadt@

Implement access to EFI variables and ESRT through an ioctl(2) interface
that is compatible with what FreeBSD and NetBSD have. Setting EFI
variables is only allowed at securelevel 0 and below.

Heavily based on work done by Sergii Dmytruk.

ok yasuoka@

Add aplefuse(4), a driver for the eFuses on Apple Silicon SoCs.

ok patrick@

Add aplpwm(4), a driver for the PWM controller found on Apple silicon.

ok tobhe@

Tedu spmi bus definition that moved over to the fdt files.

Make EFI runtime calls more robust on arm64 as well. While I have not seen
any broken implementations yet, Linux developers claim that arm64 machines
intended to run Windows are not much better than x86 machines. And I
totally believe that.

ok patrick@

Reorganize the EFI code a bit. Move the efi.h header from dev/acpi to
dev/efi and rename the arm64 efi.c to efi_machdep.c, preparing the way
for MI EFI code and an amd64 implementation of EFI runtime support.

ok deraadt@, mlarkin@

Add apldcms(4), a driver for the touchpad on M2 laptops. This driver
needs firmware that is provided on the ESP by the Asahi installer and
copied into /etc/firmware/apple by the OpenBSD installer.

ok tobhe@

Add apldc(4), apldchidev(4), apldckdb(4) and aplrtkit(4). Together these
drivers implement support for the Dockchannel-base keyboard found on
Apple M2 laptops.

ok mlarkin@, patrick@

Add aplaudio(4) and aplmca(4). The aplmca(4) driver controls the hardware
block that takes data from apldma(4), serializes it and sends it out on
the i2s ports. The aplaudio(4) driver ties together aplmca(4) and various
codecs to present an audio(4) interface to the system.

This is still WIP, but good enough to play back audio on the speaker in the
M1 mini.

ok patrick@

The NVMe firmware distributed with the macOS 13 developer beta does not
like some of the shortcuts we've taken in the code that talks to RTKit.
In particular it does not like our NAK of the request for ioreport buffers.
So allocate the requested buffers and ACK the request instead.

However, the hardware implements a address filter between the coprocessor
that the firmware runs on and main memory. So we have to add a mapping
into this address filter such that the firmware can access these buffers.
Support for this address filter is implemented in a new aplsart(4) driver.

ok dlg@

Add aplcpu(4), a driver to control the CPU performance levels on Apple SoCs.

ok gnezdo@

Add apldma(4), a driver for the DMA controller found on Apple SoCs.
This driver implements an interface for using DMA for audio output.

ok ratchov@

Add aplnco(4), a driver for the Numerically-controlled oscillator (NCO)
clock that drives the audio clocks on Apple silicon.

ok patrick@

Add aplsmc(4), a driver for the SMC found on Apple M1 SoCs.

The SMC implements a lot of functionality. For now the driver only
implements a bunch of sensors. This is a small subset of the sensors
that are made available by the SMC as we don't know what measurments
are provided for most sensors.

ok patrick@

Remove apldwusb(4). This driver is now unused.

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Add apliic(4), a driver for the I2C controller found on various Apple SoCs.

ok kettenis@

Add an implementation for the protocol to communicate with coprocessors
running firmware based Apple's RTKit OS. Use this code to bring up the
storage controller that implements Apple's flavour of NVMe.

ok dlg@, patrick@

Enable aplmbox(4).

Add aplpmgr(4), a driver for the power management controller found on
various Apple SoCs.

ok patrick@

Add support for the keyboard/touchpad on Apple M1 laptops.

ok patrick@

Add aplspi(4), a driver for the SPI controller found on the Apple M1 SoC.

ok patrick@

Add aplpinctrl(4), a driver for the Apple GPIO controller found on M1 SoCs.

ok patrick@

start working on support for Apple NVME Storage as found in apple M1 devs

the Apple NVME Storage (aka ans) controller is almost but not quite
a vanilla nvme controller. one difference is that it doesnt attach
to a pci bus, so it needs this custom bus glue. this custom bus
glue also provides us with a nice way to provide a different set
of functions to handle other things that ans does differently to
vanilla nvme controllers.

this is different to how linux deals with ans. the linux support
fakes a pci controller for ans to attach to. i assumed that at some
point a vendor would include nvme in an soc directly and made it a
bus independent driver from day 1. turns out i was right, but i
would never have guessed that the vendor would be apple.

some of the other differences between vanilla nvme and ans are
around command submission and completion. ans nvme command submission
is done via an array of command slots where the host picks a slot
and then posts every slot number it fills in to a doorbell. this
is different to vanilla nvme controllers which use a ring, and post
the producer index in that ring to a doorbell. ans also includes
some weird iommu, the handling of which we wrap up into the command
submission and completion functions.

this code is not yet enabled because it is incomplete. i'm getting
what i've done in so people with actual hardware can start poking
it more seriously.

this implementation is based on information figured out by the guys
porting linux to apple hardware.

Add aplspmi(4), a driver for the Apple SPMI controller, and aplpmu(4)
a driver for the Apple "sera" SPMI power management unit that contains
the RTC on Apple M1 systems.

ok patrick@

Add apldwusb(4), a glue driver for the Synpsys DesignWare USB 3 controllers
found on the Apple M1 SoC.

Move the pciecam stanza back into files.arm64 since armv7 has its own
pciecam(4) implementation hidden away in arch/armv7/vexpress.

Unbreaks armv7 kernel builds.

Move pciecam.c to dev/fdt/.

ok deraadt@

acpiiort(4) and smmu(4)

Add apldart(4), a driver for the IOMMU on Apple M1 SoCs.

ok patrick@

Add aplcpie(4), a (minimal) driver for the PCIe host bridge on Apple M1 SoCs.

ok patrick@

Add aplintc(4), a driver for the interrupt controller found on
Apple M1 SoCs.

ok patrick@

Enable apldog(4).

Add cryptox(4), a driver for armv8 cryptographic extensions.
The driver currently only supports AES-CBC mode but can easily
be extended to other algorithms and modes.
The aesv8-armx.S file was generated from the CRYPTOGAMS project.

Asked to commit by and ok patrick@

Port over NetBSD's arm64 disassembler for DDB.

ok kettenis@, patrick@

The dwctwo(4) FDT glue is now identical on armv7 and arm64. Move it to
/dev/fdt.

ok patrick@, visa@

Implement smbios support on arm64.

ok deraadt@, jsg@

permit maxusers to grow up to 128

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Add the apm(4) subsystem to arm64.

NB: acpi(4) is not hooked up to this yet.

previous version OK deraadt@ patrick@
OK jsg@

Add support for the GIC v3 ITS and use it to implement MSI support for
rkpcie(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

Hook up acpi(4) on arm64. Various bits of driver glue are still missing
but this is enough to boot multi-user on the mcbin with suitable firmware.

ok mlarkin@

Move pluart(4) to dev/fdt.

ok jsg@

Move com_fdt.c to dev/fdt since we can share it between armv7 and arm64 now.

ok patrick@

Add a common rules file for ofw sources to help keep the configurations
of fdt-enabled platforms in sync.

OK deraadt@

Implement a power domain framework to turn on/off so-called power
domains. This mechanism is used by the newer i.MX8M SoCs so that
drivers can call into ATF to supply power to e.g. a USB port.

ok kettenis@

use the arm64 openprom.c on arm64
ok patrick@

Move common mutex implementations to a MI place.

Archs not yet converted can to the jump by defining __USE_MI_MUTEX.

ok visa@

Implement FUCKWIT for arm64; unmap the kernel almost entirely while userland
is running. This provides protection against meltown on cores that are
vilnerable (just Cortex-A75 so far) but also seems to be an essential to
protect against spectre-like attacks against the kernel.

This implementation only exposes a single treampoline page that does not
contain any kernel virtual addresses and also hides the real virtual address
of the exception vectors, which helps on cores vulnerable to "variant 3a"
(Cortex-A57, Cortex-A72). The implementation is inspired by the work done
by Will Deacon for Linux, but there are no knobs to turn it off. The
overhead is fairly limited: around 3-4% slowdown on Cortex-A57.

ok patrick@, deraadt@

Implement support for calling EFI runtime services and use it to implement
a time-of-day clock device based on the GetTime() and SetTime() services.
The virtual memory mappings for the runtime services calls are implemented
through a separate pmap that is only activated when we make a runtime services
call.

ok tom@, visa@
tested by naddy@

Split early startup code out of locore.S into locore0.S. Adjust link
run so that this locore0.o is always at the start of the executable.
But randomize the link order of all other .o files in the kernel, so
that their exec/rodata/data/bss segments land all over the place.
Late during kernel boot, unmap the early startup code.

As a result, the internal layout of every newly build bsd kernel is
different from past kernels. Internal relative offsets are not known
to an outside attacker. The only known offsets are in the startup code,
which has been unmapped.

Ramdisk kernels cannot be compiled like this, because they are gzip'd.
When the internal pointer references change, the compression dictionary
bloats and results in poorer compression.

With guidance and ok deraadt@

Add rkgrf(4), a driver that makes the "generic register file" of the Rockchip
RK3399 available to other drivers through the regmap interface.

Add agintc(4), a driver for interrupt controllers conforming to ARM's
generic interrupt controller architecture specification v3/4.

The hard work was done by drahn@, I just cleaned it up a bit and fixed
a couple of bugs.

ok patrick@, drahn@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Implement ampintcmsi(4) in ampintc(4) to support MSI. The GICv2M is an
extension to the GIC controller, which is represented as subnode in the
device tree. There can be multiple GICv2Ms, so it makes sense to attach
those to ampintc(4) as some kind of simplebus. The GICv2M is simply an
interrupt generator that can be used by PCIe devices to ring the door
bell. There is no need for further configuration, we only need to find
out which SPIs we are allowed to use for MSI and to register an edge
triggered interrupt on a (randomly) allocated SPI.

ok kettenis@

Hook up pciecam(4), but don't compile it yet.

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

The only difference between armv7 and arm64 fdt virtio attachments is an
uneeded include. Remove the include and move to MI fdt directory.

Make the RAMDISK kernel actually try to boot the ramdisk.

openprom(4)

Add drivers for the Raspberry Pi 3.

Move virtio config from files.pci to files.pci

virtio itself is not PCI, it is a paravirtual bus on top of either PCI
or MMIO (arm). This is the first step of moving the virtio files to
dev/pv. No functional change.

arm kernel changes tested by jsg@ and patrick@
no objections from sf@ mlarkin@

Add Allwinner A64 devices to OpenBSD/arm64. This means we can run
on the Pine64. While there add a few more usb and pseudo devices.

Consolidate bus space tags into generic bus space code. Especially
considering we don't have a separate iobus and the a4x iobus wasn't
even compiled it does not make sense to declare those in iobus.
Improves code readability as well.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Implement access to EFI variables and ESRT through an ioctl(2) interface
that is compatible with what FreeBSD and NetBSD have. Setting EFI
variables is only allowed at securelevel 0 and below.

Heavily based on work done by Sergii Dmytruk.

ok yasuoka@

Add aplefuse(4), a driver for the eFuses on Apple Silicon SoCs.

ok patrick@

Add aplpwm(4), a driver for the PWM controller found on Apple silicon.

ok tobhe@

Tedu spmi bus definition that moved over to the fdt files.

Make EFI runtime calls more robust on arm64 as well. While I have not seen
any broken implementations yet, Linux developers claim that arm64 machines
intended to run Windows are not much better than x86 machines. And I
totally believe that.

ok patrick@

Reorganize the EFI code a bit. Move the efi.h header from dev/acpi to
dev/efi and rename the arm64 efi.c to efi_machdep.c, preparing the way
for MI EFI code and an amd64 implementation of EFI runtime support.

ok deraadt@, mlarkin@

Add apldcms(4), a driver for the touchpad on M2 laptops. This driver
needs firmware that is provided on the ESP by the Asahi installer and
copied into /etc/firmware/apple by the OpenBSD installer.

ok tobhe@

Add apldc(4), apldchidev(4), apldckdb(4) and aplrtkit(4). Together these
drivers implement support for the Dockchannel-base keyboard found on
Apple M2 laptops.

ok mlarkin@, patrick@

Add aplaudio(4) and aplmca(4). The aplmca(4) driver controls the hardware
block that takes data from apldma(4), serializes it and sends it out on
the i2s ports. The aplaudio(4) driver ties together aplmca(4) and various
codecs to present an audio(4) interface to the system.

This is still WIP, but good enough to play back audio on the speaker in the
M1 mini.

ok patrick@

The NVMe firmware distributed with the macOS 13 developer beta does not
like some of the shortcuts we've taken in the code that talks to RTKit.
In particular it does not like our NAK of the request for ioreport buffers.
So allocate the requested buffers and ACK the request instead.

However, the hardware implements a address filter between the coprocessor
that the firmware runs on and main memory. So we have to add a mapping
into this address filter such that the firmware can access these buffers.
Support for this address filter is implemented in a new aplsart(4) driver.

ok dlg@

Add aplcpu(4), a driver to control the CPU performance levels on Apple SoCs.

ok gnezdo@

Add apldma(4), a driver for the DMA controller found on Apple SoCs.
This driver implements an interface for using DMA for audio output.

ok ratchov@

Add aplnco(4), a driver for the Numerically-controlled oscillator (NCO)
clock that drives the audio clocks on Apple silicon.

ok patrick@

Add aplsmc(4), a driver for the SMC found on Apple M1 SoCs.

The SMC implements a lot of functionality. For now the driver only
implements a bunch of sensors. This is a small subset of the sensors
that are made available by the SMC as we don't know what measurments
are provided for most sensors.

ok patrick@

Remove apldwusb(4). This driver is now unused.

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Add apliic(4), a driver for the I2C controller found on various Apple SoCs.

ok kettenis@

Add an implementation for the protocol to communicate with coprocessors
running firmware based Apple's RTKit OS. Use this code to bring up the
storage controller that implements Apple's flavour of NVMe.

ok dlg@, patrick@

Enable aplmbox(4).

Add aplpmgr(4), a driver for the power management controller found on
various Apple SoCs.

ok patrick@

Add support for the keyboard/touchpad on Apple M1 laptops.

ok patrick@

Add aplspi(4), a driver for the SPI controller found on the Apple M1 SoC.

ok patrick@

Add aplpinctrl(4), a driver for the Apple GPIO controller found on M1 SoCs.

ok patrick@

start working on support for Apple NVME Storage as found in apple M1 devs

the Apple NVME Storage (aka ans) controller is almost but not quite
a vanilla nvme controller. one difference is that it doesnt attach
to a pci bus, so it needs this custom bus glue. this custom bus
glue also provides us with a nice way to provide a different set
of functions to handle other things that ans does differently to
vanilla nvme controllers.

this is different to how linux deals with ans. the linux support
fakes a pci controller for ans to attach to. i assumed that at some
point a vendor would include nvme in an soc directly and made it a
bus independent driver from day 1. turns out i was right, but i
would never have guessed that the vendor would be apple.

some of the other differences between vanilla nvme and ans are
around command submission and completion. ans nvme command submission
is done via an array of command slots where the host picks a slot
and then posts every slot number it fills in to a doorbell. this
is different to vanilla nvme controllers which use a ring, and post
the producer index in that ring to a doorbell. ans also includes
some weird iommu, the handling of which we wrap up into the command
submission and completion functions.

this code is not yet enabled because it is incomplete. i'm getting
what i've done in so people with actual hardware can start poking
it more seriously.

this implementation is based on information figured out by the guys
porting linux to apple hardware.

Add aplspmi(4), a driver for the Apple SPMI controller, and aplpmu(4)
a driver for the Apple "sera" SPMI power management unit that contains
the RTC on Apple M1 systems.

ok patrick@

Add apldwusb(4), a glue driver for the Synpsys DesignWare USB 3 controllers
found on the Apple M1 SoC.

Move the pciecam stanza back into files.arm64 since armv7 has its own
pciecam(4) implementation hidden away in arch/armv7/vexpress.

Unbreaks armv7 kernel builds.

Move pciecam.c to dev/fdt/.

ok deraadt@

acpiiort(4) and smmu(4)

Add apldart(4), a driver for the IOMMU on Apple M1 SoCs.

ok patrick@

Add aplcpie(4), a (minimal) driver for the PCIe host bridge on Apple M1 SoCs.

ok patrick@

Add aplintc(4), a driver for the interrupt controller found on
Apple M1 SoCs.

ok patrick@

Enable apldog(4).

Add cryptox(4), a driver for armv8 cryptographic extensions.
The driver currently only supports AES-CBC mode but can easily
be extended to other algorithms and modes.
The aesv8-armx.S file was generated from the CRYPTOGAMS project.

Asked to commit by and ok patrick@

Port over NetBSD's arm64 disassembler for DDB.

ok kettenis@, patrick@

The dwctwo(4) FDT glue is now identical on armv7 and arm64. Move it to
/dev/fdt.

ok patrick@, visa@

Implement smbios support on arm64.

ok deraadt@, jsg@

permit maxusers to grow up to 128

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Add the apm(4) subsystem to arm64.

NB: acpi(4) is not hooked up to this yet.

previous version OK deraadt@ patrick@
OK jsg@

Add support for the GIC v3 ITS and use it to implement MSI support for
rkpcie(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

Hook up acpi(4) on arm64. Various bits of driver glue are still missing
but this is enough to boot multi-user on the mcbin with suitable firmware.

ok mlarkin@

Move pluart(4) to dev/fdt.

ok jsg@

Move com_fdt.c to dev/fdt since we can share it between armv7 and arm64 now.

ok patrick@

Add a common rules file for ofw sources to help keep the configurations
of fdt-enabled platforms in sync.

OK deraadt@

Implement a power domain framework to turn on/off so-called power
domains. This mechanism is used by the newer i.MX8M SoCs so that
drivers can call into ATF to supply power to e.g. a USB port.

ok kettenis@

use the arm64 openprom.c on arm64
ok patrick@

Move common mutex implementations to a MI place.

Archs not yet converted can to the jump by defining __USE_MI_MUTEX.

ok visa@

Implement FUCKWIT for arm64; unmap the kernel almost entirely while userland
is running. This provides protection against meltown on cores that are
vilnerable (just Cortex-A75 so far) but also seems to be an essential to
protect against spectre-like attacks against the kernel.

This implementation only exposes a single treampoline page that does not
contain any kernel virtual addresses and also hides the real virtual address
of the exception vectors, which helps on cores vulnerable to "variant 3a"
(Cortex-A57, Cortex-A72). The implementation is inspired by the work done
by Will Deacon for Linux, but there are no knobs to turn it off. The
overhead is fairly limited: around 3-4% slowdown on Cortex-A57.

ok patrick@, deraadt@

Implement support for calling EFI runtime services and use it to implement
a time-of-day clock device based on the GetTime() and SetTime() services.
The virtual memory mappings for the runtime services calls are implemented
through a separate pmap that is only activated when we make a runtime services
call.

ok tom@, visa@
tested by naddy@

Split early startup code out of locore.S into locore0.S. Adjust link
run so that this locore0.o is always at the start of the executable.
But randomize the link order of all other .o files in the kernel, so
that their exec/rodata/data/bss segments land all over the place.
Late during kernel boot, unmap the early startup code.

As a result, the internal layout of every newly build bsd kernel is
different from past kernels. Internal relative offsets are not known
to an outside attacker. The only known offsets are in the startup code,
which has been unmapped.

Ramdisk kernels cannot be compiled like this, because they are gzip'd.
When the internal pointer references change, the compression dictionary
bloats and results in poorer compression.

With guidance and ok deraadt@

Add rkgrf(4), a driver that makes the "generic register file" of the Rockchip
RK3399 available to other drivers through the regmap interface.

Add agintc(4), a driver for interrupt controllers conforming to ARM's
generic interrupt controller architecture specification v3/4.

The hard work was done by drahn@, I just cleaned it up a bit and fixed
a couple of bugs.

ok patrick@, drahn@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Implement ampintcmsi(4) in ampintc(4) to support MSI. The GICv2M is an
extension to the GIC controller, which is represented as subnode in the
device tree. There can be multiple GICv2Ms, so it makes sense to attach
those to ampintc(4) as some kind of simplebus. The GICv2M is simply an
interrupt generator that can be used by PCIe devices to ring the door
bell. There is no need for further configuration, we only need to find
out which SPIs we are allowed to use for MSI and to register an edge
triggered interrupt on a (randomly) allocated SPI.

ok kettenis@

Hook up pciecam(4), but don't compile it yet.

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

The only difference between armv7 and arm64 fdt virtio attachments is an
uneeded include. Remove the include and move to MI fdt directory.

Make the RAMDISK kernel actually try to boot the ramdisk.

openprom(4)

Add drivers for the Raspberry Pi 3.

Move virtio config from files.pci to files.pci

virtio itself is not PCI, it is a paravirtual bus on top of either PCI
or MMIO (arm). This is the first step of moving the virtio files to
dev/pv. No functional change.

arm kernel changes tested by jsg@ and patrick@
no objections from sf@ mlarkin@

Add Allwinner A64 devices to OpenBSD/arm64. This means we can run
on the Pine64. While there add a few more usb and pseudo devices.

Consolidate bus space tags into generic bus space code. Especially
considering we don't have a separate iobus and the a4x iobus wasn't
even compiled it does not make sense to declare those in iobus.
Improves code readability as well.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Add aplefuse(4), a driver for the eFuses on Apple Silicon SoCs.

ok patrick@

Add aplpwm(4), a driver for the PWM controller found on Apple silicon.

ok tobhe@

Tedu spmi bus definition that moved over to the fdt files.

Make EFI runtime calls more robust on arm64 as well. While I have not seen
any broken implementations yet, Linux developers claim that arm64 machines
intended to run Windows are not much better than x86 machines. And I
totally believe that.

ok patrick@

Reorganize the EFI code a bit. Move the efi.h header from dev/acpi to
dev/efi and rename the arm64 efi.c to efi_machdep.c, preparing the way
for MI EFI code and an amd64 implementation of EFI runtime support.

ok deraadt@, mlarkin@

Add apldcms(4), a driver for the touchpad on M2 laptops. This driver
needs firmware that is provided on the ESP by the Asahi installer and
copied into /etc/firmware/apple by the OpenBSD installer.

ok tobhe@

Add apldc(4), apldchidev(4), apldckdb(4) and aplrtkit(4). Together these
drivers implement support for the Dockchannel-base keyboard found on
Apple M2 laptops.

ok mlarkin@, patrick@

Add aplaudio(4) and aplmca(4). The aplmca(4) driver controls the hardware
block that takes data from apldma(4), serializes it and sends it out on
the i2s ports. The aplaudio(4) driver ties together aplmca(4) and various
codecs to present an audio(4) interface to the system.

This is still WIP, but good enough to play back audio on the speaker in the
M1 mini.

ok patrick@

The NVMe firmware distributed with the macOS 13 developer beta does not
like some of the shortcuts we've taken in the code that talks to RTKit.
In particular it does not like our NAK of the request for ioreport buffers.
So allocate the requested buffers and ACK the request instead.

However, the hardware implements a address filter between the coprocessor
that the firmware runs on and main memory. So we have to add a mapping
into this address filter such that the firmware can access these buffers.
Support for this address filter is implemented in a new aplsart(4) driver.

ok dlg@

Add aplcpu(4), a driver to control the CPU performance levels on Apple SoCs.

ok gnezdo@

Add apldma(4), a driver for the DMA controller found on Apple SoCs.
This driver implements an interface for using DMA for audio output.

ok ratchov@

Add aplnco(4), a driver for the Numerically-controlled oscillator (NCO)
clock that drives the audio clocks on Apple silicon.

ok patrick@

Add aplsmc(4), a driver for the SMC found on Apple M1 SoCs.

The SMC implements a lot of functionality. For now the driver only
implements a bunch of sensors. This is a small subset of the sensors
that are made available by the SMC as we don't know what measurments
are provided for most sensors.

ok patrick@

Remove apldwusb(4). This driver is now unused.

Rewrite the kernel FPU handling code. The new code saves the FPU state
in cpu_switch() instead of at the kernel edge and gets rid of the FPU
state tracking in struct cpu_info and struct pcb. This fixes the random
crashes seen with SMP kernels on Apple M1.

ok patrick@

Add apliic(4), a driver for the I2C controller found on various Apple SoCs.

ok kettenis@

Add an implementation for the protocol to communicate with coprocessors
running firmware based Apple's RTKit OS. Use this code to bring up the
storage controller that implements Apple's flavour of NVMe.

ok dlg@, patrick@

Enable aplmbox(4).

Add aplpmgr(4), a driver for the power management controller found on
various Apple SoCs.

ok patrick@

Add support for the keyboard/touchpad on Apple M1 laptops.

ok patrick@

Add aplspi(4), a driver for the SPI controller found on the Apple M1 SoC.

ok patrick@

Add aplpinctrl(4), a driver for the Apple GPIO controller found on M1 SoCs.

ok patrick@

start working on support for Apple NVME Storage as found in apple M1 devs

the Apple NVME Storage (aka ans) controller is almost but not quite
a vanilla nvme controller. one difference is that it doesnt attach
to a pci bus, so it needs this custom bus glue. this custom bus
glue also provides us with a nice way to provide a different set
of functions to handle other things that ans does differently to
vanilla nvme controllers.

this is different to how linux deals with ans. the linux support
fakes a pci controller for ans to attach to. i assumed that at some
point a vendor would include nvme in an soc directly and made it a
bus independent driver from day 1. turns out i was right, but i
would never have guessed that the vendor would be apple.

some of the other differences between vanilla nvme and ans are
around command submission and completion. ans nvme command submission
is done via an array of command slots where the host picks a slot
and then posts every slot number it fills in to a doorbell. this
is different to vanilla nvme controllers which use a ring, and post
the producer index in that ring to a doorbell. ans also includes
some weird iommu, the handling of which we wrap up into the command
submission and completion functions.

this code is not yet enabled because it is incomplete. i'm getting
what i've done in so people with actual hardware can start poking
it more seriously.

this implementation is based on information figured out by the guys
porting linux to apple hardware.

Add aplspmi(4), a driver for the Apple SPMI controller, and aplpmu(4)
a driver for the Apple "sera" SPMI power management unit that contains
the RTC on Apple M1 systems.

ok patrick@

Add apldwusb(4), a glue driver for the Synpsys DesignWare USB 3 controllers
found on the Apple M1 SoC.

Move the pciecam stanza back into files.arm64 since armv7 has its own
pciecam(4) implementation hidden away in arch/armv7/vexpress.

Unbreaks armv7 kernel builds.

Move pciecam.c to dev/fdt/.

ok deraadt@

acpiiort(4) and smmu(4)

Add apldart(4), a driver for the IOMMU on Apple M1 SoCs.

ok patrick@

Add aplcpie(4), a (minimal) driver for the PCIe host bridge on Apple M1 SoCs.

ok patrick@

Add aplintc(4), a driver for the interrupt controller found on
Apple M1 SoCs.

ok patrick@

Enable apldog(4).

Add cryptox(4), a driver for armv8 cryptographic extensions.
The driver currently only supports AES-CBC mode but can easily
be extended to other algorithms and modes.
The aesv8-armx.S file was generated from the CRYPTOGAMS project.

Asked to commit by and ok patrick@

Port over NetBSD's arm64 disassembler for DDB.

ok kettenis@, patrick@

The dwctwo(4) FDT glue is now identical on armv7 and arm64. Move it to
/dev/fdt.

ok patrick@, visa@

Implement smbios support on arm64.

ok deraadt@, jsg@

permit maxusers to grow up to 128

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Add the apm(4) subsystem to arm64.

NB: acpi(4) is not hooked up to this yet.

previous version OK deraadt@ patrick@
OK jsg@

Add support for the GIC v3 ITS and use it to implement MSI support for
rkpcie(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

Hook up acpi(4) on arm64. Various bits of driver glue are still missing
but this is enough to boot multi-user on the mcbin with suitable firmware.

ok mlarkin@

Move pluart(4) to dev/fdt.

ok jsg@

Move com_fdt.c to dev/fdt since we can share it between armv7 and arm64 now.

ok patrick@

Add a common rules file for ofw sources to help keep the configurations
of fdt-enabled platforms in sync.

OK deraadt@

Implement a power domain framework to turn on/off so-called power
domains. This mechanism is used by the newer i.MX8M SoCs so that
drivers can call into ATF to supply power to e.g. a USB port.

ok kettenis@

use the arm64 openprom.c on arm64
ok patrick@

Move common mutex implementations to a MI place.

Archs not yet converted can to the jump by defining __USE_MI_MUTEX.

ok visa@

Implement FUCKWIT for arm64; unmap the kernel almost entirely while userland
is running. This provides protection against meltown on cores that are
vilnerable (just Cortex-A75 so far) but also seems to be an essential to
protect against spectre-like attacks against the kernel.

This implementation only exposes a single treampoline page that does not
contain any kernel virtual addresses and also hides the real virtual address
of the exception vectors, which helps on cores vulnerable to "variant 3a"
(Cortex-A57, Cortex-A72). The implementation is inspired by the work done
by Will Deacon for Linux, but there are no knobs to turn it off. The
overhead is fairly limited: around 3-4% slowdown on Cortex-A57.

ok patrick@, deraadt@

Implement support for calling EFI runtime services and use it to implement
a time-of-day clock device based on the GetTime() and SetTime() services.
The virtual memory mappings for the runtime services calls are implemented
through a separate pmap that is only activated when we make a runtime services
call.

ok tom@, visa@
tested by naddy@

Split early startup code out of locore.S into locore0.S. Adjust link
run so that this locore0.o is always at the start of the executable.
But randomize the link order of all other .o files in the kernel, so
that their exec/rodata/data/bss segments land all over the place.
Late during kernel boot, unmap the early startup code.

As a result, the internal layout of every newly build bsd kernel is
different from past kernels. Internal relative offsets are not known
to an outside attacker. The only known offsets are in the startup code,
which has been unmapped.

Ramdisk kernels cannot be compiled like this, because they are gzip'd.
When the internal pointer references change, the compression dictionary
bloats and results in poorer compression.

With guidance and ok deraadt@

Add rkgrf(4), a driver that makes the "generic register file" of the Rockchip
RK3399 available to other drivers through the regmap interface.

Add agintc(4), a driver for interrupt controllers conforming to ARM's
generic interrupt controller architecture specification v3/4.

The hard work was done by drahn@, I just cleaned it up a bit and fixed
a couple of bugs.

ok patrick@, drahn@

Add code to identify the CPUs on arm64 systems. The primary CPU is attached
and identified early on. For the secondary CPUs this happens late, such that
the drivers we need to spin up CPUs, such as psci(4), will be available.

This also fixes some code in simplebus(4) where the return value of
OF_getprop() was not properly checked.

Heavily based on an earlier diff from drahn@.

ok drahn@, jsg@

Implement ampintcmsi(4) in ampintc(4) to support MSI. The GICv2M is an
extension to the GIC controller, which is represented as subnode in the
device tree. There can be multiple GICv2Ms, so it makes sense to attach
those to ampintc(4) as some kind of simplebus. The GICv2M is simply an
interrupt generator that can be used by PCIe devices to ring the door
bell. There is no need for further configuration, we only need to find
out which SPIs we are allowed to use for MSI and to register an edge
triggered interrupt on a (randomly) allocated SPI.

ok kettenis@

Hook up pciecam(4), but don't compile it yet.

Move cache and tlb flush functions, which were mostly inline assembly,
into separate functions. This makes them reusable from other parts in
the kernel. Assembly and header are taken from FreeBSD, but modified
to fit our requirements and with some unnecessary stuff removed. While
there remove micro optimization for uniprocessor kernels.

The only difference between armv7 and arm64 fdt virtio attachments is an
uneeded include. Remove the include and move to MI fdt directory.

Make the RAMDISK kernel actually try to boot the ramdisk.

openprom(4)

Add drivers for the Raspberry Pi 3.

Move virtio config from files.pci to files.pci

virtio itself is not PCI, it is a paravirtual bus on top of either PCI
or MMIO (arm). This is the first step of moving the virtio files to
dev/pv. No functional change.

arm kernel changes tested by jsg@ and patrick@
no objections from sf@ mlarkin@

Add Allwinner A64 devices to OpenBSD/arm64. This means we can run
on the Pine64. While there add a few more usb and pseudo devices.

Consolidate bus space tags into generic bus space code. Especially
considering we don't have a separate iobus and the a4x iobus wasn't
even compiled it does not make sense to declare those in iobus.
Improves code readability as well.

Import of OpenBSD/arm64

This commit contains all the kernel files related to the OpenBSD/arm64
port. It is based on the PowerPC pmap, loongson, arm/armv7 code and
FreeBSD aarch64 code. Hard work done by Dale Rahn.

Add aplpwm(4), a driver for the PWM controller found on Apple silicon.

ok tobhe@