Merge from stable/11 305773:
Fix the arm64 kernel build when DDB is disabled, debug_monitor.c
depends on DDB, and is unused when it's disabled.

Approved by: re (gjb)

Merge from stable/11 305773:
Fix the arm64 kernel build when DDB is disabled, debug_monitor.c
depends on DDB, and is unused when it's disabled.

Approved by: re (gjb)

Copy stable/11@r303970 to releng/11.0 as part of the 11.0-RELEASE
cycle.

Prune svn:mergeinfo from the new branch, and rename it to RC1.

Update __FreeBSD_version.

Use the quarterly branch for the default FreeBSD.conf pkg(8) repo and
the dvd1.iso packages population.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Remove the old pre-INTRNG arm64 interrupt framework. GENERIC was switched
to INTRNG in r301565 with the old code no longer being built by default with
no reports of issues on any supported hardware.

Approved by: re (gjb)
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

Add the GICv3 ITS intrng driver. As the interface to the interrupt
framework has significantly changed the driver has moved to a new file.
While it shares some code with the existing driver this has been modified
to work better with the intrng framework.

This has been tested on the ThunderX servers in the netperf cluster and has
been used to boot them for other testing, including DTrace and hwpmc.

With this we can use intrng on all supported arm64 platforms I was able to
test on. It is expected we will move to intrng soon, and disable the old
arm64 interrupt framework.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6437

Attach the generic USB OHCI driver to the arm64 build.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

Allow building VNIC as a module

Add directory structure and fix dependencies to be able to
build and use Cavium VNIC driver as a module.

Reviewed by: zbb
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D6345

Add intrng support to the GICv3 driver. It lacks ITS support so won't handle
MSI or MSI-X interrupts, however this is enought to boot FreeBSD under the
ARM Foundation Model with a GICv3 interrupt controller.

Approved by: ABT Systems Ltd
Relnotes: yes
Sponsored by: The FreeBSD Foundation

Add support for intrng to arm64. As the GICv3 drivers will need to be
updated, and until further testing can be done, this is disabled for now.

It is expected arm64 will switch to this interface, and the old interface
will be removed before 11.0 is released.

Obtained from: ABT Systems Ltd
Relnotes: yes
Sponsored by: The FreeBSD Foundation

Move arm's devmap to some generic place, so it can be used
by other architectures.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D6091
Sponsored by: DARPA, AFRL
Sponsored by: HEIF5

arm64: bzero optimization

This optimization attempts to utylize as wide as possible register store instructions to zero large buffers.
The implementation, if possible, will use 'dc zva' to zero buffer by cache lines.

Speedup: 60x faster memory zeroing

Submitted by: Dominik Ermel <der@semihalf.com>
Obtained from: Semihalf
Sponsored by: Cavium
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D5726

Reduce OFW PCI code duplication - involves ARM, PPC and SPARC64

Import portions of the PowerPC OF PCI implementation into new file
"ofwpci.c", common for other platforms. The files ofw_pci.c and ofw_pci.h
from sys/powerpc/ofw no longer exist. All required declarations are moved
to sys/dev/ofw/ofwpci.h. This creates a new ofw_pci_write_ivar() function
and modifies some others methods. Most functions contain existing ppc
implementations in the majority unchanged. Now there is no need to have
multiple identical copies of methods for various architectures.

Requested by: jhibbits
Reviewed by: jhibbits, marius
Submitted by: Marcin Mazurek <mma@semihalf.com>
Obtained from: Semihalf
Sponsored by: Annapurna Labs
Differential Revision: https://reviews.freebsd.org/D4879

Make pci_host_generic and thunderx_pci common
* provided OFW interface for pci_host_generic (for handling devices which are present in DTS under the PCI node)
* removed support for internal PCI from arm64/cavium
* cleaned up and made most of the code common

Obtained from: Semihalf
Sponsored by: Cavium
Approved by: cognet (mentor)
Reviewed by: zbb
Differential revision: https://reviews.freebsd.org/D5261

Support PEM that is not a PCI endpoint on ThunderX

Some chip revisions don't have their external PCIe buses
behind the internal bridge. Add support for FDT-configurable
PEMs but keep ability for PCIe enumeration.

Reviewed by: andrew, wma
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5285

Framework for ARM64 instruction disassembler

Provide an easy to use framework for ARM64 DDB disassembler.
This commit does not contain full list of instruction opcodes.

Obtained from: Semihalf
Sponsored by: Cavium
Approved by: cognet (mentor)
Reviewed by: zbb, andrew, cognet
Differential revision: https://reviews.freebsd.org/D5114

Support new MDIO hierarchy in ThunderX DTB

Some firmware revisions provide different DTB tree that include
odd MDIO placement in the tree.
This commit adds support for 2 new buses:
- MRML bridge (PCIB subordinate)
- MDIO nexus (MRML subordinate)
This allows for the correct MDIO attachment with both - new and old
firmware.

Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5070

Divide ThunderX PCIe driver to general and FDT part

- Separate FDT and general PCIe driver parts
- Drop some irrelevant printfs that cannot be displayed in
FDT attach
- Move ranges parsing to FDT portion of PCIe code

Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5067

Remove fdt_fixup_table from architectures where it's unneeded. We only make
use of fdt_fixup_table on PowerPC and ARM. As such we can remove it from
other architectures as it's unneeded.

Reviewed by: nwhitehorn
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D5013

Add an OF_decode_addr() implementation for arm64.

Discussed with: andrew

Add support for usermode (vdso-like) gettimeofday(2) and
clock_gettime(2) on ARMv7 and ARMv8 systems which have architectural
generic timer hardware. It is similar how the RDTSC timer is used in
userspace on x86.

Fix a permission problem where generic timer access from EL0 (or
userspace on v7) was not properly initialized on APs.

For ARMv7, mark the stack non-executable. The shared page is added for
all arms (including ARMv8 64bit), and the signal trampoline code is
moved to the page.

Reviewed by: andrew
Discussed with: emaste, mmel
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D4209

Add support for a generic AHCI attachment. This allows us to attach to a
typically memory mapped bus, for example on the AMD Opteron A1100 the AHCI
device is mapped in the CPUs address space, and not through a PCI
controller.

Further work is needed for this to work with ACPI as this is expected to be
common on ARMv8 servers.

Reviewed by: mav, mmel
Obtained from: mmel, ABT Systems Ltd
Relnotes: yes
Sponsored by: SoftIron Inc
Differential Revision: https://reviews.freebsd.org/D4269

Fix the open solaris atomic functions on arm64. Without this we may use the
wrong value in the comparison, leading to incorrectly setting the new
value.

This has been observed in the ZFS code. Without this we can lose track of
the reference count in a zrlock object.

We should move to use the generic atomic functions, however as this has
been observed I would prefer to have this working, then move to the generic
functions.

PR: 204037
Sponsored by: ABT Systems Ltd

Add libkern ffsll() for parity with flsll()

Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D3962

Add support for CloudABI on ARM64.

It turns out that it is pretty easy to make CloudABI work on ARM64. We
essentially only need to copy over the sysvec from AMD64 and ensure that
we use ARM64 specific registers.

As there is an overlap between function argument and return registers,
we do need to extend cloudabi64_schedtail() to only set its values if
we're actually forking. Not when we're creating a new thread.

Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D3917

Introduce driver for Cavium's ThunderX MDIO

This commit adds support for MDIO present in the ThunderX SoC.
From the FDT point of view it is compatible with "octeon-3860-mdio"
however only C22 mode is used.
The code also implements lmac_if interface functions.

Obtained from: Semihalf
Sponsored by: The FreeBSD Foundation

Introduce initial support for Cavium's ThunderX networking interface

- The driver consists of three main componens: PF, VF, BGX
- Requires appropriate entries in DTS and MDIO driver
- Supports only FDT configuration
- Multiple Tx queues and single Rx queue supported
- No RSS, HW checksum and TSO support
- No more than 8 queues per-IF (only one Queue Set per IF)
- HW statistics enabled
- Works in all available MAC modes (1,10,20,40G)
- Style converted to BSD according to style(9)
- The code brings lmac_if interface used by the BGX driver to
update its logical MACs state.

Obtained from: Semihalf
Sponsored by: The FreeBSD Foundation

Move kbd.c to main sys/conf/files list

It is (optionally) used on all architectures.

Sponsored by: The FreeBSD Foundation

arm64: add kbd.c to the build for ukbd to fix the build

Pointy hat to: emaste

Add support for the dwc usb in the HiSilicon hi6220 in the HiKey board. For
this we need to force the driver into host mode, as without this the driver
fails to detect any devices.

Relnotes: yes
Sponsored by: ABT Systems Ltd

Add support for the DesignWare MMC hardware in the HiSilicon hi6220. This
SoC is used in the HiKey board from 96boards.

Currently on the SD card is working on the HiKey, as such devices 0 and 2
will need to be disabled, for example by adding the following to
loader.conf:

hint.hisi_dwmmc.0.disabled=1
hint.hisi_dwmmc.2.disabled=1

Relnotes: yes (Hikey board booting)
Sponsored by: ABT Systems Ltd

Add an option to select which SoCs we are building for. It is intended to
be used with any SoC specific drivers, for example a ThunderX nic driver
would use something like the following in files.arm64:

arm64/cavium/thunder_nic.c optional soc_cavm_thunderx thndr_nic

Reviewed by: imp
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D3479

Add support for external PCIe (PEM) on Cavium's ThunderX

Reviewed by: jhb
Obtained from: Semihalf
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3257

Introduce support for internal PCIe for Cavium's ThunderX

This driver supports internal PCIe Root Complex on
Cavium ThunderX Pass 1.1 hardware.

Reviewed by: andrew, jhb
Obtained from: Semihalf
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3031

Load the stack in stack_save and stack_save_td. This uses the generalised
unwind_frame function to read each stack frame until either the pc or stack
are no longer withing the kernel's address space.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

Add support for uma_small_alloc and uma_small_free, and make use of these.
This is copied from the amd64 version with minor changes. These should be
merged into a single file as from a quick look there are other copies of
the same file in other parts of the tree.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

Implement stubs for ACPI PCI routines

ACPI driver requires special functions to be provided by machdep code.
Add temporary stubs to satisfy the compiler when both "pci" and "acpi"
are enabled in the kernel configuration file.

Reviewed by: andrew
Obtained from: Semihalf
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3028

Add support for SMP. This uses the FDT data to find the CPUs to start on,
and psci to start them. I expect ACPI support to be added later.

This has been tested on qemu with 2 cpus as that is the current value of
MAXCPUS. This is expected to be increased in the future as FreeBSD has
been tested on 48 cores on the Cavium ThunderX hardware.

Partially based on a patch from Robin Randhawa from ARM.

Approved by: ABT Systems Ltd
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3024

Introduce ITS support for ARM64

Add ARM ITS (Interrupt Translation Services) support required
to bring-up message signalled interrupts on some ARM64 platforms.

Obtained from: Semihalf
Sponsored by: The FreeBSD Foundation

Summary: Add missing files necessary to build with IPSEC and crypto

First cut of DTrace for AArch64.

Reviewed by: andrew, emaste
Sponsored by: ARM Limited
Differential Revision: https://reviews.freebsd.org/D2738

Rename ECAM PCI driver file.

Requested by: imp

Add generic ECAM PCI device driver found in Gem5 simulator.
Work based on Cavium Thunder PCIe driver by Semihalf.

Reviewed by: andrew, jhb
Sponsored by: HEIF5
Differential Revision: https://reviews.freebsd.org/D2386

Add basic support for ACPI. It splits out the nexus driver to two new
drivers, one for fdt, one for acpi. It then uses this to decide if it will
use fdt or acpi.

The GICv2 (interrupt controller) and Generic Timer drivers have been
updated to handle both cases.

As this is early code we still need FDT to find the kernel console, and
some parts are still missing, including PCI support.

Differential Revision: https://reviews.freebsd.org/D2463
Reviewed by: jhb, jkim, emaste
Obtained from: ABT Systems Ltd
Relnotes: Yes
Sponsored by: The FreeBSD Foundation

Consider sorting.

Split out db_unwind_frame() so it can be used by DTrace.

Sponsored by: ARM Ltd.
Differential Revision: https://reviews.freebsd.org/D2741

Add Performance Monitoring Counters support for AArch64.
Family-common and CPU-specific counters implemented.

Supported CPUs: ARM Cortex A53/57/72.

Reviewed by: andrew, bz, emaste, gnn, jhb
Sponsored by: ARM Limited
Differential Revision: https://reviews.freebsd.org/D2555

Add support for ARM GICv3 interrupt controller used in some ARM64 chips

GICv3 allows to distribute interrupts to more than 8 cores served by
the previous GIC revisions. GICv3 introduces additional logic in form
of Re-Distributors associated with particular CPUs to determine
the highest priority interrupts and manage PPIs and LPIs
(Locality-specific Peripheral Interrupts). Interrupts routing is
based on CPUs' affinity numbers. CPU interface was changed to be
accessible via CPU System Registers and this is the preferred
(and supported) method in this driver.

Obtained from: Semihalf
Reviewed by: andrew, emaste, ian, imp
Sponsored by: The FreeBSD Foundation

Port x86 busdma to ARM64

The x86 busdma subsystem allows using multiple implementations.
By default the classic bounce buffer approach is used, however
on systems with IOMMU it could be in runtime switched to more
efficient hardware accelerated implementation.

This commit adds ARM64 port of the x86 busdma framework and bounce
buffer backend. It is ready to use on IO coherent systems. If the
IO coherency cannot be guaranteed, the cache management operations have
to be added to this code in places marked by /* XXX ARM64TODO (...) */
comments. Also IOMMU support might be added by registering another
busdma implementation like it is already done on the x86.

Reviewed by: andrew, emaste
Obtained from: Semihalf
Sponsored by: The FreeBSD Foundation

Introduce ddb(4) support for ARM64

Obtained from: Semihalf
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation

Add psci for arm64. This is currently used when rebooting and powering off
in qemu.

Sponsored by: The FreeBSD Foundation

Bring in the start of the arm64 kernel.

This is only the minimum set of files needed to boot in qemu. As such it is
missing a few things.

The bus_dma code is currently only stub functions with a full implementation
from the development tree to follow.

The gic driver has been copied as the interrupt framework is different. It
is expected the two drivers will be merged by the arm intrng project,
however this will need to be imported into the tree and support for arm64
would need to be added.

This includes code developed by myself, SemiHalf, Ed Maste, and Robin
Randhawa from ARM. This has been funded by the FreeBSD Foundation, with
early development by myself in my spare time with assistance from Robin.

Differential Revision: https://reviews.freebsd.org/D2199
Reviewed by: emaste, imp
Relnotes: yes
Sponsored by: The FreeBSD Foundation