busdma: tidy bus_dma_run_filter() functions

After removing filter functionality, the naming doesn't clearly
represent what the function does, so try to address this. Include some
code clarity and style improvements.

Create a common version in subr_busdma_bounce.c, used by most
implementations. powerpc still needs its own version of the function,
due to its dmat->iommu == NULL check.

No functional change intended.

Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D42896

busdma: remove parent tag tracking

Without filter functions, we do not need to keep track of tag ancestry.
All inheritance of the parent tag's parameters occurs when creating the
new child tag.

Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D42895

busdma: kill filter functionality internally

Address filter functions are unused, unsupported, and now rejected.
Simplify some busdma code by removing filter functionality completely.

Note that the chains of parent tags become useless, and will be cleaned
up in the next commit.

No functional change intended.

Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D42894

busdma: Prevent the use of filters with bus_dma_tag_create()

A deprecation notice was added to the bus_dma(9) man page by scottl@ in
September 2020 discouraging the use of filter functions. I've performed
an attentive check of all callers in the tree and everything that exists
today passes NULL for both filtfunc and filtarg. Thus, we should start
returning EINVAL if these arguments are non-NULL to prevent new usages
from popping up. Update the man page to be more clear about this.

The deprecation notice is present since at least 13.0-RELEASE, so this
is the appropriate step for the lifetime of 15, without actually
breaking the driver API. Stable branches will emit a warning instead.

This change enables the removal of a fair amount of unused complexity
across the various busdma implementations.

Reviewed by: jhb
MFC after: never
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D42852

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

arm64: Teach bus_dma on arm64 about NUMA

When allocating memory we should try to allocate from the NUMA node
closest to the device to reduce cross domain memory traffic. Teach the
arm64 bus_dma code to do this.

While here use mallocarray to guard against an unlikely integer
overflow.

Reviewed by: markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D42187

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

bus_dma: Deduplicate locking helper functions.

- Move busdma_lock_mutex to subr_bus_dma.c.

- Move _busdma_lock_dflt to subr_bus_dma.c. This function was named a
couple of different things previously. It is not a public API but
an internal helper used in place of a NULL pointer. The prototype
is in <sys/bus_dma.h> as not all backends include
<sys/bus_dma_internal.h>.

Reviewed by: kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D33694

busdma: Remove outdated comments about Giant.

MFC after: 2 weeks

Clean up the arm64 bus_dma_run_filter

- We can exit the loop as soon as the filter check passes.
- The alignment check has already passed so there is no need to also run
it here.

Sponsored by: Innovate UK

Ensure arm64 DMA alignment is passed from parents to children

This ensures the alignment check will take these alignments into account.

Sponsored by: Innovate UK

Refine the busdma template interface. Provide tools for filling in fields
that can be extended, but also ensure compile-time type checking. Refactor
common code out of arch-specific implementations. Move the mpr and mps
drivers to this new API. The template type remains visible to the consumer
so that it can be allocated on the stack, but should be considered opaque.

Introduce the concept of busdma tag templates. A template can be allocated
off the stack, initialized to default values, and then filled in with
driver-specific values, all without having to worry about the numerous
other fields in the tag. The resulting template is then passed into
busdma and the normal opaque tag object created. See the man page for
details on how to initialize a template.

Templates do not support tag filters. Filters have been broken for many
years, and only existed for an ancient make/model of hardware that had a
quirky DMA engine. Instead of breaking the ABI/API and changing the
arugment signature of bus_dma_tag_create() to remove the filter arguments,
templates allow us to ignore them, and also significantly reduce the
complexity of creating and managing tags.

Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D22906

Add support for NUMA domains to bus dma tags. This causes all memory
allocated with a tag to come from the specified domain if it meets the
other constraints provided by the tag. Automatically create a tag at
the root of each bus specifying the domain local to that bus if
available.

Reviewed by: jhb, kib
Tested by: pho
Sponsored by: Netflix, Dell/EMC Isilon
Differential Revision: https://reviews.freebsd.org/D13545

Clean up MD pollution of bus_dma.h:

--Remove special-case handling of sparc64 bus_dmamap* functions.
Replace with a more generic mechanism that allows MD busdma
implementations to generate inline mapping functions by
defining WANT_INLINE_DMAMAP in <machine/bus_dma.h>. This
is currently useful for sparc64, x86, and arm64, which all
implement non-load dmamap operations as simple wrappers
around map objects which may be bus- or device-specific.

--Remove NULL-checked bus_dmamap macros. Implement the
equivalent NULL checks in the inlined x86 implementation.
For non-x86 platforms, these checks are a minor pessimization
as those platforms do not currently allow NULL maps. NULL
maps were originally allowed on arm64, which appears to have
been the motivation behind adding arm[64]-specific barriers
to bus_dma.h, but that support was removed in r299463.

--Simplify the internal interface used by the bus_dmamap_load*
variants and move it to bus_dma_internal.h

--Fix some drivers that directly include sys/bus_dma.h
despite the recommendations of bus_dma(9)

Reviewed by: kib (previous revision), marius
Differential Revision: https://reviews.freebsd.org/D10729

Port x86 busdma to ARM64

The x86 busdma subsystem allows using multiple implementations.
By default the classic bounce buffer approach is used, however
on systems with IOMMU it could be in runtime switched to more
efficient hardware accelerated implementation.

This commit adds ARM64 port of the x86 busdma framework and bounce
buffer backend. It is ready to use on IO coherent systems. If the
IO coherency cannot be guaranteed, the cache management operations have
to be added to this code in places marked by /* XXX ARM64TODO (...) */
comments. Also IOMMU support might be added by registering another
busdma implementation like it is already done on the x86.

Reviewed by: andrew, emaste
Obtained from: Semihalf
Sponsored by: The FreeBSD Foundation

Bring in the start of the arm64 kernel.

This is only the minimum set of files needed to boot in qemu. As such it is
missing a few things.

The bus_dma code is currently only stub functions with a full implementation
from the development tree to follow.

The gic driver has been copied as the interrupt framework is different. It
is expected the two drivers will be merged by the arm intrng project,
however this will need to be imported into the tree and support for arm64
would need to be added.

This includes code developed by myself, SemiHalf, Ed Maste, and Robin
Randhawa from ARM. This has been funded by the FreeBSD Foundation, with
early development by myself in my spare time with assistance from Robin.

Differential Revision: https://reviews.freebsd.org/D2199
Reviewed by: emaste, imp
Relnotes: yes
Sponsored by: The FreeBSD Foundation