Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Mark the ThunderX and generic PCI drivers as cache-coherent when we know
this to be the case. This will mean we don't try and handle the cache in
bus_dmamap_sync when it is not needed.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6605

Handle PCI_RES_BUS on the generic and ThunderX PCIe drivers. This has been
tested on the Pass 1.1 and 2.0 ThunderX machines in the Netperf cluster.

Reviewed by: jhb
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D6453

Add a pcib interface for use by interrupt controllers that need to
translate the pci rid to a controller ID. The translation could be based
on the 'msi-map' OFW property, a similar ACPI option, or hard-coded for
hardware lacking the above options.

Reviewed by: wma
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

Push the logic to talk with the MSI/MSI-X interrupt controller to the FDT
attachment. This is where it will live when we import intrng as it will
need to look at either the msi-parent or msi-map FDT properties.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

Group the ThunderX PCIe PEM newbus methods to help find them.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

Pull out the MSI/MSI-X handling calls to simplify future intrng
integration.

Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation

Cleanup unnecessary semicolons from the kernel.

Found with devel/coccinelle.

Improve ThunderX PEM driver to work on pass2 revision

Things changed:
* do not allocate 4GB of SLI space, because it's the waste of
system resources. Allocate only small portions when needed.
* provide own implementation of activate_resource which performs
address translation between PCI bus and host PA address space.
This is temporary solution, should be replaced by bus_map_resource
once implemented.

Obtained from: Semihalf
Sponsored by: Cavium
Approved by: cognet (mentor)
Reviewed by: jhb
Differential revision: https://reviews.freebsd.org/D5294

Make pci_host_generic and thunderx_pci common
* provided OFW interface for pci_host_generic (for handling devices which are present in DTS under the PCI node)
* removed support for internal PCI from arm64/cavium
* cleaned up and made most of the code common

Obtained from: Semihalf
Sponsored by: Cavium
Approved by: cognet (mentor)
Reviewed by: zbb
Differential revision: https://reviews.freebsd.org/D5261

Introduce a RMAN_IS_DEFAULT_RANGE() macro, and use it.

This simplifies checking for default resource range for bus_alloc_resource(),
and improves readability.

This is part of, and related to, the migration of rman_res_t from u_long to
uintmax_t.

Discussed with: jhb
Suggested by: marcel

Support PEM that is not a PCI endpoint on ThunderX

Some chip revisions don't have their external PCIe buses
behind the internal bridge. Add support for FDT-configurable
PEMs but keep ability for PCIe enumeration.

Reviewed by: andrew, wma
Obtained from: Semihalf
Sponsored by: Cavium
Differential Revision: https://reviews.freebsd.org/D5285

Convert rman to use rman_res_t instead of u_long

Summary:
Migrate to using the semi-opaque type rman_res_t to specify rman resources. For
now, this is still compatible with u_long.

This is step one in migrating rman to use uintmax_t for resources instead of
u_long.

Going forward, this could feasibly be used to specify architecture-specific
definitions of resource ranges, rather than baking a specific integer type into
the API.

This change has been broken out to facilitate MFC'ing drivers back to 10 without
breaking ABI.

Reviewed By: jhb
Sponsored by: Alex Perez/Inertial Computing
Differential Revision: https://reviews.freebsd.org/D5075

Update ThunderX PCIe driver to fit new DTS layout

In recent EFI the DTS entries changed for PCIe controller.
This commit fixes internal PCIe, external is yet TBD.

Submitted by: Dominik Ermel <der@semihalf.com>
Obtained from: Semihalf
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D4976

Reduce the numbers of levels of indirection in arm64 pcib drivers by making
the MSI & MSI-X handler functions be directly callable from the driver
methods.

Sponsored by: ABT Systems Ltd

arm64 ThunderX PCIe workaround: enumerate only one slot for now

Otherwise an em(4) NIC is detected 32 times.

Submitted by: wma@semihalf.com
Obtained from: Semihalf
Differential Revision: https://reviews.freebsd.org/D3706

Add support for external PCIe (PEM) on Cavium's ThunderX

Reviewed by: jhb
Obtained from: Semihalf
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3257