Implement Multiple Message MSI support on arm64. As on amd64 this is
experimental code to assis qwx(4) development. Currently this only works
on systems that use agintcmsi(4) as the MSI controller combined with the
dwpcie(4) Hots/PCIe bridge.

ok patrick@

Skip non-MSI interrupt controllers when looking for one matching the ITS id
given in the IORT node. Using a non-MSI interrupt controller here will
crash as ic->ic_establish_msi will be NULL.

tested by phessler@
ok phessler@ patrick@

Use IORT ITS nodes to find the right ITS instance to use when establishing
interrupts. This makes MSI/MSI-X work on platforms like the Ampere Altra
which have an ITS instance for each PCI domain.

also tested by cheloha@
ok kettenis@ patrick@

Resolve namerefs in packages. Fixes legacy interrupts on machines that use
PNP0C0F PCI interrupt link devices.

ok patrick@

Support SMMUv3 IORT nodes as well in the midlayers. This allows IOMMU
mappings to reach a future SMMUv3 implementation.

ok kettenis@, mlarkin@

MSIs on the x13s are routed through both a "normal" SMMU and a "v3" SMMU.
So handle this case in acpipci(4) and kill the hack to disable MSIs.

ok patrick@, mlarkin@, deraadt@

Work around MSI and INTx issues on Qualcomm SC8280XP. This makes the NVMe
work on the Lenovo x13s.

ok dv@

The x13s uses Memory32Fixed() to describe the windows forwarded by the
host bridges. Add support for this.

ok mlarkin@, patrick@

Add support for interrupts represented by ACPI PCI Interrupt Link Devices.
This makes PCI interrupts work on QEMU's SBSA target.

ok patrick@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

Don't advertise MSI support if we don't have an MSI interrupt controller.

ok patrick@

Only check whether we have an MSI interrupt controller when we try to
establish an MSI or MSI-X interrupt. Fixes establishing legacy INTx
interrupts on machines without a (usable) MSI interrupt controller.

ok patrick@

While it seems like we can choose any I/O virtual address for peripheral
devices, this isn't really the case. It depends on the bus topology of
how devices are connected. In the case of PCIe, devices are assigned
addresses (in PCI BARs) from the PCI address spaces. Now if we take an
address from one of these address spaces for our IOVA, transfers from
from a PCI device to that address will terminate inside of the PCI bus.
This is because from the PCI buses' point-of-view, the address we chose
is part of its address space. To make sure we don't allocate addresses
from there, reserve the PCI addresses in the IOVA.

Note that smmu(4) currently gives each device its own IOVA. So the PCI
addresses will be reserved only in IOVA from PCI devices, and only the
addresses concerning the PCI bus it is connected to will be reserved.
All other devices behind an smmu(4) will not have any changes to their
IOVA.

ok kettenis@

Rename some MD structs by giving them an architecture-neutral name in
preparation for sharing PCIe host bridge drivers between arm64 and riscv64.

ok mpi@, mlarkin@, patrick@

Load MSI pages through bus_dma(9). Our interrupt controllers for MSIs
typically pass the physical address, however retrieved, to our PCIe
controller code. This physical address can in practise be directly
given to the PCIe, but it is not a given that the CPU and the PCIe
controller are able to use the same physical addresses.

This is even more obvious with an smmu(4) inbetween, which can change
the world view by introducing I/O virtual addresses. Hence for this
it is indeed necessary to map those pages, which thanks to integration
with bus_dma(9) works easily.

For this we remember the PCI devices' DMA tag in the interrupt handle
during the MSI map, so that we can use the smmu(4)-hooked DMA tag to
load the physical address.

While some systems might prefer to implement "trapping" pages for MSIs,
to make sure devices cannot trigger other devices' interrupts, we only
make sure the whole page is mapped.

Having the IOMMU create a mapping for each MSI is a bit wasteful, but
for now it's the simplest way to implement it.

Discussed with and ok kettenis@

Change API of acpiiort(4). It was written as a hook before, taking the
PCI attach args and replacing the DMA tag inside. Our other IOMMU API
though takes a DMA tag and returns the old one or a new one. To have
acpiiort(4) integrate better with non-PCI ACPI devices, change the API
so that it is more similar to the other API. This also makes the code
easier to understand.

ok kettenis@

Have acpipci(4) look for a matching SMMU in the IORT.

ok kettenis@

Add some infrastructure in the PCI chipset tag for pci_probe_device_hook()
so that we can provide IOMMU-hooked bus DMA tags for each PCI device.

ok kettenis@

Split the IORT struct into two, as the current version not only contained
the generic IORT node information but also the Root Complex's attributes.

ok kettenis@

Move IO Remapping Table (IORT) struct defines to the common ACPI header
so that it can be used by more drivers.

ok kettenis@

Implement pci_intr_disestablish(9) for acpicpi(4) on arm64.

ok patrick@

Implement address translation for bus_space_mmap(9).

ok patrick@

Re-work intr_barrier(9) on arm64 to remove layer violation. So far we
have stored the struct cpu_info * in the wrapper around the interrupt
handler cookie, but since we can have a few layers inbetween, this does
not seem very nice. Instead have each and every interrupt controller
provide a barrier function. This means that intr_barrier(9) will in the
end be executed by the interrupt controller that actually wired the pin
to a core. And that's the only place where the information is stored.

ok kettenis@

Store struct cpu_info * in arm64's interrupt wrap. intr_barrier() can
already assume every cookie is wrapped and simply retrieve the pointer
from it. It's a bit of a layer violation though, since only the intc
should actually store that kind of information. This is good enough for
now, but I'm already cooking up a diff to resolve this.

ok dlg@

To be able to have intr_barrier() on arm64, we need to be able to
somehow gain access to the struct cpu_info * used to establish the
interrupt. One possibility is to store the pointer in the cookie
returned by the establish methods. A better way would be to ask
the interrupt controller directly to do barrier.

This means that all external facing interrupt establish functions
need to wrap the cookie in a common way. We already do this for
FDT-based interrupts. Also most PCI controllers already return
the cookie from the FDT API, which is already wrapped. So arm64's
acpi_intr_establish() and acpipci(4) now need to explicitly wrap
it, since they call ic->ic_establish directly, which does not wrap.

ok dlg@

Implement pci_intr_establish_cpu() on arm64 and armv7. The function pointer
in the chipset tag for establishing interrupts now takes a struct cpu_info *.
The normal pci_intr_establish() macro passes NULL as ci, which indicates that
the primary CPU is to be used.

The PCI controller drivers can then simply pass the ci on to our arm64/armv7
interrupt establish "framework".

Prompted by dlg@
ok kettenis@

Extend the interrupt API on arm64 and armv7 to be able to pass around
a struct cpu_info *. From a driver point of view the fdt_intr_establish_*
API now also exist same functions with a *_cpu suffix. Internally the
"old" functions now call their *_cpu counterparts, passing NULL as ci.
NULL will be interpreted as primary CPU in the interrupt controller code.

The internal framework for interrupt controllers has been changed so that
the establish methods provided by an interrupt controller function always
takes a struct cpu_info *.

Some drivers, like imxgpio(4) and rkgpio(4), only have a single interrupt
line for multiple pins. On those we simply disallow trying to establish
an interrupt on a non-primary CPU, returning NULL.

Since we do not have MP yet on armv7, all armv7 interrupt controllers do
return NULL if an attempt is made to establish an interrupt on a different
CPU. That said, so far there's no way this can happen. If we ever gain
MP support, this is a reminder that the interrupt controller drivers have
to be adjusted.

Prompted by dlg@
ok kettenis@

Fix small bug in parsing the IORT tables. Mapping entries specify the number
of IDs in a range minus one.

ok patrick@

Get bus number from _CRS.

ok sthen@, deraadt@

Don't check _TTP for io windows.

ok patrick@, jsg@

Fix mapping of MSI sideband data when there are SMMU's present.

Seems the Amppere eMAG has a silicon big where the number of writable bits
of the ICC_PMR_EL1 registers varies with the value being written. Change the
value we write to probe the number of writable bits to a value that yields
the desired result.

Suggested by drahn@

ok patrick@

Change pci_intr_handle_t into a struct and replace duplicated code that
implements mapping of MSI and MSI-X interrupts with new generic functions.
Fixes a use-after-free in sone PCI device drivers that call pci_intr_string(9)
after pci_intr_establish(9).

ok deraadt@

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Pass extent for prefetchable mmio. Since there is no distinction between
prefetchable and "normal" mmio at the host bridge level we can simply pass
the same extent.

ok patrick@

Add support for multiple PCI segments. Only really implemented for arm64
for now as amd64/i386 firmware still caters for legacy OSes that only
support a single PCI segment.

ok patrick@

Make legacy interrupts work in more cases.

Use IORT table to map requester ID into MSI sideband data.

Pass PCIe requester ID as sideband data here as well.

Make use of PCI_FLAGS_MSI_ENABLED such that drivers for hardware with broken
MSI support can selectively disable the use of MSI.

Make legacy interrupts work in acpipci(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

Skip non-MSI interrupt controllers when looking for one matching the ITS id
given in the IORT node. Using a non-MSI interrupt controller here will
crash as ic->ic_establish_msi will be NULL.

tested by phessler@
ok phessler@ patrick@

Use IORT ITS nodes to find the right ITS instance to use when establishing
interrupts. This makes MSI/MSI-X work on platforms like the Ampere Altra
which have an ITS instance for each PCI domain.

also tested by cheloha@
ok kettenis@ patrick@

Resolve namerefs in packages. Fixes legacy interrupts on machines that use
PNP0C0F PCI interrupt link devices.

ok patrick@

Support SMMUv3 IORT nodes as well in the midlayers. This allows IOMMU
mappings to reach a future SMMUv3 implementation.

ok kettenis@, mlarkin@

MSIs on the x13s are routed through both a "normal" SMMU and a "v3" SMMU.
So handle this case in acpipci(4) and kill the hack to disable MSIs.

ok patrick@, mlarkin@, deraadt@

Work around MSI and INTx issues on Qualcomm SC8280XP. This makes the NVMe
work on the Lenovo x13s.

ok dv@

The x13s uses Memory32Fixed() to describe the windows forwarded by the
host bridges. Add support for this.

ok mlarkin@, patrick@

Add support for interrupts represented by ACPI PCI Interrupt Link Devices.
This makes PCI interrupts work on QEMU's SBSA target.

ok patrick@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

Don't advertise MSI support if we don't have an MSI interrupt controller.

ok patrick@

Only check whether we have an MSI interrupt controller when we try to
establish an MSI or MSI-X interrupt. Fixes establishing legacy INTx
interrupts on machines without a (usable) MSI interrupt controller.

ok patrick@

While it seems like we can choose any I/O virtual address for peripheral
devices, this isn't really the case. It depends on the bus topology of
how devices are connected. In the case of PCIe, devices are assigned
addresses (in PCI BARs) from the PCI address spaces. Now if we take an
address from one of these address spaces for our IOVA, transfers from
from a PCI device to that address will terminate inside of the PCI bus.
This is because from the PCI buses' point-of-view, the address we chose
is part of its address space. To make sure we don't allocate addresses
from there, reserve the PCI addresses in the IOVA.

Note that smmu(4) currently gives each device its own IOVA. So the PCI
addresses will be reserved only in IOVA from PCI devices, and only the
addresses concerning the PCI bus it is connected to will be reserved.
All other devices behind an smmu(4) will not have any changes to their
IOVA.

ok kettenis@

Rename some MD structs by giving them an architecture-neutral name in
preparation for sharing PCIe host bridge drivers between arm64 and riscv64.

ok mpi@, mlarkin@, patrick@

Load MSI pages through bus_dma(9). Our interrupt controllers for MSIs
typically pass the physical address, however retrieved, to our PCIe
controller code. This physical address can in practise be directly
given to the PCIe, but it is not a given that the CPU and the PCIe
controller are able to use the same physical addresses.

This is even more obvious with an smmu(4) inbetween, which can change
the world view by introducing I/O virtual addresses. Hence for this
it is indeed necessary to map those pages, which thanks to integration
with bus_dma(9) works easily.

For this we remember the PCI devices' DMA tag in the interrupt handle
during the MSI map, so that we can use the smmu(4)-hooked DMA tag to
load the physical address.

While some systems might prefer to implement "trapping" pages for MSIs,
to make sure devices cannot trigger other devices' interrupts, we only
make sure the whole page is mapped.

Having the IOMMU create a mapping for each MSI is a bit wasteful, but
for now it's the simplest way to implement it.

Discussed with and ok kettenis@

Change API of acpiiort(4). It was written as a hook before, taking the
PCI attach args and replacing the DMA tag inside. Our other IOMMU API
though takes a DMA tag and returns the old one or a new one. To have
acpiiort(4) integrate better with non-PCI ACPI devices, change the API
so that it is more similar to the other API. This also makes the code
easier to understand.

ok kettenis@

Have acpipci(4) look for a matching SMMU in the IORT.

ok kettenis@

Add some infrastructure in the PCI chipset tag for pci_probe_device_hook()
so that we can provide IOMMU-hooked bus DMA tags for each PCI device.

ok kettenis@

Split the IORT struct into two, as the current version not only contained
the generic IORT node information but also the Root Complex's attributes.

ok kettenis@

Move IO Remapping Table (IORT) struct defines to the common ACPI header
so that it can be used by more drivers.

ok kettenis@

Implement pci_intr_disestablish(9) for acpicpi(4) on arm64.

ok patrick@

Implement address translation for bus_space_mmap(9).

ok patrick@

Re-work intr_barrier(9) on arm64 to remove layer violation. So far we
have stored the struct cpu_info * in the wrapper around the interrupt
handler cookie, but since we can have a few layers inbetween, this does
not seem very nice. Instead have each and every interrupt controller
provide a barrier function. This means that intr_barrier(9) will in the
end be executed by the interrupt controller that actually wired the pin
to a core. And that's the only place where the information is stored.

ok kettenis@

Store struct cpu_info * in arm64's interrupt wrap. intr_barrier() can
already assume every cookie is wrapped and simply retrieve the pointer
from it. It's a bit of a layer violation though, since only the intc
should actually store that kind of information. This is good enough for
now, but I'm already cooking up a diff to resolve this.

ok dlg@

To be able to have intr_barrier() on arm64, we need to be able to
somehow gain access to the struct cpu_info * used to establish the
interrupt. One possibility is to store the pointer in the cookie
returned by the establish methods. A better way would be to ask
the interrupt controller directly to do barrier.

This means that all external facing interrupt establish functions
need to wrap the cookie in a common way. We already do this for
FDT-based interrupts. Also most PCI controllers already return
the cookie from the FDT API, which is already wrapped. So arm64's
acpi_intr_establish() and acpipci(4) now need to explicitly wrap
it, since they call ic->ic_establish directly, which does not wrap.

ok dlg@

Implement pci_intr_establish_cpu() on arm64 and armv7. The function pointer
in the chipset tag for establishing interrupts now takes a struct cpu_info *.
The normal pci_intr_establish() macro passes NULL as ci, which indicates that
the primary CPU is to be used.

The PCI controller drivers can then simply pass the ci on to our arm64/armv7
interrupt establish "framework".

Prompted by dlg@
ok kettenis@

Extend the interrupt API on arm64 and armv7 to be able to pass around
a struct cpu_info *. From a driver point of view the fdt_intr_establish_*
API now also exist same functions with a *_cpu suffix. Internally the
"old" functions now call their *_cpu counterparts, passing NULL as ci.
NULL will be interpreted as primary CPU in the interrupt controller code.

The internal framework for interrupt controllers has been changed so that
the establish methods provided by an interrupt controller function always
takes a struct cpu_info *.

Some drivers, like imxgpio(4) and rkgpio(4), only have a single interrupt
line for multiple pins. On those we simply disallow trying to establish
an interrupt on a non-primary CPU, returning NULL.

Since we do not have MP yet on armv7, all armv7 interrupt controllers do
return NULL if an attempt is made to establish an interrupt on a different
CPU. That said, so far there's no way this can happen. If we ever gain
MP support, this is a reminder that the interrupt controller drivers have
to be adjusted.

Prompted by dlg@
ok kettenis@

Fix small bug in parsing the IORT tables. Mapping entries specify the number
of IDs in a range minus one.

ok patrick@

Get bus number from _CRS.

ok sthen@, deraadt@

Don't check _TTP for io windows.

ok patrick@, jsg@

Fix mapping of MSI sideband data when there are SMMU's present.

Seems the Amppere eMAG has a silicon big where the number of writable bits
of the ICC_PMR_EL1 registers varies with the value being written. Change the
value we write to probe the number of writable bits to a value that yields
the desired result.

Suggested by drahn@

ok patrick@

Change pci_intr_handle_t into a struct and replace duplicated code that
implements mapping of MSI and MSI-X interrupts with new generic functions.
Fixes a use-after-free in sone PCI device drivers that call pci_intr_string(9)
after pci_intr_establish(9).

ok deraadt@

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Pass extent for prefetchable mmio. Since there is no distinction between
prefetchable and "normal" mmio at the host bridge level we can simply pass
the same extent.

ok patrick@

Add support for multiple PCI segments. Only really implemented for arm64
for now as amd64/i386 firmware still caters for legacy OSes that only
support a single PCI segment.

ok patrick@

Make legacy interrupts work in more cases.

Use IORT table to map requester ID into MSI sideband data.

Pass PCIe requester ID as sideband data here as well.

Make use of PCI_FLAGS_MSI_ENABLED such that drivers for hardware with broken
MSI support can selectively disable the use of MSI.

Make legacy interrupts work in acpipci(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

Use IORT ITS nodes to find the right ITS instance to use when establishing
interrupts. This makes MSI/MSI-X work on platforms like the Ampere Altra
which have an ITS instance for each PCI domain.

also tested by cheloha@
ok kettenis@ patrick@

Resolve namerefs in packages. Fixes legacy interrupts on machines that use
PNP0C0F PCI interrupt link devices.

ok patrick@

Support SMMUv3 IORT nodes as well in the midlayers. This allows IOMMU
mappings to reach a future SMMUv3 implementation.

ok kettenis@, mlarkin@

MSIs on the x13s are routed through both a "normal" SMMU and a "v3" SMMU.
So handle this case in acpipci(4) and kill the hack to disable MSIs.

ok patrick@, mlarkin@, deraadt@

Work around MSI and INTx issues on Qualcomm SC8280XP. This makes the NVMe
work on the Lenovo x13s.

ok dv@

The x13s uses Memory32Fixed() to describe the windows forwarded by the
host bridges. Add support for this.

ok mlarkin@, patrick@

Add support for interrupts represented by ACPI PCI Interrupt Link Devices.
This makes PCI interrupts work on QEMU's SBSA target.

ok patrick@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

Don't advertise MSI support if we don't have an MSI interrupt controller.

ok patrick@

Only check whether we have an MSI interrupt controller when we try to
establish an MSI or MSI-X interrupt. Fixes establishing legacy INTx
interrupts on machines without a (usable) MSI interrupt controller.

ok patrick@

While it seems like we can choose any I/O virtual address for peripheral
devices, this isn't really the case. It depends on the bus topology of
how devices are connected. In the case of PCIe, devices are assigned
addresses (in PCI BARs) from the PCI address spaces. Now if we take an
address from one of these address spaces for our IOVA, transfers from
from a PCI device to that address will terminate inside of the PCI bus.
This is because from the PCI buses' point-of-view, the address we chose
is part of its address space. To make sure we don't allocate addresses
from there, reserve the PCI addresses in the IOVA.

Note that smmu(4) currently gives each device its own IOVA. So the PCI
addresses will be reserved only in IOVA from PCI devices, and only the
addresses concerning the PCI bus it is connected to will be reserved.
All other devices behind an smmu(4) will not have any changes to their
IOVA.

ok kettenis@

Rename some MD structs by giving them an architecture-neutral name in
preparation for sharing PCIe host bridge drivers between arm64 and riscv64.

ok mpi@, mlarkin@, patrick@

Load MSI pages through bus_dma(9). Our interrupt controllers for MSIs
typically pass the physical address, however retrieved, to our PCIe
controller code. This physical address can in practise be directly
given to the PCIe, but it is not a given that the CPU and the PCIe
controller are able to use the same physical addresses.

This is even more obvious with an smmu(4) inbetween, which can change
the world view by introducing I/O virtual addresses. Hence for this
it is indeed necessary to map those pages, which thanks to integration
with bus_dma(9) works easily.

For this we remember the PCI devices' DMA tag in the interrupt handle
during the MSI map, so that we can use the smmu(4)-hooked DMA tag to
load the physical address.

While some systems might prefer to implement "trapping" pages for MSIs,
to make sure devices cannot trigger other devices' interrupts, we only
make sure the whole page is mapped.

Having the IOMMU create a mapping for each MSI is a bit wasteful, but
for now it's the simplest way to implement it.

Discussed with and ok kettenis@

Change API of acpiiort(4). It was written as a hook before, taking the
PCI attach args and replacing the DMA tag inside. Our other IOMMU API
though takes a DMA tag and returns the old one or a new one. To have
acpiiort(4) integrate better with non-PCI ACPI devices, change the API
so that it is more similar to the other API. This also makes the code
easier to understand.

ok kettenis@

Have acpipci(4) look for a matching SMMU in the IORT.

ok kettenis@

Add some infrastructure in the PCI chipset tag for pci_probe_device_hook()
so that we can provide IOMMU-hooked bus DMA tags for each PCI device.

ok kettenis@

Split the IORT struct into two, as the current version not only contained
the generic IORT node information but also the Root Complex's attributes.

ok kettenis@

Move IO Remapping Table (IORT) struct defines to the common ACPI header
so that it can be used by more drivers.

ok kettenis@

Implement pci_intr_disestablish(9) for acpicpi(4) on arm64.

ok patrick@

Implement address translation for bus_space_mmap(9).

ok patrick@

Re-work intr_barrier(9) on arm64 to remove layer violation. So far we
have stored the struct cpu_info * in the wrapper around the interrupt
handler cookie, but since we can have a few layers inbetween, this does
not seem very nice. Instead have each and every interrupt controller
provide a barrier function. This means that intr_barrier(9) will in the
end be executed by the interrupt controller that actually wired the pin
to a core. And that's the only place where the information is stored.

ok kettenis@

Store struct cpu_info * in arm64's interrupt wrap. intr_barrier() can
already assume every cookie is wrapped and simply retrieve the pointer
from it. It's a bit of a layer violation though, since only the intc
should actually store that kind of information. This is good enough for
now, but I'm already cooking up a diff to resolve this.

ok dlg@

To be able to have intr_barrier() on arm64, we need to be able to
somehow gain access to the struct cpu_info * used to establish the
interrupt. One possibility is to store the pointer in the cookie
returned by the establish methods. A better way would be to ask
the interrupt controller directly to do barrier.

This means that all external facing interrupt establish functions
need to wrap the cookie in a common way. We already do this for
FDT-based interrupts. Also most PCI controllers already return
the cookie from the FDT API, which is already wrapped. So arm64's
acpi_intr_establish() and acpipci(4) now need to explicitly wrap
it, since they call ic->ic_establish directly, which does not wrap.

ok dlg@

Implement pci_intr_establish_cpu() on arm64 and armv7. The function pointer
in the chipset tag for establishing interrupts now takes a struct cpu_info *.
The normal pci_intr_establish() macro passes NULL as ci, which indicates that
the primary CPU is to be used.

The PCI controller drivers can then simply pass the ci on to our arm64/armv7
interrupt establish "framework".

Prompted by dlg@
ok kettenis@

Extend the interrupt API on arm64 and armv7 to be able to pass around
a struct cpu_info *. From a driver point of view the fdt_intr_establish_*
API now also exist same functions with a *_cpu suffix. Internally the
"old" functions now call their *_cpu counterparts, passing NULL as ci.
NULL will be interpreted as primary CPU in the interrupt controller code.

The internal framework for interrupt controllers has been changed so that
the establish methods provided by an interrupt controller function always
takes a struct cpu_info *.

Some drivers, like imxgpio(4) and rkgpio(4), only have a single interrupt
line for multiple pins. On those we simply disallow trying to establish
an interrupt on a non-primary CPU, returning NULL.

Since we do not have MP yet on armv7, all armv7 interrupt controllers do
return NULL if an attempt is made to establish an interrupt on a different
CPU. That said, so far there's no way this can happen. If we ever gain
MP support, this is a reminder that the interrupt controller drivers have
to be adjusted.

Prompted by dlg@
ok kettenis@

Fix small bug in parsing the IORT tables. Mapping entries specify the number
of IDs in a range minus one.

ok patrick@

Get bus number from _CRS.

ok sthen@, deraadt@

Don't check _TTP for io windows.

ok patrick@, jsg@

Fix mapping of MSI sideband data when there are SMMU's present.

Seems the Amppere eMAG has a silicon big where the number of writable bits
of the ICC_PMR_EL1 registers varies with the value being written. Change the
value we write to probe the number of writable bits to a value that yields
the desired result.

Suggested by drahn@

ok patrick@

Change pci_intr_handle_t into a struct and replace duplicated code that
implements mapping of MSI and MSI-X interrupts with new generic functions.
Fixes a use-after-free in sone PCI device drivers that call pci_intr_string(9)
after pci_intr_establish(9).

ok deraadt@

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Pass extent for prefetchable mmio. Since there is no distinction between
prefetchable and "normal" mmio at the host bridge level we can simply pass
the same extent.

ok patrick@

Add support for multiple PCI segments. Only really implemented for arm64
for now as amd64/i386 firmware still caters for legacy OSes that only
support a single PCI segment.

ok patrick@

Make legacy interrupts work in more cases.

Use IORT table to map requester ID into MSI sideband data.

Pass PCIe requester ID as sideband data here as well.

Make use of PCI_FLAGS_MSI_ENABLED such that drivers for hardware with broken
MSI support can selectively disable the use of MSI.

Make legacy interrupts work in acpipci(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

Resolve namerefs in packages. Fixes legacy interrupts on machines that use
PNP0C0F PCI interrupt link devices.

ok patrick@

Support SMMUv3 IORT nodes as well in the midlayers. This allows IOMMU
mappings to reach a future SMMUv3 implementation.

ok kettenis@, mlarkin@

MSIs on the x13s are routed through both a "normal" SMMU and a "v3" SMMU.
So handle this case in acpipci(4) and kill the hack to disable MSIs.

ok patrick@, mlarkin@, deraadt@

Work around MSI and INTx issues on Qualcomm SC8280XP. This makes the NVMe
work on the Lenovo x13s.

ok dv@

The x13s uses Memory32Fixed() to describe the windows forwarded by the
host bridges. Add support for this.

ok mlarkin@, patrick@

Add support for interrupts represented by ACPI PCI Interrupt Link Devices.
This makes PCI interrupts work on QEMU's SBSA target.

ok patrick@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

Don't advertise MSI support if we don't have an MSI interrupt controller.

ok patrick@

Only check whether we have an MSI interrupt controller when we try to
establish an MSI or MSI-X interrupt. Fixes establishing legacy INTx
interrupts on machines without a (usable) MSI interrupt controller.

ok patrick@

While it seems like we can choose any I/O virtual address for peripheral
devices, this isn't really the case. It depends on the bus topology of
how devices are connected. In the case of PCIe, devices are assigned
addresses (in PCI BARs) from the PCI address spaces. Now if we take an
address from one of these address spaces for our IOVA, transfers from
from a PCI device to that address will terminate inside of the PCI bus.
This is because from the PCI buses' point-of-view, the address we chose
is part of its address space. To make sure we don't allocate addresses
from there, reserve the PCI addresses in the IOVA.

Note that smmu(4) currently gives each device its own IOVA. So the PCI
addresses will be reserved only in IOVA from PCI devices, and only the
addresses concerning the PCI bus it is connected to will be reserved.
All other devices behind an smmu(4) will not have any changes to their
IOVA.

ok kettenis@

Rename some MD structs by giving them an architecture-neutral name in
preparation for sharing PCIe host bridge drivers between arm64 and riscv64.

ok mpi@, mlarkin@, patrick@

Load MSI pages through bus_dma(9). Our interrupt controllers for MSIs
typically pass the physical address, however retrieved, to our PCIe
controller code. This physical address can in practise be directly
given to the PCIe, but it is not a given that the CPU and the PCIe
controller are able to use the same physical addresses.

This is even more obvious with an smmu(4) inbetween, which can change
the world view by introducing I/O virtual addresses. Hence for this
it is indeed necessary to map those pages, which thanks to integration
with bus_dma(9) works easily.

For this we remember the PCI devices' DMA tag in the interrupt handle
during the MSI map, so that we can use the smmu(4)-hooked DMA tag to
load the physical address.

While some systems might prefer to implement "trapping" pages for MSIs,
to make sure devices cannot trigger other devices' interrupts, we only
make sure the whole page is mapped.

Having the IOMMU create a mapping for each MSI is a bit wasteful, but
for now it's the simplest way to implement it.

Discussed with and ok kettenis@

Change API of acpiiort(4). It was written as a hook before, taking the
PCI attach args and replacing the DMA tag inside. Our other IOMMU API
though takes a DMA tag and returns the old one or a new one. To have
acpiiort(4) integrate better with non-PCI ACPI devices, change the API
so that it is more similar to the other API. This also makes the code
easier to understand.

ok kettenis@

Have acpipci(4) look for a matching SMMU in the IORT.

ok kettenis@

Add some infrastructure in the PCI chipset tag for pci_probe_device_hook()
so that we can provide IOMMU-hooked bus DMA tags for each PCI device.

ok kettenis@

Split the IORT struct into two, as the current version not only contained
the generic IORT node information but also the Root Complex's attributes.

ok kettenis@

Move IO Remapping Table (IORT) struct defines to the common ACPI header
so that it can be used by more drivers.

ok kettenis@

Implement pci_intr_disestablish(9) for acpicpi(4) on arm64.

ok patrick@

Implement address translation for bus_space_mmap(9).

ok patrick@

Re-work intr_barrier(9) on arm64 to remove layer violation. So far we
have stored the struct cpu_info * in the wrapper around the interrupt
handler cookie, but since we can have a few layers inbetween, this does
not seem very nice. Instead have each and every interrupt controller
provide a barrier function. This means that intr_barrier(9) will in the
end be executed by the interrupt controller that actually wired the pin
to a core. And that's the only place where the information is stored.

ok kettenis@

Store struct cpu_info * in arm64's interrupt wrap. intr_barrier() can
already assume every cookie is wrapped and simply retrieve the pointer
from it. It's a bit of a layer violation though, since only the intc
should actually store that kind of information. This is good enough for
now, but I'm already cooking up a diff to resolve this.

ok dlg@

To be able to have intr_barrier() on arm64, we need to be able to
somehow gain access to the struct cpu_info * used to establish the
interrupt. One possibility is to store the pointer in the cookie
returned by the establish methods. A better way would be to ask
the interrupt controller directly to do barrier.

This means that all external facing interrupt establish functions
need to wrap the cookie in a common way. We already do this for
FDT-based interrupts. Also most PCI controllers already return
the cookie from the FDT API, which is already wrapped. So arm64's
acpi_intr_establish() and acpipci(4) now need to explicitly wrap
it, since they call ic->ic_establish directly, which does not wrap.

ok dlg@

Implement pci_intr_establish_cpu() on arm64 and armv7. The function pointer
in the chipset tag for establishing interrupts now takes a struct cpu_info *.
The normal pci_intr_establish() macro passes NULL as ci, which indicates that
the primary CPU is to be used.

The PCI controller drivers can then simply pass the ci on to our arm64/armv7
interrupt establish "framework".

Prompted by dlg@
ok kettenis@

Extend the interrupt API on arm64 and armv7 to be able to pass around
a struct cpu_info *. From a driver point of view the fdt_intr_establish_*
API now also exist same functions with a *_cpu suffix. Internally the
"old" functions now call their *_cpu counterparts, passing NULL as ci.
NULL will be interpreted as primary CPU in the interrupt controller code.

The internal framework for interrupt controllers has been changed so that
the establish methods provided by an interrupt controller function always
takes a struct cpu_info *.

Some drivers, like imxgpio(4) and rkgpio(4), only have a single interrupt
line for multiple pins. On those we simply disallow trying to establish
an interrupt on a non-primary CPU, returning NULL.

Since we do not have MP yet on armv7, all armv7 interrupt controllers do
return NULL if an attempt is made to establish an interrupt on a different
CPU. That said, so far there's no way this can happen. If we ever gain
MP support, this is a reminder that the interrupt controller drivers have
to be adjusted.

Prompted by dlg@
ok kettenis@

Fix small bug in parsing the IORT tables. Mapping entries specify the number
of IDs in a range minus one.

ok patrick@

Get bus number from _CRS.

ok sthen@, deraadt@

Don't check _TTP for io windows.

ok patrick@, jsg@

Fix mapping of MSI sideband data when there are SMMU's present.

Seems the Amppere eMAG has a silicon big where the number of writable bits
of the ICC_PMR_EL1 registers varies with the value being written. Change the
value we write to probe the number of writable bits to a value that yields
the desired result.

Suggested by drahn@

ok patrick@

Change pci_intr_handle_t into a struct and replace duplicated code that
implements mapping of MSI and MSI-X interrupts with new generic functions.
Fixes a use-after-free in sone PCI device drivers that call pci_intr_string(9)
after pci_intr_establish(9).

ok deraadt@

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Pass extent for prefetchable mmio. Since there is no distinction between
prefetchable and "normal" mmio at the host bridge level we can simply pass
the same extent.

ok patrick@

Add support for multiple PCI segments. Only really implemented for arm64
for now as amd64/i386 firmware still caters for legacy OSes that only
support a single PCI segment.

ok patrick@

Make legacy interrupts work in more cases.

Use IORT table to map requester ID into MSI sideband data.

Pass PCIe requester ID as sideband data here as well.

Make use of PCI_FLAGS_MSI_ENABLED such that drivers for hardware with broken
MSI support can selectively disable the use of MSI.

Make legacy interrupts work in acpipci(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

Support SMMUv3 IORT nodes as well in the midlayers. This allows IOMMU
mappings to reach a future SMMUv3 implementation.

ok kettenis@, mlarkin@

MSIs on the x13s are routed through both a "normal" SMMU and a "v3" SMMU.
So handle this case in acpipci(4) and kill the hack to disable MSIs.

ok patrick@, mlarkin@, deraadt@

Work around MSI and INTx issues on Qualcomm SC8280XP. This makes the NVMe
work on the Lenovo x13s.

ok dv@

The x13s uses Memory32Fixed() to describe the windows forwarded by the
host bridges. Add support for this.

ok mlarkin@, patrick@

Add support for interrupts represented by ACPI PCI Interrupt Link Devices.
This makes PCI interrupts work on QEMU's SBSA target.

ok patrick@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

Don't advertise MSI support if we don't have an MSI interrupt controller.

ok patrick@

Only check whether we have an MSI interrupt controller when we try to
establish an MSI or MSI-X interrupt. Fixes establishing legacy INTx
interrupts on machines without a (usable) MSI interrupt controller.

ok patrick@

While it seems like we can choose any I/O virtual address for peripheral
devices, this isn't really the case. It depends on the bus topology of
how devices are connected. In the case of PCIe, devices are assigned
addresses (in PCI BARs) from the PCI address spaces. Now if we take an
address from one of these address spaces for our IOVA, transfers from
from a PCI device to that address will terminate inside of the PCI bus.
This is because from the PCI buses' point-of-view, the address we chose
is part of its address space. To make sure we don't allocate addresses
from there, reserve the PCI addresses in the IOVA.

Note that smmu(4) currently gives each device its own IOVA. So the PCI
addresses will be reserved only in IOVA from PCI devices, and only the
addresses concerning the PCI bus it is connected to will be reserved.
All other devices behind an smmu(4) will not have any changes to their
IOVA.

ok kettenis@

Rename some MD structs by giving them an architecture-neutral name in
preparation for sharing PCIe host bridge drivers between arm64 and riscv64.

ok mpi@, mlarkin@, patrick@

Load MSI pages through bus_dma(9). Our interrupt controllers for MSIs
typically pass the physical address, however retrieved, to our PCIe
controller code. This physical address can in practise be directly
given to the PCIe, but it is not a given that the CPU and the PCIe
controller are able to use the same physical addresses.

This is even more obvious with an smmu(4) inbetween, which can change
the world view by introducing I/O virtual addresses. Hence for this
it is indeed necessary to map those pages, which thanks to integration
with bus_dma(9) works easily.

For this we remember the PCI devices' DMA tag in the interrupt handle
during the MSI map, so that we can use the smmu(4)-hooked DMA tag to
load the physical address.

While some systems might prefer to implement "trapping" pages for MSIs,
to make sure devices cannot trigger other devices' interrupts, we only
make sure the whole page is mapped.

Having the IOMMU create a mapping for each MSI is a bit wasteful, but
for now it's the simplest way to implement it.

Discussed with and ok kettenis@

Change API of acpiiort(4). It was written as a hook before, taking the
PCI attach args and replacing the DMA tag inside. Our other IOMMU API
though takes a DMA tag and returns the old one or a new one. To have
acpiiort(4) integrate better with non-PCI ACPI devices, change the API
so that it is more similar to the other API. This also makes the code
easier to understand.

ok kettenis@

Have acpipci(4) look for a matching SMMU in the IORT.

ok kettenis@

Add some infrastructure in the PCI chipset tag for pci_probe_device_hook()
so that we can provide IOMMU-hooked bus DMA tags for each PCI device.

ok kettenis@

Split the IORT struct into two, as the current version not only contained
the generic IORT node information but also the Root Complex's attributes.

ok kettenis@

Move IO Remapping Table (IORT) struct defines to the common ACPI header
so that it can be used by more drivers.

ok kettenis@

Implement pci_intr_disestablish(9) for acpicpi(4) on arm64.

ok patrick@

Implement address translation for bus_space_mmap(9).

ok patrick@

Re-work intr_barrier(9) on arm64 to remove layer violation. So far we
have stored the struct cpu_info * in the wrapper around the interrupt
handler cookie, but since we can have a few layers inbetween, this does
not seem very nice. Instead have each and every interrupt controller
provide a barrier function. This means that intr_barrier(9) will in the
end be executed by the interrupt controller that actually wired the pin
to a core. And that's the only place where the information is stored.

ok kettenis@

Store struct cpu_info * in arm64's interrupt wrap. intr_barrier() can
already assume every cookie is wrapped and simply retrieve the pointer
from it. It's a bit of a layer violation though, since only the intc
should actually store that kind of information. This is good enough for
now, but I'm already cooking up a diff to resolve this.

ok dlg@

To be able to have intr_barrier() on arm64, we need to be able to
somehow gain access to the struct cpu_info * used to establish the
interrupt. One possibility is to store the pointer in the cookie
returned by the establish methods. A better way would be to ask
the interrupt controller directly to do barrier.

This means that all external facing interrupt establish functions
need to wrap the cookie in a common way. We already do this for
FDT-based interrupts. Also most PCI controllers already return
the cookie from the FDT API, which is already wrapped. So arm64's
acpi_intr_establish() and acpipci(4) now need to explicitly wrap
it, since they call ic->ic_establish directly, which does not wrap.

ok dlg@

Implement pci_intr_establish_cpu() on arm64 and armv7. The function pointer
in the chipset tag for establishing interrupts now takes a struct cpu_info *.
The normal pci_intr_establish() macro passes NULL as ci, which indicates that
the primary CPU is to be used.

The PCI controller drivers can then simply pass the ci on to our arm64/armv7
interrupt establish "framework".

Prompted by dlg@
ok kettenis@

Extend the interrupt API on arm64 and armv7 to be able to pass around
a struct cpu_info *. From a driver point of view the fdt_intr_establish_*
API now also exist same functions with a *_cpu suffix. Internally the
"old" functions now call their *_cpu counterparts, passing NULL as ci.
NULL will be interpreted as primary CPU in the interrupt controller code.

The internal framework for interrupt controllers has been changed so that
the establish methods provided by an interrupt controller function always
takes a struct cpu_info *.

Some drivers, like imxgpio(4) and rkgpio(4), only have a single interrupt
line for multiple pins. On those we simply disallow trying to establish
an interrupt on a non-primary CPU, returning NULL.

Since we do not have MP yet on armv7, all armv7 interrupt controllers do
return NULL if an attempt is made to establish an interrupt on a different
CPU. That said, so far there's no way this can happen. If we ever gain
MP support, this is a reminder that the interrupt controller drivers have
to be adjusted.

Prompted by dlg@
ok kettenis@

Fix small bug in parsing the IORT tables. Mapping entries specify the number
of IDs in a range minus one.

ok patrick@

Get bus number from _CRS.

ok sthen@, deraadt@

Don't check _TTP for io windows.

ok patrick@, jsg@

Fix mapping of MSI sideband data when there are SMMU's present.

Seems the Amppere eMAG has a silicon big where the number of writable bits
of the ICC_PMR_EL1 registers varies with the value being written. Change the
value we write to probe the number of writable bits to a value that yields
the desired result.

Suggested by drahn@

ok patrick@

Change pci_intr_handle_t into a struct and replace duplicated code that
implements mapping of MSI and MSI-X interrupts with new generic functions.
Fixes a use-after-free in sone PCI device drivers that call pci_intr_string(9)
after pci_intr_establish(9).

ok deraadt@

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Pass extent for prefetchable mmio. Since there is no distinction between
prefetchable and "normal" mmio at the host bridge level we can simply pass
the same extent.

ok patrick@

Add support for multiple PCI segments. Only really implemented for arm64
for now as amd64/i386 firmware still caters for legacy OSes that only
support a single PCI segment.

ok patrick@

Make legacy interrupts work in more cases.

Use IORT table to map requester ID into MSI sideband data.

Pass PCIe requester ID as sideband data here as well.

Make use of PCI_FLAGS_MSI_ENABLED such that drivers for hardware with broken
MSI support can selectively disable the use of MSI.

Make legacy interrupts work in acpipci(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

Work around MSI and INTx issues on Qualcomm SC8280XP. This makes the NVMe
work on the Lenovo x13s.

ok dv@

The x13s uses Memory32Fixed() to describe the windows forwarded by the
host bridges. Add support for this.

ok mlarkin@, patrick@

Add support for interrupts represented by ACPI PCI Interrupt Link Devices.
This makes PCI interrupts work on QEMU's SBSA target.

ok patrick@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

Don't advertise MSI support if we don't have an MSI interrupt controller.

ok patrick@

Only check whether we have an MSI interrupt controller when we try to
establish an MSI or MSI-X interrupt. Fixes establishing legacy INTx
interrupts on machines without a (usable) MSI interrupt controller.

ok patrick@

While it seems like we can choose any I/O virtual address for peripheral
devices, this isn't really the case. It depends on the bus topology of
how devices are connected. In the case of PCIe, devices are assigned
addresses (in PCI BARs) from the PCI address spaces. Now if we take an
address from one of these address spaces for our IOVA, transfers from
from a PCI device to that address will terminate inside of the PCI bus.
This is because from the PCI buses' point-of-view, the address we chose
is part of its address space. To make sure we don't allocate addresses
from there, reserve the PCI addresses in the IOVA.

Note that smmu(4) currently gives each device its own IOVA. So the PCI
addresses will be reserved only in IOVA from PCI devices, and only the
addresses concerning the PCI bus it is connected to will be reserved.
All other devices behind an smmu(4) will not have any changes to their
IOVA.

ok kettenis@

Rename some MD structs by giving them an architecture-neutral name in
preparation for sharing PCIe host bridge drivers between arm64 and riscv64.

ok mpi@, mlarkin@, patrick@

Load MSI pages through bus_dma(9). Our interrupt controllers for MSIs
typically pass the physical address, however retrieved, to our PCIe
controller code. This physical address can in practise be directly
given to the PCIe, but it is not a given that the CPU and the PCIe
controller are able to use the same physical addresses.

This is even more obvious with an smmu(4) inbetween, which can change
the world view by introducing I/O virtual addresses. Hence for this
it is indeed necessary to map those pages, which thanks to integration
with bus_dma(9) works easily.

For this we remember the PCI devices' DMA tag in the interrupt handle
during the MSI map, so that we can use the smmu(4)-hooked DMA tag to
load the physical address.

While some systems might prefer to implement "trapping" pages for MSIs,
to make sure devices cannot trigger other devices' interrupts, we only
make sure the whole page is mapped.

Having the IOMMU create a mapping for each MSI is a bit wasteful, but
for now it's the simplest way to implement it.

Discussed with and ok kettenis@

Change API of acpiiort(4). It was written as a hook before, taking the
PCI attach args and replacing the DMA tag inside. Our other IOMMU API
though takes a DMA tag and returns the old one or a new one. To have
acpiiort(4) integrate better with non-PCI ACPI devices, change the API
so that it is more similar to the other API. This also makes the code
easier to understand.

ok kettenis@

Have acpipci(4) look for a matching SMMU in the IORT.

ok kettenis@

Add some infrastructure in the PCI chipset tag for pci_probe_device_hook()
so that we can provide IOMMU-hooked bus DMA tags for each PCI device.

ok kettenis@

Split the IORT struct into two, as the current version not only contained
the generic IORT node information but also the Root Complex's attributes.

ok kettenis@

Move IO Remapping Table (IORT) struct defines to the common ACPI header
so that it can be used by more drivers.

ok kettenis@

Implement pci_intr_disestablish(9) for acpicpi(4) on arm64.

ok patrick@

Implement address translation for bus_space_mmap(9).

ok patrick@

Re-work intr_barrier(9) on arm64 to remove layer violation. So far we
have stored the struct cpu_info * in the wrapper around the interrupt
handler cookie, but since we can have a few layers inbetween, this does
not seem very nice. Instead have each and every interrupt controller
provide a barrier function. This means that intr_barrier(9) will in the
end be executed by the interrupt controller that actually wired the pin
to a core. And that's the only place where the information is stored.

ok kettenis@

Store struct cpu_info * in arm64's interrupt wrap. intr_barrier() can
already assume every cookie is wrapped and simply retrieve the pointer
from it. It's a bit of a layer violation though, since only the intc
should actually store that kind of information. This is good enough for
now, but I'm already cooking up a diff to resolve this.

ok dlg@

To be able to have intr_barrier() on arm64, we need to be able to
somehow gain access to the struct cpu_info * used to establish the
interrupt. One possibility is to store the pointer in the cookie
returned by the establish methods. A better way would be to ask
the interrupt controller directly to do barrier.

This means that all external facing interrupt establish functions
need to wrap the cookie in a common way. We already do this for
FDT-based interrupts. Also most PCI controllers already return
the cookie from the FDT API, which is already wrapped. So arm64's
acpi_intr_establish() and acpipci(4) now need to explicitly wrap
it, since they call ic->ic_establish directly, which does not wrap.

ok dlg@

Implement pci_intr_establish_cpu() on arm64 and armv7. The function pointer
in the chipset tag for establishing interrupts now takes a struct cpu_info *.
The normal pci_intr_establish() macro passes NULL as ci, which indicates that
the primary CPU is to be used.

The PCI controller drivers can then simply pass the ci on to our arm64/armv7
interrupt establish "framework".

Prompted by dlg@
ok kettenis@

Extend the interrupt API on arm64 and armv7 to be able to pass around
a struct cpu_info *. From a driver point of view the fdt_intr_establish_*
API now also exist same functions with a *_cpu suffix. Internally the
"old" functions now call their *_cpu counterparts, passing NULL as ci.
NULL will be interpreted as primary CPU in the interrupt controller code.

The internal framework for interrupt controllers has been changed so that
the establish methods provided by an interrupt controller function always
takes a struct cpu_info *.

Some drivers, like imxgpio(4) and rkgpio(4), only have a single interrupt
line for multiple pins. On those we simply disallow trying to establish
an interrupt on a non-primary CPU, returning NULL.

Since we do not have MP yet on armv7, all armv7 interrupt controllers do
return NULL if an attempt is made to establish an interrupt on a different
CPU. That said, so far there's no way this can happen. If we ever gain
MP support, this is a reminder that the interrupt controller drivers have
to be adjusted.

Prompted by dlg@
ok kettenis@

Fix small bug in parsing the IORT tables. Mapping entries specify the number
of IDs in a range minus one.

ok patrick@

Get bus number from _CRS.

ok sthen@, deraadt@

Don't check _TTP for io windows.

ok patrick@, jsg@

Fix mapping of MSI sideband data when there are SMMU's present.

Seems the Amppere eMAG has a silicon big where the number of writable bits
of the ICC_PMR_EL1 registers varies with the value being written. Change the
value we write to probe the number of writable bits to a value that yields
the desired result.

Suggested by drahn@

ok patrick@

Change pci_intr_handle_t into a struct and replace duplicated code that
implements mapping of MSI and MSI-X interrupts with new generic functions.
Fixes a use-after-free in sone PCI device drivers that call pci_intr_string(9)
after pci_intr_establish(9).

ok deraadt@

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Pass extent for prefetchable mmio. Since there is no distinction between
prefetchable and "normal" mmio at the host bridge level we can simply pass
the same extent.

ok patrick@

Add support for multiple PCI segments. Only really implemented for arm64
for now as amd64/i386 firmware still caters for legacy OSes that only
support a single PCI segment.

ok patrick@

Make legacy interrupts work in more cases.

Use IORT table to map requester ID into MSI sideband data.

Pass PCIe requester ID as sideband data here as well.

Make use of PCI_FLAGS_MSI_ENABLED such that drivers for hardware with broken
MSI support can selectively disable the use of MSI.

Make legacy interrupts work in acpipci(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

The x13s uses Memory32Fixed() to describe the windows forwarded by the
host bridges. Add support for this.

ok mlarkin@, patrick@

Add support for interrupts represented by ACPI PCI Interrupt Link Devices.
This makes PCI interrupts work on QEMU's SBSA target.

ok patrick@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

Don't advertise MSI support if we don't have an MSI interrupt controller.

ok patrick@

Only check whether we have an MSI interrupt controller when we try to
establish an MSI or MSI-X interrupt. Fixes establishing legacy INTx
interrupts on machines without a (usable) MSI interrupt controller.

ok patrick@

While it seems like we can choose any I/O virtual address for peripheral
devices, this isn't really the case. It depends on the bus topology of
how devices are connected. In the case of PCIe, devices are assigned
addresses (in PCI BARs) from the PCI address spaces. Now if we take an
address from one of these address spaces for our IOVA, transfers from
from a PCI device to that address will terminate inside of the PCI bus.
This is because from the PCI buses' point-of-view, the address we chose
is part of its address space. To make sure we don't allocate addresses
from there, reserve the PCI addresses in the IOVA.

Note that smmu(4) currently gives each device its own IOVA. So the PCI
addresses will be reserved only in IOVA from PCI devices, and only the
addresses concerning the PCI bus it is connected to will be reserved.
All other devices behind an smmu(4) will not have any changes to their
IOVA.

ok kettenis@

Rename some MD structs by giving them an architecture-neutral name in
preparation for sharing PCIe host bridge drivers between arm64 and riscv64.

ok mpi@, mlarkin@, patrick@

Load MSI pages through bus_dma(9). Our interrupt controllers for MSIs
typically pass the physical address, however retrieved, to our PCIe
controller code. This physical address can in practise be directly
given to the PCIe, but it is not a given that the CPU and the PCIe
controller are able to use the same physical addresses.

This is even more obvious with an smmu(4) inbetween, which can change
the world view by introducing I/O virtual addresses. Hence for this
it is indeed necessary to map those pages, which thanks to integration
with bus_dma(9) works easily.

For this we remember the PCI devices' DMA tag in the interrupt handle
during the MSI map, so that we can use the smmu(4)-hooked DMA tag to
load the physical address.

While some systems might prefer to implement "trapping" pages for MSIs,
to make sure devices cannot trigger other devices' interrupts, we only
make sure the whole page is mapped.

Having the IOMMU create a mapping for each MSI is a bit wasteful, but
for now it's the simplest way to implement it.

Discussed with and ok kettenis@

Change API of acpiiort(4). It was written as a hook before, taking the
PCI attach args and replacing the DMA tag inside. Our other IOMMU API
though takes a DMA tag and returns the old one or a new one. To have
acpiiort(4) integrate better with non-PCI ACPI devices, change the API
so that it is more similar to the other API. This also makes the code
easier to understand.

ok kettenis@

Have acpipci(4) look for a matching SMMU in the IORT.

ok kettenis@

Add some infrastructure in the PCI chipset tag for pci_probe_device_hook()
so that we can provide IOMMU-hooked bus DMA tags for each PCI device.

ok kettenis@

Split the IORT struct into two, as the current version not only contained
the generic IORT node information but also the Root Complex's attributes.

ok kettenis@

Move IO Remapping Table (IORT) struct defines to the common ACPI header
so that it can be used by more drivers.

ok kettenis@

Implement pci_intr_disestablish(9) for acpicpi(4) on arm64.

ok patrick@

Implement address translation for bus_space_mmap(9).

ok patrick@

Re-work intr_barrier(9) on arm64 to remove layer violation. So far we
have stored the struct cpu_info * in the wrapper around the interrupt
handler cookie, but since we can have a few layers inbetween, this does
not seem very nice. Instead have each and every interrupt controller
provide a barrier function. This means that intr_barrier(9) will in the
end be executed by the interrupt controller that actually wired the pin
to a core. And that's the only place where the information is stored.

ok kettenis@

Store struct cpu_info * in arm64's interrupt wrap. intr_barrier() can
already assume every cookie is wrapped and simply retrieve the pointer
from it. It's a bit of a layer violation though, since only the intc
should actually store that kind of information. This is good enough for
now, but I'm already cooking up a diff to resolve this.

ok dlg@

To be able to have intr_barrier() on arm64, we need to be able to
somehow gain access to the struct cpu_info * used to establish the
interrupt. One possibility is to store the pointer in the cookie
returned by the establish methods. A better way would be to ask
the interrupt controller directly to do barrier.

This means that all external facing interrupt establish functions
need to wrap the cookie in a common way. We already do this for
FDT-based interrupts. Also most PCI controllers already return
the cookie from the FDT API, which is already wrapped. So arm64's
acpi_intr_establish() and acpipci(4) now need to explicitly wrap
it, since they call ic->ic_establish directly, which does not wrap.

ok dlg@

Implement pci_intr_establish_cpu() on arm64 and armv7. The function pointer
in the chipset tag for establishing interrupts now takes a struct cpu_info *.
The normal pci_intr_establish() macro passes NULL as ci, which indicates that
the primary CPU is to be used.

The PCI controller drivers can then simply pass the ci on to our arm64/armv7
interrupt establish "framework".

Prompted by dlg@
ok kettenis@

Extend the interrupt API on arm64 and armv7 to be able to pass around
a struct cpu_info *. From a driver point of view the fdt_intr_establish_*
API now also exist same functions with a *_cpu suffix. Internally the
"old" functions now call their *_cpu counterparts, passing NULL as ci.
NULL will be interpreted as primary CPU in the interrupt controller code.

The internal framework for interrupt controllers has been changed so that
the establish methods provided by an interrupt controller function always
takes a struct cpu_info *.

Some drivers, like imxgpio(4) and rkgpio(4), only have a single interrupt
line for multiple pins. On those we simply disallow trying to establish
an interrupt on a non-primary CPU, returning NULL.

Since we do not have MP yet on armv7, all armv7 interrupt controllers do
return NULL if an attempt is made to establish an interrupt on a different
CPU. That said, so far there's no way this can happen. If we ever gain
MP support, this is a reminder that the interrupt controller drivers have
to be adjusted.

Prompted by dlg@
ok kettenis@

Fix small bug in parsing the IORT tables. Mapping entries specify the number
of IDs in a range minus one.

ok patrick@

Get bus number from _CRS.

ok sthen@, deraadt@

Don't check _TTP for io windows.

ok patrick@, jsg@

Fix mapping of MSI sideband data when there are SMMU's present.

Seems the Amppere eMAG has a silicon big where the number of writable bits
of the ICC_PMR_EL1 registers varies with the value being written. Change the
value we write to probe the number of writable bits to a value that yields
the desired result.

Suggested by drahn@

ok patrick@

Change pci_intr_handle_t into a struct and replace duplicated code that
implements mapping of MSI and MSI-X interrupts with new generic functions.
Fixes a use-after-free in sone PCI device drivers that call pci_intr_string(9)
after pci_intr_establish(9).

ok deraadt@

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Pass extent for prefetchable mmio. Since there is no distinction between
prefetchable and "normal" mmio at the host bridge level we can simply pass
the same extent.

ok patrick@

Add support for multiple PCI segments. Only really implemented for arm64
for now as amd64/i386 firmware still caters for legacy OSes that only
support a single PCI segment.

ok patrick@

Make legacy interrupts work in more cases.

Use IORT table to map requester ID into MSI sideband data.

Pass PCIe requester ID as sideband data here as well.

Make use of PCI_FLAGS_MSI_ENABLED such that drivers for hardware with broken
MSI support can selectively disable the use of MSI.

Make legacy interrupts work in acpipci(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

Add support for interrupts represented by ACPI PCI Interrupt Link Devices.
This makes PCI interrupts work on QEMU's SBSA target.

ok patrick@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

Don't advertise MSI support if we don't have an MSI interrupt controller.

ok patrick@

Only check whether we have an MSI interrupt controller when we try to
establish an MSI or MSI-X interrupt. Fixes establishing legacy INTx
interrupts on machines without a (usable) MSI interrupt controller.

ok patrick@

While it seems like we can choose any I/O virtual address for peripheral
devices, this isn't really the case. It depends on the bus topology of
how devices are connected. In the case of PCIe, devices are assigned
addresses (in PCI BARs) from the PCI address spaces. Now if we take an
address from one of these address spaces for our IOVA, transfers from
from a PCI device to that address will terminate inside of the PCI bus.
This is because from the PCI buses' point-of-view, the address we chose
is part of its address space. To make sure we don't allocate addresses
from there, reserve the PCI addresses in the IOVA.

Note that smmu(4) currently gives each device its own IOVA. So the PCI
addresses will be reserved only in IOVA from PCI devices, and only the
addresses concerning the PCI bus it is connected to will be reserved.
All other devices behind an smmu(4) will not have any changes to their
IOVA.

ok kettenis@

Rename some MD structs by giving them an architecture-neutral name in
preparation for sharing PCIe host bridge drivers between arm64 and riscv64.

ok mpi@, mlarkin@, patrick@

Load MSI pages through bus_dma(9). Our interrupt controllers for MSIs
typically pass the physical address, however retrieved, to our PCIe
controller code. This physical address can in practise be directly
given to the PCIe, but it is not a given that the CPU and the PCIe
controller are able to use the same physical addresses.

This is even more obvious with an smmu(4) inbetween, which can change
the world view by introducing I/O virtual addresses. Hence for this
it is indeed necessary to map those pages, which thanks to integration
with bus_dma(9) works easily.

For this we remember the PCI devices' DMA tag in the interrupt handle
during the MSI map, so that we can use the smmu(4)-hooked DMA tag to
load the physical address.

While some systems might prefer to implement "trapping" pages for MSIs,
to make sure devices cannot trigger other devices' interrupts, we only
make sure the whole page is mapped.

Having the IOMMU create a mapping for each MSI is a bit wasteful, but
for now it's the simplest way to implement it.

Discussed with and ok kettenis@

Change API of acpiiort(4). It was written as a hook before, taking the
PCI attach args and replacing the DMA tag inside. Our other IOMMU API
though takes a DMA tag and returns the old one or a new one. To have
acpiiort(4) integrate better with non-PCI ACPI devices, change the API
so that it is more similar to the other API. This also makes the code
easier to understand.

ok kettenis@

Have acpipci(4) look for a matching SMMU in the IORT.

ok kettenis@

Add some infrastructure in the PCI chipset tag for pci_probe_device_hook()
so that we can provide IOMMU-hooked bus DMA tags for each PCI device.

ok kettenis@

Split the IORT struct into two, as the current version not only contained
the generic IORT node information but also the Root Complex's attributes.

ok kettenis@

Move IO Remapping Table (IORT) struct defines to the common ACPI header
so that it can be used by more drivers.

ok kettenis@

Implement pci_intr_disestablish(9) for acpicpi(4) on arm64.

ok patrick@

Implement address translation for bus_space_mmap(9).

ok patrick@

Re-work intr_barrier(9) on arm64 to remove layer violation. So far we
have stored the struct cpu_info * in the wrapper around the interrupt
handler cookie, but since we can have a few layers inbetween, this does
not seem very nice. Instead have each and every interrupt controller
provide a barrier function. This means that intr_barrier(9) will in the
end be executed by the interrupt controller that actually wired the pin
to a core. And that's the only place where the information is stored.

ok kettenis@

Store struct cpu_info * in arm64's interrupt wrap. intr_barrier() can
already assume every cookie is wrapped and simply retrieve the pointer
from it. It's a bit of a layer violation though, since only the intc
should actually store that kind of information. This is good enough for
now, but I'm already cooking up a diff to resolve this.

ok dlg@

To be able to have intr_barrier() on arm64, we need to be able to
somehow gain access to the struct cpu_info * used to establish the
interrupt. One possibility is to store the pointer in the cookie
returned by the establish methods. A better way would be to ask
the interrupt controller directly to do barrier.

This means that all external facing interrupt establish functions
need to wrap the cookie in a common way. We already do this for
FDT-based interrupts. Also most PCI controllers already return
the cookie from the FDT API, which is already wrapped. So arm64's
acpi_intr_establish() and acpipci(4) now need to explicitly wrap
it, since they call ic->ic_establish directly, which does not wrap.

ok dlg@

Implement pci_intr_establish_cpu() on arm64 and armv7. The function pointer
in the chipset tag for establishing interrupts now takes a struct cpu_info *.
The normal pci_intr_establish() macro passes NULL as ci, which indicates that
the primary CPU is to be used.

The PCI controller drivers can then simply pass the ci on to our arm64/armv7
interrupt establish "framework".

Prompted by dlg@
ok kettenis@

Extend the interrupt API on arm64 and armv7 to be able to pass around
a struct cpu_info *. From a driver point of view the fdt_intr_establish_*
API now also exist same functions with a *_cpu suffix. Internally the
"old" functions now call their *_cpu counterparts, passing NULL as ci.
NULL will be interpreted as primary CPU in the interrupt controller code.

The internal framework for interrupt controllers has been changed so that
the establish methods provided by an interrupt controller function always
takes a struct cpu_info *.

Some drivers, like imxgpio(4) and rkgpio(4), only have a single interrupt
line for multiple pins. On those we simply disallow trying to establish
an interrupt on a non-primary CPU, returning NULL.

Since we do not have MP yet on armv7, all armv7 interrupt controllers do
return NULL if an attempt is made to establish an interrupt on a different
CPU. That said, so far there's no way this can happen. If we ever gain
MP support, this is a reminder that the interrupt controller drivers have
to be adjusted.

Prompted by dlg@
ok kettenis@

Fix small bug in parsing the IORT tables. Mapping entries specify the number
of IDs in a range minus one.

ok patrick@

Get bus number from _CRS.

ok sthen@, deraadt@

Don't check _TTP for io windows.

ok patrick@, jsg@

Fix mapping of MSI sideband data when there are SMMU's present.

Seems the Amppere eMAG has a silicon big where the number of writable bits
of the ICC_PMR_EL1 registers varies with the value being written. Change the
value we write to probe the number of writable bits to a value that yields
the desired result.

Suggested by drahn@

ok patrick@

Change pci_intr_handle_t into a struct and replace duplicated code that
implements mapping of MSI and MSI-X interrupts with new generic functions.
Fixes a use-after-free in sone PCI device drivers that call pci_intr_string(9)
after pci_intr_establish(9).

ok deraadt@

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Pass extent for prefetchable mmio. Since there is no distinction between
prefetchable and "normal" mmio at the host bridge level we can simply pass
the same extent.

ok patrick@

Add support for multiple PCI segments. Only really implemented for arm64
for now as amd64/i386 firmware still caters for legacy OSes that only
support a single PCI segment.

ok patrick@

Make legacy interrupts work in more cases.

Use IORT table to map requester ID into MSI sideband data.

Pass PCIe requester ID as sideband data here as well.

Make use of PCI_FLAGS_MSI_ENABLED such that drivers for hardware with broken
MSI support can selectively disable the use of MSI.

Make legacy interrupts work in acpipci(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

Constify struct cfattach.

ok visa@ a long time ago, ok patrick@

Don't advertise MSI support if we don't have an MSI interrupt controller.

ok patrick@

Only check whether we have an MSI interrupt controller when we try to
establish an MSI or MSI-X interrupt. Fixes establishing legacy INTx
interrupts on machines without a (usable) MSI interrupt controller.

ok patrick@

While it seems like we can choose any I/O virtual address for peripheral
devices, this isn't really the case. It depends on the bus topology of
how devices are connected. In the case of PCIe, devices are assigned
addresses (in PCI BARs) from the PCI address spaces. Now if we take an
address from one of these address spaces for our IOVA, transfers from
from a PCI device to that address will terminate inside of the PCI bus.
This is because from the PCI buses' point-of-view, the address we chose
is part of its address space. To make sure we don't allocate addresses
from there, reserve the PCI addresses in the IOVA.

Note that smmu(4) currently gives each device its own IOVA. So the PCI
addresses will be reserved only in IOVA from PCI devices, and only the
addresses concerning the PCI bus it is connected to will be reserved.
All other devices behind an smmu(4) will not have any changes to their
IOVA.

ok kettenis@

Rename some MD structs by giving them an architecture-neutral name in
preparation for sharing PCIe host bridge drivers between arm64 and riscv64.

ok mpi@, mlarkin@, patrick@

Load MSI pages through bus_dma(9). Our interrupt controllers for MSIs
typically pass the physical address, however retrieved, to our PCIe
controller code. This physical address can in practise be directly
given to the PCIe, but it is not a given that the CPU and the PCIe
controller are able to use the same physical addresses.

This is even more obvious with an smmu(4) inbetween, which can change
the world view by introducing I/O virtual addresses. Hence for this
it is indeed necessary to map those pages, which thanks to integration
with bus_dma(9) works easily.

For this we remember the PCI devices' DMA tag in the interrupt handle
during the MSI map, so that we can use the smmu(4)-hooked DMA tag to
load the physical address.

While some systems might prefer to implement "trapping" pages for MSIs,
to make sure devices cannot trigger other devices' interrupts, we only
make sure the whole page is mapped.

Having the IOMMU create a mapping for each MSI is a bit wasteful, but
for now it's the simplest way to implement it.

Discussed with and ok kettenis@

Change API of acpiiort(4). It was written as a hook before, taking the
PCI attach args and replacing the DMA tag inside. Our other IOMMU API
though takes a DMA tag and returns the old one or a new one. To have
acpiiort(4) integrate better with non-PCI ACPI devices, change the API
so that it is more similar to the other API. This also makes the code
easier to understand.

ok kettenis@

Have acpipci(4) look for a matching SMMU in the IORT.

ok kettenis@

Add some infrastructure in the PCI chipset tag for pci_probe_device_hook()
so that we can provide IOMMU-hooked bus DMA tags for each PCI device.

ok kettenis@

Split the IORT struct into two, as the current version not only contained
the generic IORT node information but also the Root Complex's attributes.

ok kettenis@

Move IO Remapping Table (IORT) struct defines to the common ACPI header
so that it can be used by more drivers.

ok kettenis@

Implement pci_intr_disestablish(9) for acpicpi(4) on arm64.

ok patrick@

Implement address translation for bus_space_mmap(9).

ok patrick@

Re-work intr_barrier(9) on arm64 to remove layer violation. So far we
have stored the struct cpu_info * in the wrapper around the interrupt
handler cookie, but since we can have a few layers inbetween, this does
not seem very nice. Instead have each and every interrupt controller
provide a barrier function. This means that intr_barrier(9) will in the
end be executed by the interrupt controller that actually wired the pin
to a core. And that's the only place where the information is stored.

ok kettenis@

Store struct cpu_info * in arm64's interrupt wrap. intr_barrier() can
already assume every cookie is wrapped and simply retrieve the pointer
from it. It's a bit of a layer violation though, since only the intc
should actually store that kind of information. This is good enough for
now, but I'm already cooking up a diff to resolve this.

ok dlg@

To be able to have intr_barrier() on arm64, we need to be able to
somehow gain access to the struct cpu_info * used to establish the
interrupt. One possibility is to store the pointer in the cookie
returned by the establish methods. A better way would be to ask
the interrupt controller directly to do barrier.

This means that all external facing interrupt establish functions
need to wrap the cookie in a common way. We already do this for
FDT-based interrupts. Also most PCI controllers already return
the cookie from the FDT API, which is already wrapped. So arm64's
acpi_intr_establish() and acpipci(4) now need to explicitly wrap
it, since they call ic->ic_establish directly, which does not wrap.

ok dlg@

Implement pci_intr_establish_cpu() on arm64 and armv7. The function pointer
in the chipset tag for establishing interrupts now takes a struct cpu_info *.
The normal pci_intr_establish() macro passes NULL as ci, which indicates that
the primary CPU is to be used.

The PCI controller drivers can then simply pass the ci on to our arm64/armv7
interrupt establish "framework".

Prompted by dlg@
ok kettenis@

Extend the interrupt API on arm64 and armv7 to be able to pass around
a struct cpu_info *. From a driver point of view the fdt_intr_establish_*
API now also exist same functions with a *_cpu suffix. Internally the
"old" functions now call their *_cpu counterparts, passing NULL as ci.
NULL will be interpreted as primary CPU in the interrupt controller code.

The internal framework for interrupt controllers has been changed so that
the establish methods provided by an interrupt controller function always
takes a struct cpu_info *.

Some drivers, like imxgpio(4) and rkgpio(4), only have a single interrupt
line for multiple pins. On those we simply disallow trying to establish
an interrupt on a non-primary CPU, returning NULL.

Since we do not have MP yet on armv7, all armv7 interrupt controllers do
return NULL if an attempt is made to establish an interrupt on a different
CPU. That said, so far there's no way this can happen. If we ever gain
MP support, this is a reminder that the interrupt controller drivers have
to be adjusted.

Prompted by dlg@
ok kettenis@

Fix small bug in parsing the IORT tables. Mapping entries specify the number
of IDs in a range minus one.

ok patrick@

Get bus number from _CRS.

ok sthen@, deraadt@

Don't check _TTP for io windows.

ok patrick@, jsg@

Fix mapping of MSI sideband data when there are SMMU's present.

Seems the Amppere eMAG has a silicon big where the number of writable bits
of the ICC_PMR_EL1 registers varies with the value being written. Change the
value we write to probe the number of writable bits to a value that yields
the desired result.

Suggested by drahn@

ok patrick@

Change pci_intr_handle_t into a struct and replace duplicated code that
implements mapping of MSI and MSI-X interrupts with new generic functions.
Fixes a use-after-free in sone PCI device drivers that call pci_intr_string(9)
after pci_intr_establish(9).

ok deraadt@

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Pass extent for prefetchable mmio. Since there is no distinction between
prefetchable and "normal" mmio at the host bridge level we can simply pass
the same extent.

ok patrick@

Add support for multiple PCI segments. Only really implemented for arm64
for now as amd64/i386 firmware still caters for legacy OSes that only
support a single PCI segment.

ok patrick@

Make legacy interrupts work in more cases.

Use IORT table to map requester ID into MSI sideband data.

Pass PCIe requester ID as sideband data here as well.

Make use of PCI_FLAGS_MSI_ENABLED such that drivers for hardware with broken
MSI support can selectively disable the use of MSI.

Make legacy interrupts work in acpipci(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

Don't advertise MSI support if we don't have an MSI interrupt controller.

ok patrick@

Only check whether we have an MSI interrupt controller when we try to
establish an MSI or MSI-X interrupt. Fixes establishing legacy INTx
interrupts on machines without a (usable) MSI interrupt controller.

ok patrick@

While it seems like we can choose any I/O virtual address for peripheral
devices, this isn't really the case. It depends on the bus topology of
how devices are connected. In the case of PCIe, devices are assigned
addresses (in PCI BARs) from the PCI address spaces. Now if we take an
address from one of these address spaces for our IOVA, transfers from
from a PCI device to that address will terminate inside of the PCI bus.
This is because from the PCI buses' point-of-view, the address we chose
is part of its address space. To make sure we don't allocate addresses
from there, reserve the PCI addresses in the IOVA.

Note that smmu(4) currently gives each device its own IOVA. So the PCI
addresses will be reserved only in IOVA from PCI devices, and only the
addresses concerning the PCI bus it is connected to will be reserved.
All other devices behind an smmu(4) will not have any changes to their
IOVA.

ok kettenis@

Rename some MD structs by giving them an architecture-neutral name in
preparation for sharing PCIe host bridge drivers between arm64 and riscv64.

ok mpi@, mlarkin@, patrick@

Load MSI pages through bus_dma(9). Our interrupt controllers for MSIs
typically pass the physical address, however retrieved, to our PCIe
controller code. This physical address can in practise be directly
given to the PCIe, but it is not a given that the CPU and the PCIe
controller are able to use the same physical addresses.

This is even more obvious with an smmu(4) inbetween, which can change
the world view by introducing I/O virtual addresses. Hence for this
it is indeed necessary to map those pages, which thanks to integration
with bus_dma(9) works easily.

For this we remember the PCI devices' DMA tag in the interrupt handle
during the MSI map, so that we can use the smmu(4)-hooked DMA tag to
load the physical address.

While some systems might prefer to implement "trapping" pages for MSIs,
to make sure devices cannot trigger other devices' interrupts, we only
make sure the whole page is mapped.

Having the IOMMU create a mapping for each MSI is a bit wasteful, but
for now it's the simplest way to implement it.

Discussed with and ok kettenis@

Change API of acpiiort(4). It was written as a hook before, taking the
PCI attach args and replacing the DMA tag inside. Our other IOMMU API
though takes a DMA tag and returns the old one or a new one. To have
acpiiort(4) integrate better with non-PCI ACPI devices, change the API
so that it is more similar to the other API. This also makes the code
easier to understand.

ok kettenis@

Have acpipci(4) look for a matching SMMU in the IORT.

ok kettenis@

Add some infrastructure in the PCI chipset tag for pci_probe_device_hook()
so that we can provide IOMMU-hooked bus DMA tags for each PCI device.

ok kettenis@

Split the IORT struct into two, as the current version not only contained
the generic IORT node information but also the Root Complex's attributes.

ok kettenis@

Move IO Remapping Table (IORT) struct defines to the common ACPI header
so that it can be used by more drivers.

ok kettenis@

Implement pci_intr_disestablish(9) for acpicpi(4) on arm64.

ok patrick@

Implement address translation for bus_space_mmap(9).

ok patrick@

Re-work intr_barrier(9) on arm64 to remove layer violation. So far we
have stored the struct cpu_info * in the wrapper around the interrupt
handler cookie, but since we can have a few layers inbetween, this does
not seem very nice. Instead have each and every interrupt controller
provide a barrier function. This means that intr_barrier(9) will in the
end be executed by the interrupt controller that actually wired the pin
to a core. And that's the only place where the information is stored.

ok kettenis@

Store struct cpu_info * in arm64's interrupt wrap. intr_barrier() can
already assume every cookie is wrapped and simply retrieve the pointer
from it. It's a bit of a layer violation though, since only the intc
should actually store that kind of information. This is good enough for
now, but I'm already cooking up a diff to resolve this.

ok dlg@

To be able to have intr_barrier() on arm64, we need to be able to
somehow gain access to the struct cpu_info * used to establish the
interrupt. One possibility is to store the pointer in the cookie
returned by the establish methods. A better way would be to ask
the interrupt controller directly to do barrier.

This means that all external facing interrupt establish functions
need to wrap the cookie in a common way. We already do this for
FDT-based interrupts. Also most PCI controllers already return
the cookie from the FDT API, which is already wrapped. So arm64's
acpi_intr_establish() and acpipci(4) now need to explicitly wrap
it, since they call ic->ic_establish directly, which does not wrap.

ok dlg@

Implement pci_intr_establish_cpu() on arm64 and armv7. The function pointer
in the chipset tag for establishing interrupts now takes a struct cpu_info *.
The normal pci_intr_establish() macro passes NULL as ci, which indicates that
the primary CPU is to be used.

The PCI controller drivers can then simply pass the ci on to our arm64/armv7
interrupt establish "framework".

Prompted by dlg@
ok kettenis@

Extend the interrupt API on arm64 and armv7 to be able to pass around
a struct cpu_info *. From a driver point of view the fdt_intr_establish_*
API now also exist same functions with a *_cpu suffix. Internally the
"old" functions now call their *_cpu counterparts, passing NULL as ci.
NULL will be interpreted as primary CPU in the interrupt controller code.

The internal framework for interrupt controllers has been changed so that
the establish methods provided by an interrupt controller function always
takes a struct cpu_info *.

Some drivers, like imxgpio(4) and rkgpio(4), only have a single interrupt
line for multiple pins. On those we simply disallow trying to establish
an interrupt on a non-primary CPU, returning NULL.

Since we do not have MP yet on armv7, all armv7 interrupt controllers do
return NULL if an attempt is made to establish an interrupt on a different
CPU. That said, so far there's no way this can happen. If we ever gain
MP support, this is a reminder that the interrupt controller drivers have
to be adjusted.

Prompted by dlg@
ok kettenis@

Fix small bug in parsing the IORT tables. Mapping entries specify the number
of IDs in a range minus one.

ok patrick@

Get bus number from _CRS.

ok sthen@, deraadt@

Don't check _TTP for io windows.

ok patrick@, jsg@

Fix mapping of MSI sideband data when there are SMMU's present.

Seems the Amppere eMAG has a silicon big where the number of writable bits
of the ICC_PMR_EL1 registers varies with the value being written. Change the
value we write to probe the number of writable bits to a value that yields
the desired result.

Suggested by drahn@

ok patrick@

Change pci_intr_handle_t into a struct and replace duplicated code that
implements mapping of MSI and MSI-X interrupts with new generic functions.
Fixes a use-after-free in sone PCI device drivers that call pci_intr_string(9)
after pci_intr_establish(9).

ok deraadt@

Add MSI-X support for acpipci(4). This splits out some generic code into
a new pci_machdep.c file such that it can be re-used by other arm64
PCI host bridge drivers in the future.

ok patrick@

Pass extent for prefetchable mmio. Since there is no distinction between
prefetchable and "normal" mmio at the host bridge level we can simply pass
the same extent.

ok patrick@

Add support for multiple PCI segments. Only really implemented for arm64
for now as amd64/i386 firmware still caters for legacy OSes that only
support a single PCI segment.

ok patrick@

Make legacy interrupts work in more cases.

Use IORT table to map requester ID into MSI sideband data.

Pass PCIe requester ID as sideband data here as well.

Make use of PCI_FLAGS_MSI_ENABLED such that drivers for hardware with broken
MSI support can selectively disable the use of MSI.

Make legacy interrupts work in acpipci(4).

ok patrick@

Add acpipci(4), a driver that supports generic ECAM-compatible PCI host
bridges based on information provided by ACPI.

ok mlarkin@

While it seems like we can choose any I/O virtual address for peripheral
devices, this isn't really the case. It depends on the bus topology of
how devices are connected. In the case of PCIe, devices are assigned
addresses (in PCI BARs) from the PCI address spaces. Now if we take an
address from one of these address spaces for our IOVA, transfers from
from a PCI device to that address will terminate inside of the PCI bus.
This is because from the PCI buses' point-of-view, the address we chose
is part of its address space. To make sure we don't allocate addresses
from there, reserve the PCI addresses in the IOVA.

Note that smmu(4) currently gives each device its own IOVA. So the PCI
addresses will be reserved only in IOVA from PCI devices, and only the
addresses concerning the PCI bus it is connected to will be reserved.
All other devices behind an smmu(4) will not have any changes to their
IOVA.

ok kettenis@

Rename some MD structs by giving them an architecture-neutral name in
preparation for sharing PCIe host bridge drivers between arm64 and riscv64.

ok mpi@, mlarkin@, patrick@

Load MSI pages through bus_dma(9). Our interrupt controllers for MSIs
typically pass the physical address, however retrieved, to our PCIe
controller code. This physical address can in practise be directly
given to the PCIe, but it is not a given that the CPU and the PCIe
controller are able to use the same physical addresses.

This is even more obvious with an smmu(4) inbetween, which can change
the world view by introducing I/O virtual addresses. Hence for this
it is indeed necessary to map those pages, which thanks to integration
with bus_dma(9) works easily.

For this we remember the PCI devices' DMA tag in the interrupt handle
during the MSI map, so that we can use the smmu(4)-hooked DMA tag to
load the physical address.

While some systems might prefer to implement "trapping" pages for MSIs,
to make sure devices cannot trigger other devices' interrupts, we only
make sure the whole page is mapped.

Having the IOMMU create a mapping for each MSI is a bit wasteful, but
for now it's the simplest way to implement it.

Discussed with and ok kettenis@

Change API of acpiiort(4). It was written as a hook before, taking the
PCI attach args and replacing the DMA tag inside. Our other IOMMU API
though takes a DMA tag and returns the old one or a new one. To have
acpiiort(4) integrate better with non-PCI ACPI devices, change the API
so that it is more similar to the other API. This also makes the code
easier to understand.

ok kettenis@

Have acpipci(4) look for a matching SMMU in the IORT.

ok kettenis@

Add some infrastructure in the PCI chipset tag for pci_probe_device_hook()
so that we can provide IOMMU-hooked bus DMA tags for each PCI device.

ok kettenis@

Split the IORT struct into two, as the current version not only contained
the generic IORT node information but also the Root Complex's attributes.

ok kettenis@

Move IO Remapping Table (IORT) struct defines to the common ACPI header
so that it can be used by more drivers.

ok kettenis@

Implement pci_intr_disestablish(9) for acpicpi(4) on arm64.

ok patrick@

Implement address translation for bus_space_mmap(9).

ok patrick@

Re-work intr_barrier(9) on arm64 to remove layer violation. So far we
have stored the struct cpu_info * in the wrapper around the interrupt
handler cookie, but since we can have a few layers inbetween, this does
not seem very nice. Instead have each and every interrupt controller
provide a barrier function. This means that intr_barrier(9) will in the
end be executed by the interrupt controller that actually wired the pin
to a core. And that's the only place where the information is stored.

ok kettenis@

Store struct cpu_info * in arm64's interrupt wrap. intr_barrier() can
already assume every cookie is wrapped and simply retrieve the pointer
from it. It's a bit of a layer violation though, since only the intc
should actually store that kind of information. This is good enough for
now, but I'm already cooking up a diff to resolve this.

ok dlg@

To be able to have intr_barrier() on arm64, we need to be able to
somehow gain access to the struct cpu_info * used to establish the
interrupt. One possibility is to store the pointer in the cookie
returned by the establish methods. A better way would be to ask
the interrupt controller directly to do barrier.

This means that all external facing interrupt establish functions
need to wrap the cookie in a common way. We already do this for
FDT-based interrupts. Also most PCI controllers already return
the cookie from the FDT API, which is already wrapped. So arm64's
acpi_intr_establish() and acpipci(4) now need to explicitly wrap
it, since they call ic->ic_establish directly, which does not wrap.

ok dlg@

Implement pci_intr_establish_cpu() on arm64 and armv7. The function pointer
in the chipset tag for establishing interrupts now takes a struct cpu_info *.
The normal pci_intr_establish() macro passes NULL as ci, which indicates that
the primary CPU is to be used.

The PCI controller drivers can then simply pass the ci on to our arm64/armv7
interrupt establish "framework".

Prompted by dlg@
ok kettenis@