#
40d951bc |
|
24-May-2024 |
Konstantin Belousov <kib@FreeBSD.org> |
x86/iommu: extract useful utilities into x86_iommu.c related to the page tables page allocation and mapping. Sponsored by: The FreeBSD Foundation Sponsored by: Advanced Micro Devices (AMD) MFC after: 1 week
|
#
fdafd315 |
|
24-Nov-2023 |
Warner Losh <imp@FreeBSD.org> |
sys: Automated cleanup of cdefs and other formatting Apply the following automated changes to try to eliminate no-longer-needed sys/cdefs.h includes as well as now-empty blank lines in a row. Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/ Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/ Remove /\n+#if.*\n#endif.*\n+/ Remove /^#if.*\n#endif.*\n/ Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/ Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/ Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/ Sponsored by: Netflix
|
#
685dc743 |
|
16-Aug-2023 |
Warner Losh <imp@FreeBSD.org> |
sys: Remove $FreeBSD$: one-line .c pattern Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/
|
#
4d846d26 |
|
10-May-2023 |
Warner Losh <imp@FreeBSD.org> |
spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch up to that fact and revert to their recommended match of BSD-2-Clause. Discussed with: pfg MFC After: 3 days Sponsored by: Netflix
|
#
661bd70b |
|
21-Oct-2021 |
Konstantin Belousov <kib@FreeBSD.org> |
DMAR: clean up warnings about write-only variables For some of them, used only when KTR or KMSAN are configured, apply __unused attribute directly. Sponsored by: The FreeBSD Foundation MFC after: 1 week
|
#
9feff969 |
|
08-Aug-2021 |
Ed Maste <emaste@FreeBSD.org> |
Remove "All Rights Reserved" from FreeBSD Foundation sys/ copyrights These ones were unambiguous cases where the Foundation was the only listed copyright holder (in the associated license block). Sponsored by: The FreeBSD Foundation
|
#
ab6c81a2 |
|
01-Sep-2020 |
Mateusz Guzik <mjg@FreeBSD.org> |
x86: clean up empty lines in .c and .h files
|
#
16696f60 |
|
06-Aug-2020 |
Ruslan Bukin <br@FreeBSD.org> |
Add iommu_domain constructor and destructor. Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25956
|
#
78b51754 |
|
04-Aug-2020 |
Ruslan Bukin <br@FreeBSD.org> |
Add a few macroses for conversion between DMAR unit, domain, ctx and IOMMU unit, domain, ctx. Reviewed by: kib Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D25926
|
#
0eed04c8 |
|
31-Jul-2020 |
Ruslan Bukin <br@FreeBSD.org> |
Add iommu_domain_map_ops virtual table with map/unmap methods so x86 can support Intel DMAR and AMD IOMMU simultaneously. Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25894
|
#
c8597a1f |
|
29-Jul-2020 |
Ruslan Bukin <br@FreeBSD.org> |
o Don't include headers from iommu.h, include them from the header consumers instead; o Order includes properly. Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25878
|
#
15f6baf4 |
|
25-Jul-2020 |
Ruslan Bukin <br@FreeBSD.org> |
Rename DMAR flags: o DMAR_DOMAIN_* -> IOMMU_DOMAIN_* o DMAR_PGF_* -> IOMMU_PGF_* Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25812
|
#
62ad310c |
|
25-Jul-2020 |
Ruslan Bukin <br@FreeBSD.org> |
Split-out the Intel GAS (Guest Address Space) management component from Intel DMAR support, so it can be used on other IOMMU systems. Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25743
|
#
f2b2f317 |
|
21-Jul-2020 |
Ruslan Bukin <br@FreeBSD.org> |
Move the Intel DMAR busdma backend to a generic place so it can be used on other IOMMU systems. Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25720
|
#
59e37c8a |
|
14-Jul-2020 |
Ruslan Bukin <br@FreeBSD.org> |
Start splitting-out the Intel DMAR busdma backend to a generic place, so it can be used on other IOMMU systems. Provide MI iommu_unit, iommu_domain and iommu_ctx structs in sys/iommu.h; use them as a first member of MD dmar_unit, dmar_domain and dmar_ctx. Change the namespace in DMAR backend: use iommu_ prefix instead of dmar_. Move some macroses and function prototypes to sys/iommu.h. Reviewed by: kib Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D25574
|
#
685666aa |
|
18-Nov-2019 |
Konstantin Belousov <kib@FreeBSD.org> |
bus_dma_dmar_set_buswide(9): KPI to indicate that the whole dmar context should share page tables. Practically it means that dma requests from any device on the bus are translated according to the entries loaded for the bus:0:0 device. KPI requires that the slot and function of the device be 0:0, and that no tags for other devices on the bus were used. The intended use are NTBs which pass TLPs from the downstream to the host with slot:func of the downstream originator. Reviewed and tested by: mav Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D22434
|
#
b119329d |
|
25-Sep-2019 |
Mark Johnston <markj@FreeBSD.org> |
Complete the removal of the "wire_count" field from struct vm_page. Convert all remaining references to that field to "ref_count" and update comments accordingly. No functional change intended. Reviewed by: alc, kib Sponsored by: Intel, Netflix Differential Revision: https://reviews.freebsd.org/D21768
|
#
ebf5747b |
|
27-Nov-2017 |
Pedro F. Giffuni <pfg@FreeBSD.org> |
sys/x86: further adoption of SPDX licensing ID tags. Mainly focus on files that use BSD 2-Clause license, however the tool I was using misidentified many licenses so this was mostly a manual - error prone - task. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts.
|
#
0a44024a |
|
14-Aug-2015 |
Konstantin Belousov <kib@FreeBSD.org> |
Comment only change, fix grammar and somewhat clarify the action. Sponsored by: The FreeBSD Foundation MFC after: 3 days
|
#
9b3df93b |
|
20-Jul-2015 |
Konstantin Belousov <kib@FreeBSD.org> |
Typo in comment.
|
#
1abfd355 |
|
26-Jun-2015 |
Konstantin Belousov <kib@FreeBSD.org> |
Split the DMAR unit domains and contexts. Domains carry address space and related data structures. Contexts attach requests initiators to domains. There is still 1:1 correspondence between contexts and domains on the running system, since only busdma currently allocates them, using dmar_get_ctx_for_dev(). Large part of the change is formal rename of the ctx to domain, but patch also reworks the context allocation and free to allow for independent domain creation. The helper dmar_move_ctx_to_domain() is introduced for future use, to reassign request initiator from one domain to another. The hard issue which is not yet resolved with the context move is proper handling (or reserving) RMRR entries in the destination domain as required by ACPI DMAR table for moved context. Tested by: pho Sponsored by: The FreeBSD Foundation
|
#
69baeadc |
|
29-May-2015 |
Konstantin Belousov <kib@FreeBSD.org> |
Remove several write-only variables, all reported by the gcc 4.9 buildkernel run. Some of them were write-only under some kernel options, e.g. variables keeping values only used by CTR() macros. It costs nothing to the code readability and correctness to eliminate the warnings in those cases too by removing the local cached values used only for single-access. Review: https://reviews.freebsd.org/D2665 Reviewed by: rodrigc Looked at by: bjk Sponsored by: The FreeBSD Foundation MFC after: 1 week
|
#
a3d78402 |
|
23-Mar-2015 |
Konstantin Belousov <kib@FreeBSD.org> |
Assert that the mapping loop makes progress. Sponsored by: The FreeBSD Foundation MFC after: 1 week
|
#
0a110d5b |
|
19-Mar-2015 |
Konstantin Belousov <kib@FreeBSD.org> |
Use VT-d interrupt remapping block (IR) to perform FSB messages translation. In particular, despite IO-APICs only take 8bit apic id, IR translation structures accept 32bit APIC Id, which allows x2APIC mode to function properly. Extend msi_cpu of struct msi_intrsrc and io_cpu of ioapic_intsrc to full int from one byte. KPI of IR is isolated into the x86/iommu/iommu_intrmap.h, to avoid bringing all dmar headers into interrupt code. The non-PCI(e) devices which generate message interrupts on FSB require special handling. The HPET FSB interrupts are remapped, while DMAR interrupts are not. For each msi and ioapic interrupt source, the iommu cookie is added, which is in fact index of the IRE (interrupt remap entry) in the IR table. Cookie is made at the source allocation time, and then used at the map time to fill both IRE and device registers. The MSI address/data registers and IO-APIC redirection registers are programmed with the special values which are recognized by IR and used to restore the IRE index, to find proper delivery mode and target. Map all MSI interrupts in the block when msi_map() is called. Since an interrupt source setup and dismantle code are done in the non-sleepable context, flushing interrupt entries cache in the IR hardware, which is done async and ideally waits for the interrupt, requires busy-wait for queue to drain. The dmar_qi_wait_for_seq() is modified to take a boolean argument requesting busy-wait for the written sequence number instead of waiting for interrupt. Some interrupts are configured before IR is initialized, e.g. ACPI SCI. Add intr_reprogram() function to reprogram all already configured interrupts, and call it immediately before an IR unit is enabled. There is still a small window after the IO-APIC redirection entry is reprogrammed with cookie but before the unit is enabled, but to fix this properly, IR must be started much earlier. Add workarounds for 5500 and X58 northbridges, some revisions of which have severe flaws in handling IR. Use the same identification methods as employed by Linux. Review: https://reviews.freebsd.org/D1892 Reviewed by: neel Discussed with: jhb Tested by: glebius, pho (previous versions) Sponsored by: The FreeBSD Foundation MFC after: 3 weeks
|
#
6b7c46af |
|
11-Jan-2015 |
Konstantin Belousov <kib@FreeBSD.org> |
Right now, for non-coherent DMARs, page table update code flushes the cache for whole page containing modified pte, and more, only last page in the series of the consequtive pages is flushed (i.e. the affected mappings should be larger than 2MB). Avoid excessive flushing and do missed neccessary flushing, by splitting invalidation and unmapping. For now, flush exactly the range of the changed pte. This is still somewhat bigger than neccessary, since pte is 8 bytes, while cache flush line is at least 32 bytes. The originator of the issue reports that after the change, 'dmar_bus_dmamap_unload went from 13,288 cycles down to 3,257. dmar_bus_dmamap_load_buffer went from 9,686 cycles down to 3,517. and I am now able to get line 1GbE speed with Netperf TCP (even with 1K message size).' Diagnosed and tested by: Nadav Amit <nadav.amit@gmail.com> Sponsored by: The FreeBSD Foundation MFC after: 1 week
|
#
d4e70c80 |
|
09-Nov-2013 |
Dimitry Andric <dim@FreeBSD.org> |
Fix (erroneous) gcc warnings about usage of uninitialized variables in sys/x86/iommu/intel_idpgtbl.c. Reviewed by: kib
|
#
68eeb96a |
|
01-Nov-2013 |
Konstantin Belousov <kib@FreeBSD.org> |
Add support for queued invalidation. Right now, the semaphore write is scheduled after each batch, which is not optimal and must be tuned. Discussed with: alc Tested by: pho MFC after: 1 month
|
#
86be9f0d |
|
28-Oct-2013 |
Konstantin Belousov <kib@FreeBSD.org> |
Import the driver for VT-d DMAR hardware, as specified in the revision 1.3 of Intelб╝ Virtualization Technology for Directed I/O Architecture Specification. The Extended Context and PASIDs from the rev. 2.2 are not supported, but I am not aware of any released hardware which implements them. Code does not use queued invalidation, see comments for the reason, and does not provide interrupt remapping services. Code implements the management of the guest address space per domain and allows to establish and tear down arbitrary mappings, but not partial unmapping. The superpages are created as needed, but not promoted. Faults are recorded, fault records could be obtained programmatically, and printed on the console. Implement the busdma(9) using DMARs. This busdma backend avoids bouncing and provides security against misbehaving hardware and driver bad programming, preventing leaks and corruption of the memory by wild DMA accesses. By default, the implementation is compiled into amd64 GENERIC kernel but disabled; to enable, set hw.dmar.enable=1 loader tunable. Code is written to work on i386, but testing there was low priority, and driver is not enabled in GENERIC. Even with the DMAR turned on, individual devices could be directed to use the bounce busdma with the hw.busdma.pci<domain>:<bus>:<device>:<function>.bounce=1 tunable. If DMARs are capable of the pass-through translations, it is used, otherwise, an identity-mapping page table is constructed. The driver was tested on Xeon 5400/5500 chipset legacy machine, Haswell desktop and E5 SandyBridge dual-socket boxes, with ahci(4), ata(4), bce(4), ehci(4), mfi(4), uhci(4), xhci(4) devices. It also works with em(4) and igb(4), but there some fixes are needed for drivers, which are not committed yet. Intel GPUs do not work with DMAR (yet). Many thanks to John Baldwin, who explained me the newbus integration; Peter Holm, who did all testing and helped me to discover and understand several incredible bugs; and to Jim Harris for the access to the EDS and BWG and for listening when I have to explain my findings to somebody. Sponsored by: The FreeBSD Foundation MFC after: 1 month
|