#
6b252cf4 |
|
04-Aug-2023 |
Danilo Krummrich <dakr@redhat.com> |
drm/nouveau: nvkm/vmm: implement raw ops to manage uvmm The new VM_BIND UAPI uses the DRM GPU VA manager to manage the VA space. Hence, we a need a way to manipulate the MMUs page tables without going through the internal range allocator implemented by nvkm/vmm. This patch adds a raw interface for nvkm/vmm to pass the resposibility for managing the address space and the corresponding map/unmap/sparse operations to the upper layers. Reviewed-by: Dave Airlie <airlied@redhat.com> Signed-off-by: Danilo Krummrich <dakr@redhat.com> Link: https://patchwork.freedesktop.org/patch/msgid/20230804182406.5222-11-dakr@redhat.com
|
#
8f187163 |
|
30-Jun-2021 |
Alistair Popple <apopple@nvidia.com> |
nouveau/svm: implement atomic SVM access Some NVIDIA GPUs do not support direct atomic access to system memory via PCIe. Instead this must be emulated by granting the GPU exclusive access to the memory. This is achieved by replacing CPU page table entries with special swap entries that fault on userspace access. The driver then grants the GPU permission to update the page undergoing atomic access via the GPU page tables. When CPU access to the page is required a CPU fault is raised which calls into the device driver via MMU notifiers to revoke the atomic access. The original page table entries are then restored allowing CPU access to proceed. Link: https://lkml.kernel.org/r/20210616105937.23201-11-apopple@nvidia.com Signed-off-by: Alistair Popple <apopple@nvidia.com> Reviewed-by: Ben Skeggs <bskeggs@redhat.com> Cc: Christoph Hellwig <hch@lst.de> Cc: Hugh Dickins <hughd@google.com> Cc: Jason Gunthorpe <jgg@nvidia.com> Cc: John Hubbard <jhubbard@nvidia.com> Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org> Cc: Peter Xu <peterx@redhat.com> Cc: Ralph Campbell <rcampbell@nvidia.com> Cc: Shakeel Butt <shakeelb@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
#
e5c7864f |
|
22-Jun-2020 |
Ralph Campbell <rcampbell@nvidia.com> |
drm/nouveau/mmu: make nvkm_vmm_ctor() static The function nvkm_vmm_ctor() is not called outside of the file defining it, so make it static. Signed-off-by: Ralph Campbell <rcampbell@nvidia.com> Reviewed-by: John Hubbard <jhubbard@nvidia.com> Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
94db9a3b |
|
24-Apr-2020 |
Zheng Bin <zhengbin13@huawei.com> |
drm/nouveau/mmu: Remove unneeded semicolon Fixes coccicheck warning: drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h:307:2-3: Unneeded semicolon drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.c:583:2-3: Unneeded semicolon Reported-by: Hulk Robot <hulkci@huawei.com> Signed-off-by: Zheng Bin <zhengbin13@huawei.com> Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
ab2ee9ff |
|
08-May-2018 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/gp100-: support vmms with gcc/tex replayable faults enabled Some GPU units are capable of supporting "replayable" page faults, where the execution unit will wait for SW to fixup GPU page tables rather than triggering a channel-fatal fault. This feature isn't useful (it's harmful, even) unless something like HMM is being used to manage events appearing in the replayable fault buffer, so, it's disabled by default. This commit allows a client to request it be enabled. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
71871aa6 |
|
09-Jul-2018 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/gp100-: add privileged methods for fault replay/cancel Host methods exist to do at least some of what we need, but we are not currently pushing replay/cancels through a channel like UVM does as it's not clear whether it's necessary in our case (UVM also updates PTEs with the GPU). UVM also pushes a software method for fault cancels on Pascal, seemingly because the host methods don't appear to be sufficient. If/when we want to push the replay/cancel on the GPU, we can re-purpose the cancellation code here to implement that swmthd. Keep it simple for now, until we figure out exactly what we need here. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
a5ff307f |
|
06-Jul-2018 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu: add a privileged method to directly manage PTEs This provides a somewhat more direct method of manipulating the GPU page tables, which will be required to support SVM. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
2606f291 |
|
13-Jun-2018 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu: support initialisation of client-managed address-spaces NVKM is currently responsible for managing the allocation of a client's GPU address-space, but there's various use-cases (ie. HMM address-space mirroring) where giving a client more direct control is desirable. This commit allows for a VMM to be created where the area allocated for NVKM is limited to a client-specified window, the remainder of address- space is controlled directly by the client. Leaving a window is necessary to support various internal requirements, but also to support existing allocation interfaces as not all of the HW is capable of working with a HMM allocation. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
d389fd4f |
|
12-Feb-2019 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/gf100-: virtualise setting pdb base address for invalidation It appears that Pascal and newer need something different. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
874c1b56 |
|
12-Feb-2019 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/gf100-: make mmu invalidate function more general Will want to reuse this for fault replay/cancellation swmthds. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
c011b254 |
|
16-Jan-2019 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/tu102: rename implementation from tu104 Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
7986f813 |
|
10-Dec-2018 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/tu104: initial support New flush method. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
729eba33 |
|
10-Dec-2018 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu: add more general vmm free/node handling functions Aside from being a nice cleanup, these will to allow the upcoming direct page mapping interfaces to play nicely with normal mappings. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
edf50395 |
|
08-May-2018 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/gv100: initial support VEID support hacked in here, as it's the most convenient place for now. Will be refined once it's better understood. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
2ffa64eb |
|
18-Jan-2018 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/mcp77: fix regressions in stolen memory handling - Fixes addition of stolen memory base address to PTEs. - Removes support for compression. Signed-off-by: Ben Skeggs <bskeggs@redhat.com> Tested-by: Pierre Moreau <pierre.morrow@free.fr>
|
#
632b740c |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu: remove old vmm frontend Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
f9463a4b |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu: implement new vmm frontend These are the new priviledged interfaces to the VMM backends, and expose some functionality that wasn't previously available. It's now possible to allocate a chunk of address-space (even all of it), without causing page tables to be allocated up-front, and then map into it at arbitrary locations. This is the basic primitive used to support features such as sparse mapping, or to allow userspace control over its own address-space, or HMM (where the GPU driver isn't in control of the address-space layout). Rather than being tied to a subtle combination of memory object and VMA properties, arguments that control map flags (ro, kind, etc) are passed explicitly at map time. The compatibility hacks to implement the old frontend on top of the new driver backends have been replaced with something similar to implement the old frontend's interfaces on top of the new frontend. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
f9400afb |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/gp100,gp10b: implement new vmm backend Adds support for: - 64KiB/2MiB big page sizes (128KiB not supported by HW with new PT layout). - System-memory PTs. - LPTE "invalid" state. - (Tegra) Use of video memory aperture. - Sparse PDEs/PTEs. - Additional blocklinear kinds. - 49-bit address-space. GP100 supports an entirely new 5-level page table layout that provides an expanded 49-bit address-space. It also supports the layout present on previous generations, which we've been making do with until now. This commit implements support for the new layout, and enables it by default. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
b5977643 |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/gk104,gk20a: implement new vmm backend Adds support for: - 64KiB big page size. - System-memory PTs. - LPTE "invalid" state. - (Tegra) Use of video memory aperture. Adds support for marking LPTEs invalid, resulting in the corresponding SPTEs being ignored, which is supposed to speed up TLB invalidates. On The Tegra side, this will switch to using the video memory aperture for all mappings. The HW will still target non-coherent system memory, but this aperture needs to be selected in order to support compression. Tegra's instmem backend somewhat cheated to get this effect previously. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
b77791da |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/gf100: implement new vmm backend Adds support for: - 64KiB big page size. - System-memory PTs. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
dd12d158 |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/nv04: implement new vmm backend Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
eb813999 |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu: implement new vmm backend This is the common code to support a rework of the VMM backends. It adds support for more than 2 levels of page table nesting, which is required to be able to support GP100's MMU layout. Sparse mappings (that don't cause MMU faults when accessed) are now supported, where the backend provides it. Dual-PT handling had to become more sophisticated to support sparse, but this also allows us to support an optimisation the MMU provides on GK104 and newer. Certain operations can now be combined into a single page tree walk to avoid some overhead, but also enables optimsations like skipping PTE unmap writes when the PT will be destroyed anyway. The old backend has been hacked up to forward requests onto the new backend, if present, so that it's possible to bisect between issues in the backend changes vs the upcoming frontend changes. Until the new frontend has been merged, new backends will leak BAR2 page tables on module unload. This is expected, and it's not worth the effort of hacking around this as it doesn't effect runtime. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
8e39abff |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/gp100,gp10b: implement vmm on top of new base Adds support for: - Selection of old/new-style page table layout (GP100MmuLayout=0/1). - System-memory PDs. New layout disabled by default for the moment, as we don't have a backend that can handle it yet. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
5f300fed |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/gm200,gm20b: implement vmm on top of new base Adds support for: - Per-VMM selection of big page size. - System-memory PDs. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
7de078aa |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/gk104,gk20a: implement vmm on top of new base Adds support for: - Selection of a 64KiB big page size (NvFbBigPage=16). - System-memory PDs. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
540a1dde |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/gf100: implement vmm on top of new base Adds support for: - Selection of a 64KiB big page size (NvFbBigPage=16). - System-memory PDs. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
9f6219fd |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/nv50,g84: implement vmm on top of new base Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
03b0ba7b |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/nv44: implement vmm on top of new base Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
77783435 |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/nv41: implement vmm on top of new base Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
5b17f362 |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu/nv04: implement vmm on top of new base Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|
#
806a7335 |
|
31-Oct-2017 |
Ben Skeggs <bskeggs@redhat.com> |
drm/nouveau/mmu: implement base for new vm management This is the first chunk of the new VMM code that provides the structures needed to describe a GPU virtual address-space layout, as well as common interfaces to handle VMM creation, and connecting instances to a VMM. The constructor now allocates the PD itself, rather than having the user handle that manually. This won't/can't be used until after all backends have been ported to these interfaces, so a little bit of memory will be wasted on Fermi and newer for a couple of commits in the series. Compatibility has been hacked into the old code to allow each GPU backend to be ported individually. Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
|