#
4a0e7b3c |
|
26-Feb-2024 |
Christian König <ckoenig.leichtzumerken@gmail.com> |
drm/i915: fix applying placement flag Switching from a separate list to flags introduced a bug here. We were accidentially ORing the flag before initailizing the placement and not after. So this code didn't do nothing except producing a warning. Signed-off-by: Christian König <christian.koenig@amd.com> Reported-by: Stephen Rothwell <sfr@canb.auug.org.au> Fixes: a78a8da51b36 ("drm/ttm: replace busy placement with flags v6") Link: https://patchwork.freedesktop.org/patch/msgid/20240226142759.93130-1-christian.koenig@amd.com Tested-by: Stephen Rothwell <sfr@canb.auug.org.au> # compile only Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>
|
#
3c0fa9f4 |
|
02-Feb-2024 |
Ville Syrjälä <ville.syrjala@linux.intel.com> |
drm/i915: Use struct resource for memory region IO as well mem->region is a struct resource, but mem->io_start and mem->io_size are not for whatever reason. Let's unify this and convert the io stuff into a struct resource as well. Should make life a little less annoying when you don't have juggle between two different approaches all the time. Mostly done using cocci (with manual tweaks at all the places where we mutate io_size by hand): @@ struct intel_memory_region *M; expression START, SIZE; @@ - M->io_start = START; - M->io_size = SIZE; + M->io = DEFINE_RES_MEM(START, SIZE); @@ struct intel_memory_region *M; @@ - M->io_start + M->io.start @@ struct intel_memory_region M; @@ - M.io_start + M.io.start @@ expression M; @@ - M->io_size + resource_size(&M->io) @@ expression M; @@ - M.io_size + resource_size(&M.io) Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com> Acked-by: Nirmoy Das <nirmoy.das@intel.com> Tested-by: Paz Zcharya <pazz@chromium.org> Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20240202224340.30647-2-ville.syrjala@linux.intel.com
|
#
a78a8da5 |
|
13-Nov-2023 |
Somalapuram Amaranath <Amaranath.Somalapuram@amd.com> |
drm/ttm: replace busy placement with flags v6 Instead of a list of separate busy placement add flags which indicate that a placement should only be used when there is room or if we need to evict. v2: add missing TTM_PL_FLAG_IDLE for i915 v3: fix auto build test ERROR on drm-tip/drm-tip v4: fix some typos pointed out by checkpatch v5: cleanup some rebase problems with VMWGFX v6: implement some missing VMWGFX functionality pointed out by Zack, rename the flags as suggested by Michel, rebase on drm-tip and adjust XE as well Signed-off-by: Christian König <christian.koenig@amd.com> Signed-off-by: Somalapuram Amaranath <Amaranath.Somalapuram@amd.com> Reviewed-by: Zack Rusin <zack.rusin@broadcom.com> Reviewed-by: Thomas Zimmermann <tzimmermann@suse.de> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20240112125158.2748-4-christian.koenig@amd.com
|
#
47cdb66a |
|
06-Dec-2023 |
Nirmoy Das <nirmoy.das@intel.com> |
drm/i915: Ratelimit debug log in vm_fault_ttm Test like i915_gem_mman_live_selftests/igt_mmap_migrate can cause dmesg spamming. Use ratelimit api to reduce log rate. References: https://gitlab.freedesktop.org/drm/intel/-/issues/7038 Signed-off-by: Nirmoy Das <nirmoy.das@intel.com> Cc: Matthew Auld <matthew.auld@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> Signed-off-by: Andi Shyti <andi.shyti@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20231206210948.106238-3-andi.shyti@linux.intel.com
|
#
98a1dacc |
|
31-Mar-2023 |
Lee Jones <lee@kernel.org> |
drm/i915/gem/i915_gem_ttm: Demote half-filled kerneldoc Hopefully someone knowledgable will follow-up to complete it. Fixes the following W=1 kernel build warning(s): drivers/gpu/drm/i915/gem/i915_gem_ttm.c:1292: warning: Function parameter or member 'offset' not described in '__i915_gem_ttm_object_init' drivers/gpu/drm/i915/gem/i915_gem_ttm.c:1292: warning: Function parameter or member 'page_size' not described in '__i915_gem_ttm_object_init' Cc: Jani Nikula <jani.nikula@linux.intel.com> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Cc: David Airlie <airlied@gmail.com> Cc: Daniel Vetter <daniel@ffwll.ch> Cc: Sumit Semwal <sumit.semwal@linaro.org> Cc: "Christian König" <christian.koenig@amd.com> Cc: Matthew Auld <matthew.auld@intel.com> Cc: intel-gfx@lists.freedesktop.org Cc: dri-devel@lists.freedesktop.org Cc: linux-media@vger.kernel.org Cc: linaro-mm-sig@lists.linaro.org Signed-off-by: Lee Jones <lee@kernel.org> Signed-off-by: Jani Nikula <jani.nikula@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20230331092607.700644-11-lee@kernel.org
|
#
58c7ee06 |
|
29-Jan-2023 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: audit remaining bo->resource In the near future TTM will have NULL bo->resource when the object is initially created, plus after calling into pipeline-gutting. Try to handle the remaining cases. In practice NULL bo->resource should be taken to mean swapped-out or purged object. v2 (Andrzej): - Rather make i915_ttm_cpu_maps_iomem() return false with NULL resource. References: 516198d317d8 ("drm/i915: audit bo->resource usage v3") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Nirmoy Das <nirmoy.das@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20230130101230.25347-2-matthew.auld@intel.com Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com> Acked-by: Christian König <ckoenig.leichtzumerken@gmail.com> Signed-off-by: Christian König <christian.koenig@amd.com>
|
#
fde789e8 |
|
29-Jan-2023 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: fix sparse warning Sparse complains with: drivers/gpu/drm/i915/gem/i915_gem_ttm.c:1066:21: sparse: expected restricted vm_fault_t [assigned] [usertype] ret drivers/gpu/drm/i915/gem/i915_gem_ttm.c:1066:21: sparse: got int Fixes: 516198d317d8 ("drm/i915: audit bo->resource usage v3") Reported-by: kernel test robot <lkp@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20230130101230.25347-1-matthew.auld@intel.com Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Acked-by: Christian König <ckoenig.leichtzumerken@gmail.com> Signed-off-by: Christian König <christian.koenig@amd.com>
|
#
f1b17f42 |
|
29-Jan-2023 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: fix sparse warning Sparse complains with: drivers/gpu/drm/i915/gem/i915_gem_ttm.c:1066:21: sparse: expected restricted vm_fault_t [assigned] [usertype] ret drivers/gpu/drm/i915/gem/i915_gem_ttm.c:1066:21: sparse: got int Fixes: 516198d317d8 ("drm/i915: audit bo->resource usage v3") Reported-by: kernel test robot <lkp@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20230130101230.25347-1-matthew.auld@intel.com Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Acked-by: Christian König <ckoenig.leichtzumerken@gmail.com> Signed-off-by: Christian König <christian.koenig@amd.com> (cherry picked from commit fde789e8339c60c8c58e5a71fa819fcfe52d839e) Signed-off-by: Jani Nikula <jani.nikula@intel.com>
|
#
516198d3 |
|
01-Sep-2022 |
Christian König <ckoenig.leichtzumerken@gmail.com> |
drm/i915: audit bo->resource usage v3 Make sure we can at least move and alloc TT objects without backing store. v2: clear the tt object even when no resource is allocated. v3: add Matthews changes for i915 as well. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20230124125726.13323-1-christian.koenig@amd.com
|
#
18f968cb |
|
28-Dec-2022 |
Gwan-gyeong Mun <gwan-gyeong.mun@intel.com> |
drm/i915: Use error code as -E2BIG when the size of gem ttm object is too large The ttm_bo_init_reserved() functions returns -ENOSPC if the size is too big to add vma. The direct function that returns -ENOSPC is drm_mm_insert_node_in_range(). To handle the same error as other code returning -E2BIG when the size is too large, it converts return value to -E2BIG. Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Signed-off-by: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org> Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221228192252.917299-6-gwan-gyeong.mun@intel.com
|
#
6949aa0e |
|
28-Dec-2022 |
Gwan-gyeong Mun <gwan-gyeong.mun@intel.com> |
drm/i915: Check for integer truncation on the configuration of ttm place There is an impedance mismatch between the first/last valid page frame number of ttm place in unsigned and our memory/page accounting in unsigned long. As the object size is under the control of userspace, we have to be prudent and catch the conversion errors. To catch the implicit truncation as we switch from unsigned long to unsigned, we use overflows_type check and report E2BIG or overflow_type prior to the operation. v3: Not to change execution inside a macro. (Mauro) Add safe_conversion_gem_bug_on() macro and remove temporal SAFE_CONVERSION() macro. v4: Fix unhandled GEM_BUG_ON() macro call from safe_conversion_gem_bug_on() v6: Fix to follow general use case for GEM_BUG_ON(). (Jani) v7: Fix to use WARN_ON() macro where GEM_BUG_ON() macro was used. (Jani) v8: Replace safe_conversion() with check_assign() (Kees) v14: Split one macro of assignment with checking of overflow to two steps, first overflow check, and second assignment. Cc: Chris Wilson <chris@chris-wilson.co.uk> Cc: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Cc: Jani Nikula <jani.nikula@intel.com> Signed-off-by: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> (v2) Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org> (v3) Reported-by: kernel test robot <lkp@intel.com> Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com> (v5) Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221228192252.917299-4-gwan-gyeong.mun@intel.com
|
#
c3bfba9a |
|
28-Dec-2022 |
Chris Wilson <chris@chris-wilson.co.uk> |
drm/i915: Check for integer truncation on scatterlist creation There is an impedance mismatch between the scatterlist API using unsigned int and our memory/page accounting in unsigned long. That is we may try to create a scatterlist for a large object that overflows returning a small table into which we try to fit very many pages. As the object size is under the control of userspace, we have to be prudent and catch the conversion errors. To catch the implicit truncation we check before calling scattterlist creation Apis. we use overflows_type check and report E2BIG if the overflows may raise. When caller does not return errno, use WARN_ON to report a problem. This is already used in our create ioctls to indicate if the uABI request is simply too large for the backing store. Failing that type check, we have a second check at sg_alloc_table time to make sure the values we are passing into the scatterlist API are not truncated. v2: Move added i915_utils's macro into drm_util header (Jani N) v5: Fix macros to be enclosed in parentheses for complex values Fix too long line warning v8: Replace safe_conversion() with check_assign() (Kees) v14: Remove shadowing macros of scatterlist creation api and fix to explicitly overflow check where the scatterlist creation APIs are called. (Jani) v15: Add missing returning of error code when the WARN_ON() has been detected. (Jani) Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Cc: Brian Welty <brian.welty@intel.com> Cc: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Co-developed-by: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com> Signed-off-by: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org> Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com> Acked-by: Jani Nikula <jani.nikula@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221228192252.917299-3-gwan-gyeong.mun@intel.com
|
#
f47e6306 |
|
28-Dec-2022 |
Chris Wilson <chris@chris-wilson.co.uk> |
drm/i915/gem: Typecheck page lookups We need to check that we avoid integer overflows when looking up a page, and so fix all the instances where we have mistakenly used a plain integer instead of a more suitable long. Be pedantic and add integer typechecking to the lookup so that we can be sure that we are safe. And it also uses pgoff_t as our page lookups must remain compatible with the page cache, pgoff_t is currently exactly unsigned long. v2: Move added i915_utils's macro into drm_util header (Jani N) v3: Make not use the same macro name on a function. (Mauro) For kernel-doc, macros and functions are handled in the same namespace, the same macro name on a function prevents ever adding documentation for it. v4: Add kernel-doc markups to the kAPI functions and macros (Mauoro) v5: Fix an alignment to match open parenthesis v6: Rebase v10: Use assert_typable instead of exactly_pgoff_t() macro. (Kees) v11: Change the use of assert_typable to assert_same_typable (G.G) v12: Change to use static_assert(__castable_to_type(n ,T)) style since the assert_same_typable() macro has been dropped. (G.G) v13: Change the use of __castable_to_type() to castable_to_type() Remove an unnecessary header include line. (G.G) v16: Fix "ERROR:SPACING" Checkpatch report (G.G) Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Cc: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Cc: Kees Cook <keescook@chromium.org> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Co-developed-by: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com> Signed-off-by: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> (v2) Reviewed-by: Mauro Carvalho Chehab <mchehab@kernel.org> (v3) Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com> (v5) Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221228192252.917299-2-gwan-gyeong.mun@intel.com
|
#
58377de4 |
|
24-Nov-2022 |
Christian König <christian.koenig@amd.com> |
drm/i915: stop using ttm_bo_wait TTM is just wrapping core DMA functionality here, remove the mid-layer. No functional change. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221125102137.1801-7-christian.koenig@amd.com
|
#
a3185f91 |
|
09-May-2022 |
Christian König <christian.koenig@amd.com> |
drm/ttm: merge ttm_bo_api.h and ttm_bo_driver.h v2 Merge and cleanup the two headers into a single description of the object API. Also move all the documentation to the implementation and drop unnecessary includes from the header. No functional change. v2: minimal checkpatch.pl cleanup Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221125102137.1801-4-christian.koenig@amd.com
|
#
5524b5e5 |
|
15-Nov-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: never purge busy objects In i915_gem_madvise_ioctl() we immediately purge the object is not currently used, like when the mm.pages are NULL. With shmem the pages might still be hanging around or are perhaps swapped out. Similarly with ttm we might still have the pages hanging around on the ttm resource, like with lmem or shmem, but here we need to be extra careful since async unbinds are possible as well as in-progress kernel moves. In i915_ttm_purge() we expect the pipeline-gutting to nuke the ttm resource for us, however if it's busy the memory is only moved to a ghost object, which then leads to broken behaviour when for example clearing the i915_tt->filp, since the actual ttm_tt is still alive and populated, even though it's been moved to the ghost object. When we later destroy the ghost object we hit the following, since the filp is now NULL: [ +0.006982] #PF: supervisor read access in kernel mode [ +0.005149] #PF: error_code(0x0000) - not-present page [ +0.005147] PGD 11631d067 P4D 11631d067 PUD 115972067 PMD 0 [ +0.005676] Oops: 0000 [#1] PREEMPT SMP NOPTI [ +0.012962] Workqueue: events ttm_device_delayed_workqueue [ttm] [ +0.006022] RIP: 0010:i915_ttm_tt_unpopulate+0x3a/0x70 [i915] [ +0.005879] Code: 89 fb 48 85 f6 74 11 8b 55 4c 48 8b 7d 30 45 31 c0 31 c9 e8 18 6a e5 e0 80 7d 60 00 74 20 48 8b 45 68 8b 55 08 4c 89 e7 5b 5d <48> 8b 40 20 83 e2 01 41 5c 89 d1 48 8b 70 30 e9 42 b2 ff ff 4c 89 [ +0.018782] RSP: 0000:ffffc9000bf6fd70 EFLAGS: 00010202 [ +0.005244] RAX: 0000000000000000 RBX: ffff8883e12ae380 RCX: 0000000000000000 [ +0.007150] RDX: 000000008000000e RSI: ffffffff823559b4 RDI: ffff8883e12ae3c0 [ +0.007142] RBP: ffff888103b65d48 R08: 0000000000000001 R09: 0000000000000001 [ +0.007144] R10: 0000000000000001 R11: ffff88829c2c8040 R12: ffff8883e12ae3c0 [ +0.007148] R13: 0000000000000001 R14: ffff888115184140 R15: ffff888115184248 [ +0.007154] FS: 0000000000000000(0000) GS:ffff88844db00000(0000) knlGS:0000000000000000 [ +0.008108] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ +0.005763] CR2: 0000000000000020 CR3: 000000013fdb4004 CR4: 00000000003706e0 [ +0.007152] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ +0.007145] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ +0.007154] Call Trace: [ +0.002459] <TASK> [ +0.002126] ttm_tt_unpopulate.part.0+0x17/0x70 [ttm] [ +0.005068] ttm_bo_tt_destroy+0x1c/0x50 [ttm] [ +0.004464] ttm_bo_cleanup_memtype_use+0x25/0x40 [ttm] [ +0.005244] ttm_bo_cleanup_refs+0x90/0x2c0 [ttm] [ +0.004721] ttm_bo_delayed_delete+0x235/0x250 [ttm] [ +0.004981] ttm_device_delayed_workqueue+0x13/0x40 [ttm] [ +0.005422] process_one_work+0x248/0x560 [ +0.004028] worker_thread+0x4b/0x390 [ +0.003682] ? process_one_work+0x560/0x560 [ +0.004199] kthread+0xeb/0x120 [ +0.003163] ? kthread_complete_and_exit+0x20/0x20 [ +0.004815] ret_from_fork+0x1f/0x30 v2: - Just use ttm_bo_wait() directly (Niranjana) - Add testcase reference Testcase: igt@gem_madvise@dontneed-evict-race Fixes: 213d50927763 ("drm/i915/ttm: Introduce a TTM i915 gem object backend") Reported-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Andrzej Hajda <andrzej.hajda@intel.com> Cc: Nirmoy Das <nirmoy.das@intel.com> Cc: <stable@vger.kernel.org> # v5.15+ Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com> Acked-by: Nirmoy Das <Nirmoy.Das@intel.com> Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221115104620.120432-1-matthew.auld@intel.com
|
#
6915819f |
|
24-Oct-2022 |
Nirmoy Das <nirmoy.das@intel.com> |
drm/i915: Remove unwanted ghost obj check vm_fault_ttm() should not expect ttm ghost obj so remove that check. Suggested-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Nirmoy Das <nirmoy.das@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221024144558.27747-1-nirmoy.das@intel.com
|
#
8c949515 |
|
08-Nov-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915: use i915_sg_dma_sizes() for all backends We rely on page_sizes.sg in setup_scratch_page() reporting the correct value if the underlying sgl is not contiguous, however in get_pages_internal() we are only looking at the layout of the created pages when calculating the sg_page_sizes, and not the final sgl, which could in theory be completely different. In such a situation we might incorrectly think we have a 64K scratch page, when it is actually only 4K or similar split over multiple non-contiguous entries, which could lead to broken behaviour when touching the scratch space within the padding of a 64K GTT page-table. For most of the other backends we already just call i915_sg_dma_sizes() on the final mapping, so rather just move that into __i915_gem_object_set_pages() to avoid such issues coming back to bite us later. v2: Update missing conversion in gvt Suggested-by: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Stuart Summers <stuart.summers@intel.com> Cc: Andrzej Hajda <andrzej.hajda@intel.com> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221108103238.165447-1-matthew.auld@intel.com
|
#
ccb0e027 |
|
07-Nov-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: add some sanity checks for lmem_userfault_list Rather than getting some hard to debug uaf, add some warns to hopefully catch issues with userfault_count being non-zero when destroying the object. Also if we somehow add an object to lmem_userfault_list that somehow doesn't map lmem. References: https://gitlab.freedesktop.org/drm/intel/-/issues/7469 Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Anshuman Gupta <anshuman.gupta@intel.com> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com> Cc: Andrzej Hajda <andrzej.hajda@intel.com> Cc: Nirmoy Das <nirmoy.das@intel.com> Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221107165414.56970-2-matthew.auld@intel.com
|
#
625b7446 |
|
07-Nov-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: fix uaf with lmem_userfault_list handling In the fault handler, make sure we check if the BO maps lmem after we schedule the migration, since the current resource might change from lmem to smem, if the pages are in the non-cpu visible portion of lmem. This then leads to adding the object to the lmem_userfault_list even though the current resource is no longer lmem. If we then destroy the object, the list might still contain a link to the now free object, since we only remove it if the object is still in lmem. Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/7469 Fixes: ad74457a6b5a ("drm/i915/dgfx: Release mmap on rpm suspend") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Anshuman Gupta <anshuman.gupta@intel.com> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com> Cc: Andrzej Hajda <andrzej.hajda@intel.com> Cc: Nirmoy Das <nirmoy.das@intel.com> Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221107165414.56970-1-matthew.auld@intel.com
|
#
1cacd689 |
|
27-Oct-2022 |
Anshuman Gupta <anshuman.gupta@intel.com> |
drm/i915/dgfx: Grab wakeref at i915_ttm_unmap_virtual We had already grabbed the rpm wakeref at obj destruction path, but it also required to grab the wakeref when object moves. When i915_gem_object_release_mmap_offset() gets called by i915_ttm_move_notify(), it will release the mmap offset without grabbing the wakeref. We want to avoid that therefore, grab the wakeref at i915_ttm_unmap_virtual() accordingly. While doing that also changed the lmem_userfault_lock from mutex to spinlock, as spinlock widely used for list. Also changed if (obj->userfault_count) to GEM_BUG_ON(!obj->userfault_count). v2: - Removed lmem_userfault_{list,lock} from intel_gt. [Matt Auld] Fixes: ad74457a6b5a ("drm/i915/dgfx: Release mmap on rpm suspend") Suggested-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221027092242.1476080-3-anshuman.gupta@intel.com
|
#
e66c8dcf |
|
27-Oct-2022 |
Anshuman Gupta <anshuman.gupta@intel.com> |
drm/i915: Encapsulate lmem rpm stuff in intel_runtime_pm Runtime pm is not really per GT, therefore it make sense to move lmem_userfault_list, lmem_userfault_lock and userfault_wakeref from intel_gt to intel_runtime_pm structure, which is embedded to i915. No functional change. v2: - Fixes the code comment nit. [Matt Auld] Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Acked-by: Rodrigo Vivi <rodrigo.vivi@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221027092242.1476080-2-anshuman.gupta@intel.com
|
#
e3c92eb4 |
|
27-Oct-2022 |
Somalapuram Amaranath <Amaranath.Somalapuram@amd.com> |
drm/ttm: rework on ttm_resource to use size_t type Change ttm_resource structure from num_pages to size_t size in bytes. v1 -> v2: change PFN_UP(dst_mem->size) to ttm->num_pages v1 -> v2: change bo->resource->size to bo->base.size at some places v1 -> v2: remove the local variable v1 -> v2: cleanup cmp_size_smaller_first() v2 -> v3: adding missing PFN_UP in ttm_bo_vm_fault_reserved Signed-off-by: Somalapuram Amaranath <Amaranath.Somalapuram@amd.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221027091237.983582-1-Amaranath.Somalapuram@amd.com Reviewed-by: Christian König <christian.koenig@amd.com> Signed-off-by: Christian König <christian.koenig@amd.com>
|
#
78a07fe7 |
|
20-Oct-2022 |
Robert Beckett <bob.beckett@collabora.com> |
drm/i915: stop abusing swiotlb_max_segment swiotlb_max_segment used to return either the maximum size that swiotlb could bounce, or for Xen PV PAGE_SIZE even if swiotlb could bounce buffer larger mappings. This made i915 on Xen PV work as it bypasses the coherency aspect of the DMA API and can't cope with bounce buffering and this avoided bounce buffering for the Xen/PV case. So instead of adding this hack back, check for Xen/PV directly in i915 for the Xen case and otherwise use the proper DMA API helper to query the maximum mapping size. Replace swiotlb_max_segment() calls with dma_max_mapping_size(). In i915_gem_object_get_pages_internal() no longer consider max_segment only if CONFIG_SWIOTLB is enabled. There can be other (iommu related) causes of specific max segment sizes. Fixes: a2daa27c0c61 ("swiotlb: simplify swiotlb_max_segment") Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> Signed-off-by: Robert Beckett <bob.beckett@collabora.com> Signed-off-by: Christoph Hellwig <hch@lst.de> [hch: added the Xen hack, rewrote the changelog] Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221020110308.1582518-1-hch@lst.de
|
#
6667d78a |
|
14-Oct-2022 |
Nirmoy Das <nirmoy.das@intel.com> |
drm/i915: Refactor ttm ghost obj detection Currently i915_ttm_to_gem() returns NULL for ttm ghost object which makes it unclear when we should add a NULL check for a caller of i915_ttm_to_gem() as ttm ghost objects are expected behaviour for certain cases. Create a separate function to detect ttm ghost object and use that in places where we expect a ghost obj from ttm. Signed-off-by: Nirmoy Das <nirmoy.das@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221014131427.21102-1-nirmoy.das@intel.com
|
#
999f4562 |
|
04-Oct-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915: allow control over the flags when migrating In the next patch we want to move the object (if the current resource is not compatible), to the mappable part of lmem for some display buffers. Currently that requires being able to unset the I915_BO_ALLOC_GPU_ONLY hint. Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Jianshui Yu <jianshui.yu@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Nirmoy Das <nirmoy.das@intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221004131916.233474-3-matthew.auld@intel.com
|
#
26b15eb0 |
|
03-Oct-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: implement access_memory It looks like we need this for local-memory, if we want to use ptrace. Something more is still needed if we want to handle non-mappable memory, which looks quite annoying. v2: - ttm_bo_kmap doesn't seem to work well here, and seems to expect contiguous resource. v3(Andi): - s/PAGE_SIZE/bytes/ when passing in the size of the mapping. References: https://gitlab.freedesktop.org/drm/intel/-/issues/6989 Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Andrzej Hajda <andrzej.hajda@intel.com> Cc: Nirmoy Das <nirmoy.das@intel.com> Cc: Andi Shyti <andi.shyti@linux.intel.com> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221003172819.99245-1-matthew.auld@intel.com
|
#
e5cedf98 |
|
23-Sep-2022 |
Nirmoy Das <nirmoy.das@intel.com> |
drm/i915: Improve debug print in vm_fault_ttm Print the error code returned by __i915_ttm_migrate() for better debuggability. v2: Fix kernel test robot warning. v3: Fix dim checkpatch warning. References: https://gitlab.freedesktop.org/drm/intel/-/issues/6889 Acked-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Nirmoy Das <nirmoy.das@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220923143730.13498-1-nirmoy.das@intel.com
|
#
fb781898 |
|
20-Sep-2022 |
Nirmoy Das <nirmoy.das@intel.com> |
drm/i915: Do not cleanup obj with NULL bo->resource For delayed BO release i915_ttm_delete_mem_notify() gets called twice, once with proper bo->resource and another time with NULL. We shouldn't do anything for the 2nd time as we already cleaned up the obj once. References: https://gitlab.freedesktop.org/drm/intel/-/issues/6850 Fixes: ad74457a6b5a96 ("drm/i915/dgfx: Release mmap on rpm suspend") Signed-off-by: Nirmoy Das <nirmoy.das@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220920170628.3391-1-nirmoy.das@intel.com
|
#
00a6c36c |
|
15-Nov-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: never purge busy objects In i915_gem_madvise_ioctl() we immediately purge the object is not currently used, like when the mm.pages are NULL. With shmem the pages might still be hanging around or are perhaps swapped out. Similarly with ttm we might still have the pages hanging around on the ttm resource, like with lmem or shmem, but here we need to be extra careful since async unbinds are possible as well as in-progress kernel moves. In i915_ttm_purge() we expect the pipeline-gutting to nuke the ttm resource for us, however if it's busy the memory is only moved to a ghost object, which then leads to broken behaviour when for example clearing the i915_tt->filp, since the actual ttm_tt is still alive and populated, even though it's been moved to the ghost object. When we later destroy the ghost object we hit the following, since the filp is now NULL: [ +0.006982] #PF: supervisor read access in kernel mode [ +0.005149] #PF: error_code(0x0000) - not-present page [ +0.005147] PGD 11631d067 P4D 11631d067 PUD 115972067 PMD 0 [ +0.005676] Oops: 0000 [#1] PREEMPT SMP NOPTI [ +0.012962] Workqueue: events ttm_device_delayed_workqueue [ttm] [ +0.006022] RIP: 0010:i915_ttm_tt_unpopulate+0x3a/0x70 [i915] [ +0.005879] Code: 89 fb 48 85 f6 74 11 8b 55 4c 48 8b 7d 30 45 31 c0 31 c9 e8 18 6a e5 e0 80 7d 60 00 74 20 48 8b 45 68 8b 55 08 4c 89 e7 5b 5d <48> 8b 40 20 83 e2 01 41 5c 89 d1 48 8b 70 30 e9 42 b2 ff ff 4c 89 [ +0.018782] RSP: 0000:ffffc9000bf6fd70 EFLAGS: 00010202 [ +0.005244] RAX: 0000000000000000 RBX: ffff8883e12ae380 RCX: 0000000000000000 [ +0.007150] RDX: 000000008000000e RSI: ffffffff823559b4 RDI: ffff8883e12ae3c0 [ +0.007142] RBP: ffff888103b65d48 R08: 0000000000000001 R09: 0000000000000001 [ +0.007144] R10: 0000000000000001 R11: ffff88829c2c8040 R12: ffff8883e12ae3c0 [ +0.007148] R13: 0000000000000001 R14: ffff888115184140 R15: ffff888115184248 [ +0.007154] FS: 0000000000000000(0000) GS:ffff88844db00000(0000) knlGS:0000000000000000 [ +0.008108] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ +0.005763] CR2: 0000000000000020 CR3: 000000013fdb4004 CR4: 00000000003706e0 [ +0.007152] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ +0.007145] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ +0.007154] Call Trace: [ +0.002459] <TASK> [ +0.002126] ttm_tt_unpopulate.part.0+0x17/0x70 [ttm] [ +0.005068] ttm_bo_tt_destroy+0x1c/0x50 [ttm] [ +0.004464] ttm_bo_cleanup_memtype_use+0x25/0x40 [ttm] [ +0.005244] ttm_bo_cleanup_refs+0x90/0x2c0 [ttm] [ +0.004721] ttm_bo_delayed_delete+0x235/0x250 [ttm] [ +0.004981] ttm_device_delayed_workqueue+0x13/0x40 [ttm] [ +0.005422] process_one_work+0x248/0x560 [ +0.004028] worker_thread+0x4b/0x390 [ +0.003682] ? process_one_work+0x560/0x560 [ +0.004199] kthread+0xeb/0x120 [ +0.003163] ? kthread_complete_and_exit+0x20/0x20 [ +0.004815] ret_from_fork+0x1f/0x30 v2: - Just use ttm_bo_wait() directly (Niranjana) - Add testcase reference Testcase: igt@gem_madvise@dontneed-evict-race Fixes: 213d50927763 ("drm/i915/ttm: Introduce a TTM i915 gem object backend") Reported-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Andrzej Hajda <andrzej.hajda@intel.com> Cc: Nirmoy Das <nirmoy.das@intel.com> Cc: <stable@vger.kernel.org> # v5.15+ Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com> Acked-by: Nirmoy Das <Nirmoy.Das@intel.com> Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221115104620.120432-1-matthew.auld@intel.com (cherry picked from commit 5524b5e52e08f675116a93296fe5bee60bc43c03) Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
|
#
4f68332b |
|
07-Nov-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: fix uaf with lmem_userfault_list handling In the fault handler, make sure we check if the BO maps lmem after we schedule the migration, since the current resource might change from lmem to smem, if the pages are in the non-cpu visible portion of lmem. This then leads to adding the object to the lmem_userfault_list even though the current resource is no longer lmem. If we then destroy the object, the list might still contain a link to the now free object, since we only remove it if the object is still in lmem. Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/7469 Fixes: ad74457a6b5a ("drm/i915/dgfx: Release mmap on rpm suspend") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Anshuman Gupta <anshuman.gupta@intel.com> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com> Cc: Andrzej Hajda <andrzej.hajda@intel.com> Cc: Nirmoy Das <nirmoy.das@intel.com> Reviewed-by: Andrzej Hajda <andrzej.hajda@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221107165414.56970-1-matthew.auld@intel.com (cherry picked from commit 625b74460ec0978979f883fbee117e1b97e6e35e) Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
|
#
d3f6bacf |
|
20-Oct-2022 |
Robert Beckett <bob.beckett@collabora.com> |
drm/i915: stop abusing swiotlb_max_segment swiotlb_max_segment used to return either the maximum size that swiotlb could bounce, or for Xen PV PAGE_SIZE even if swiotlb could bounce buffer larger mappings. This made i915 on Xen PV work as it bypasses the coherency aspect of the DMA API and can't cope with bounce buffering and this avoided bounce buffering for the Xen/PV case. So instead of adding this hack back, check for Xen/PV directly in i915 for the Xen case and otherwise use the proper DMA API helper to query the maximum mapping size. Replace swiotlb_max_segment() calls with dma_max_mapping_size(). In i915_gem_object_get_pages_internal() no longer consider max_segment only if CONFIG_SWIOTLB is enabled. There can be other (iommu related) causes of specific max segment sizes. Fixes: a2daa27c0c61 ("swiotlb: simplify swiotlb_max_segment") Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> Signed-off-by: Robert Beckett <bob.beckett@collabora.com> Signed-off-by: Christoph Hellwig <hch@lst.de> [hch: added the Xen hack, rewrote the changelog] Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221020110308.1582518-1-hch@lst.de (cherry picked from commit 78a07fe777c42800bd1adaec12abe5dcee43919e) Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
|
#
695ddc93 |
|
04-Oct-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915: allow control over the flags when migrating In the next patch we want to move the object (if the current resource is not compatible), to the mappable part of lmem for some display buffers. Currently that requires being able to unset the I915_BO_ALLOC_GPU_ONLY hint. Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Jianshui Yu <jianshui.yu@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Cc: Nirmoy Das <nirmoy.das@intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20221004131916.233474-3-matthew.auld@intel.com (cherry picked from commit 999f4562077208b683f0519e5f1aa1e5c2fd2191) Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
|
#
86a4e84a |
|
20-Sep-2022 |
Nirmoy Das <nirmoy.das@intel.com> |
drm/i915: Do not cleanup obj with NULL bo->resource For delayed BO release i915_ttm_delete_mem_notify() gets called twice, once with proper bo->resource and another time with NULL. We shouldn't do anything for the 2nd time as we already cleaned up the obj once. References: https://gitlab.freedesktop.org/drm/intel/-/issues/6850 Fixes: ad74457a6b5a96 ("drm/i915/dgfx: Release mmap on rpm suspend") Signed-off-by: Nirmoy Das <nirmoy.das@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220920170628.3391-1-nirmoy.das@intel.com (cherry picked from commit fb7818989976317cc2e78008aa2df7b9fe423c86) Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
|
#
ad74457a |
|
13-Sep-2022 |
Anshuman Gupta <anshuman.gupta@intel.com> |
drm/i915/dgfx: Release mmap on rpm suspend Release all mmap mapping for all lmem objects which are associated with userfault such that, while pcie function in D3hot, any access to memory mappings will raise a userfault. Runtime resume the dgpu(when gem object lies in lmem). This will transition the dgpu graphics function to D0 state if it was in D3 in order to access the mmap memory mappings. v2: - Squashes the patches. [Matt Auld] - Add adequate locking for lmem_userfault_list addition. [Matt Auld] - Reused obj->userfault_count to avoid double addition. [Matt Auld] - Added i915_gem_object_lock to check i915_gem_object_is_lmem. [Matt Auld] v3: - Use i915_ttm_cpu_maps_iomem. [Matt Auld] - Fix 'ret == 0 to ret == VM_FAULT_NOPAGE'. [Matt Auld] - Reuse obj->userfault_count as a bool 0 or 1. [Matt Auld] - Delete the mmaped obj from lmem_userfault_list in obj destruction path. [Matt Auld] - Get a wakeref for object destruction patch. [Matt Auld] - Use intel_wakeref_auto to delay runtime PM. [Matt Auld] v4: - Avoid using mmo offset to get the vma_node. [Matt Auld] - Added comment to use the lmem_userfault_lock. [Matt Auld] - Get lmem_userfault_lock in i915_gem_object_release_mmap_offset. [Matt Auld] - Fixed kernel test robot generated warning. v5: - Addressed the cosmetics comments. [Andi] - Changed i915_gem_runtime_pm_object_release_mmap_offset() name to i915_gem_object_runtime_pm_release_mmap_offset() to be rhythmic. PCIe Specs 5.3.1.4.1 Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/6331 Cc: Matthew Auld <matthew.auld@intel.com> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com> Signed-off-by: Anshuman Gupta <anshuman.gupta@intel.com> Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220913152714.16541-3-anshuman.gupta@intel.com
|
#
873fef88 |
|
05-Sep-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915: consider HAS_FLAT_CCS() in needs_ccs_pages Just move the HAS_FLAT_CCS() check into needs_ccs_pages. This also then fixes i915_ttm_memcpy_allowed() which was incorrectly reporting true on DG1, even though it doesn't have small-BAR or flat-CCS. References: https://gitlab.freedesktop.org/drm/intel/-/issues/6605 Fixes: efeb3caf4341 ("drm/i915/ttm: disallow CPU fallback mode for ccs pages") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Nirmoy Das <nirmoy.das@intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220905105329.41455-1-matthew.auld@intel.com
|
#
92b2b55e |
|
20-Aug-2022 |
Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com> |
drm/i915: Implement intersect/compatible functions Implemented a new intersect and compatible callback function fetching start offset from drm buddy allocator. v3: move the bits that are specific to buddy_man (Matthew) v4: consider the block size /range (Matthew) Signed-off-by: Christian König <christian.koenig@amd.com> Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Reviewed-by: Christian König <christian.koenig@amd.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220820073304.178444-4-Arunpravin.PaneerSelvam@amd.com
|
#
347987a2 |
|
18-Feb-2022 |
Christian König <christian.koenig@amd.com> |
drm/ttm: rename and cleanup ttm_bo_init Rename ttm_bo_init to ttm_bo_init_validate since that better matches what the function is actually doing. Remove the unused size parameter, move the function's kerneldoc to the implementation and cleanup the whole error handling. Signed-off-by: Christian König <christian.koenig@amd.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220707102453.3633-2-christian.koenig@amd.com Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com>
|
#
151e0e0f |
|
05-Sep-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915: consider HAS_FLAT_CCS() in needs_ccs_pages Just move the HAS_FLAT_CCS() check into needs_ccs_pages. This also then fixes i915_ttm_memcpy_allowed() which was incorrectly reporting true on DG1, even though it doesn't have small-BAR or flat-CCS. References: https://gitlab.freedesktop.org/drm/intel/-/issues/6605 Fixes: efeb3caf4341 ("drm/i915/ttm: disallow CPU fallback mode for ccs pages") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Nirmoy Das <nirmoy.das@intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220905105329.41455-1-matthew.auld@intel.com (cherry picked from commit 873fef8833ea794526b7f4179088e565078fe0e8) Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
|
#
9306b2b2 |
|
12-Jul-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: fix 32b build Since segment_pages is no longer a compile time constant, it looks the DIV_ROUND_UP(node->size, segment_pages) breaks the 32b build. Simplest is just to use the ULL variant, but really we should need not need more than u32 for the page alignment (also we are limited by that due to the sg->length type), so also make it all u32. Reported-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Fixes: bc99f1209f19 ("drm/i915/ttm: fix sg_table construction") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Nirmoy Das <nirmoy.das@linux.intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220712174050.592550-1-matthew.auld@intel.com
|
#
bc99f120 |
|
11-Jul-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: fix sg_table construction If we encounter some monster sized local-memory page that exceeds the maximum sg length (UINT32_MAX), ensure that don't end up with some misaligned address in the entry that follows, leading to fireworks later. Also ensure we have some coverage of this in the selftests. v2(Chris): - Use round_down consistently to avoid udiv errors v3(Nirmoy): - Also update the max_segment in the selftest Fixes: f701b16d4cc5 ("drm/i915/ttm: add i915_sg_from_buddy_resource") Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/6379 Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Cc: Nirmoy Das <nirmoy.das@linux.intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220711085859.24198-1-matthew.auld@intel.com
|
#
efeb3caf |
|
29-Jun-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: disallow CPU fallback mode for ccs pages Falling back to memcpy/memset shouldn't be allowed if we know we have CCS state to manage using the blitter. Otherwise we are potentially leaving the aux CCS state in an unknown state, which smells like an info leak. Fixes: 48760ffe923a ("drm/i915/gt: Clear compress metadata for Flat-ccs objects") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Cc: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Cc: Jon Bloomfield <jon.bloomfield@intel.com> Cc: Daniel Vetter <daniel.vetter@ffwll.ch> Cc: Jordan Justen <jordan.l.justen@intel.com> Cc: Kenneth Graunke <kenneth@whitecape.org> Cc: Akeem G Abodunrin <akeem.g.abodunrin@intel.com> Cc: Ramalingam C <ramalingam.c@intel.com> Reviewed-by: Ramalingam C <ramalingam.c@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220629174350.384910-12-matthew.auld@intel.com
|
#
bfe53be2 |
|
29-Jun-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: handle blitter failure on DG2 If the move or clear operation somehow fails, and the memory underneath is not cleared, like when moving to lmem, then we currently fallback to memcpy or memset. However with small-BAR systems this fallback might no longer be possible. For now we use the set_wedged sledgehammer if we ever encounter such a scenario, and mark the object as borked to plug any holes where access to the memory underneath can happen. Add some basic selftests to exercise this. v2: - In the selftests make sure we grab the runtime pm around the reset. Also make sure we grab the reset lock before checking if the device is wedged, since the wedge might still be in-progress and hence the bit might not be set yet. - Don't wedge or put the object into an unknown state, if the request construction fails (or similar). Just returning an error and skipping the fallback should be safe here. - Make sure we wedge each gt. (Thomas) - Peek at the unknown_state in io_reserve, that way we don't have to export or hand roll the fault_wait_for_idle. (Thomas) - Add the missing read-side barriers for the unknown_state. (Thomas) - Some kernel-doc fixes. (Thomas) v3: - Tweak the ordering of the set_wedged, also add FIXME. Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Cc: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Cc: Jon Bloomfield <jon.bloomfield@intel.com> Cc: Daniel Vetter <daniel.vetter@ffwll.ch> Cc: Jordan Justen <jordan.l.justen@intel.com> Cc: Kenneth Graunke <kenneth@whitecape.org> Cc: Akeem G Abodunrin <akeem.g.abodunrin@intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220629174350.384910-11-matthew.auld@intel.com
|
#
ced7866d |
|
12-Jul-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: fix 32b build Since segment_pages is no longer a compile time constant, it looks the DIV_ROUND_UP(node->size, segment_pages) breaks the 32b build. Simplest is just to use the ULL variant, but really we should need not need more than u32 for the page alignment (also we are limited by that due to the sg->length type), so also make it all u32. Reported-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Fixes: aff1e0b09b54 ("drm/i915/ttm: fix sg_table construction") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Nirmoy Das <nirmoy.das@linux.intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220712174050.592550-1-matthew.auld@intel.com (cherry picked from commit 9306b2b2dfce6931241ef804783692cee526599c) Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
|
#
aff1e0b0 |
|
11-Jul-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: fix sg_table construction If we encounter some monster sized local-memory page that exceeds the maximum sg length (UINT32_MAX), ensure that don't end up with some misaligned address in the entry that follows, leading to fireworks later. Also ensure we have some coverage of this in the selftests. v2(Chris): - Use round_down consistently to avoid udiv errors v3(Nirmoy): - Also update the max_segment in the selftest Fixes: f701b16d4cc5 ("drm/i915/ttm: add i915_sg_from_buddy_resource") Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/6379 Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Cc: Nirmoy Das <nirmoy.das@linux.intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220711085859.24198-1-matthew.auld@intel.com (cherry picked from commit bc99f1209f19fefa3ee11e77464ccfae541f4291) Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
|
#
76a6d563 |
|
05-Apr-2022 |
Ramalingam C <ramalingam.c@intel.com> |
drm/i915/gem: Add extra pages in ttm_tt for ccs data On Xe-HP and later devices, dedicated compression control state (CCS) stored in local memory is used for each surface, to support the 3D and media compression formats. The memory required for the CCS of the entire local memory is 1/256 of the local memory size. So before the kernel boot, the required memory is reserved for the CCS data and a secure register will be programmed with the CCS base address So when an object is allocated in local memory, dont need to explicitly allocate the space for ccs data. But when the obj is evicted into the smem, to hold the compression related data along with the obj extra space is needed in smem. i.e obj_size + (obj_size/256). Hence when a smem pages are allocated for an obj with lmem placement possibility we create with the extra pages required for the ccs data for the obj size. v2: Used imperative wording [Thomas] v3: Inflate the pages only when obj's placement is lmem only v4: GEM_BUG_ON if the ttm->num_pages > obj page size [Thomas] Signed-off-by: Ramalingam C <ramalingam.c@intel.com> cc: Christian Koenig <christian.koenig@amd.com> cc: Hellstrom Thomas <thomas.hellstrom@intel.com> Reviewed-by: Thomas Hellstrom <thomas.hellstrom@linux.intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220405150840.29351-9-ramalingam.c@intel.com
|
#
e36764ec |
|
01-Apr-2022 |
Ramalingam C <ramalingam.c@intel.com> |
drm/ttm: Add a parameter to add extra pages into ttm_tt Add a parameter called "extra_pages" for ttm_tt_init, to indicate that driver needs extra pages in ttm_tt. v2: Used imperative wording [Thomas and Christian] Signed-off-by: Ramalingam C <ramalingam.c@intel.com> cc: Christian Koenig <christian.koenig@amd.com> cc: Hellstrom Thomas <thomas.hellstrom@intel.com> Reviewed-by: Thomas Hellstrom <thomas.hellstrom@linux.intel.com> Reviewed-by: Christian Konig <christian.koenig@amd.com> Reviewed-by: Nirmoy Das <nirmoy.das@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220401123751.27771-8-ramalingam.c@intel.com
|
#
fee2ede1 |
|
24-Jan-2022 |
Christian König <christian.koenig@amd.com> |
drm/ttm: rework bulk move handling v5 Instead of providing the bulk move structure for each LRU update set this as property of the BO. This should avoid costly bulk move rebuilds with some games under RADV. v2: some name polishing, add a few more kerneldoc words. v3: add some lockdep v4: fix bugs, handle pin/unpin as well v5: improve kerneldoc Signed-off-by: Christian König <christian.koenig@amd.com> Tested-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: https://patchwork.freedesktop.org/patch/msgid/20220321132601.2161-5-christian.koenig@amd.com
|
#
66ddc693 |
|
24-Mar-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: limit where we apply TTM_PL_FLAG_CONTIGUOUS We only need this when allocating device local-memory, where this influences the drm_buddy. Currently there is some funny behaviour where an "in limbo" system memory object is lacking the relevant placement flags etc. before we first allocate the ttm_tt, leading to ttm performing a move when not needed, since the current placement is seen as not compatible. Suggested-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Fixes: 2ed38cec5606 ("drm/i915: opportunistically apply ALLOC_CONTIGIOUS") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Nirmoy Das <nirmoy.das@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Acked-by: Nirmoy Das <nirmoy.das@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220324172143.377104-1-matthew.auld@intel.com
|
#
6a9b0289 |
|
16-Jul-2021 |
Christian König <christian.koenig@amd.com> |
drm/ttm: move the LRU into resource handling v4 This way we finally fix the problem that new resource are not immediately evict-able after allocation. That has caused numerous problems including OOM on GDS handling and not being able to use TTM as general resource manager. v2: stop assuming in ttm_resource_fini that res->bo is still valid. v3: cleanup kerneldoc, add more lockdep annotation v4: consistently use res->num_pages Signed-off-by: Christian König <christian.koenig@amd.com> Tested-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: https://patchwork.freedesktop.org/patch/msgid/20220321132601.2161-1-christian.koenig@amd.com
|
#
ecbf2060 |
|
15-Mar-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: wire up the object offset For the ttm backend we can use existing placements fpfn and lpfn to force the allocator to place the object at the requested offset, potentially evicting stuff if the spot is currently occupied. Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220315181425.576828-5-matthew.auld@intel.com
|
#
9b78b5da |
|
15-Mar-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915: add i915_gem_object_create_region_at() Add a generic interface for allocating an object at some specific offset, and convert stolen over. Later we will want to hook this up to different backends. Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Nirmoy Das <nirmoy.das@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220315181425.576828-4-matthew.auld@intel.com
|
#
503725c2 |
|
27-Feb-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: mappable migration on fault The end goal is to have userspace tell the kernel what buffers will require CPU access, however if we ever reach the CPU fault handler, and the current resource is not mappable, then we should attempt to migrate the buffer to the mappable portion of LMEM, or even system memory, if the allowable placements permit it. Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220228123607.580432-2-matthew.auld@intel.com
|
#
93735059 |
|
27-Feb-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: make eviction mappable aware If we need to make room for some mappable object, then we should only victimize objects that have one or pages that occupy the visible portion of LMEM. Let's also create a new priority hint for objects that are placed in mappable memory, where we know that CPU access was requested, that way we hopefully victimize these last. v2(Thomas): s/TTM_PL_PRIV/I915_PL_LMEM0/ Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220228123607.580432-1-matthew.auld@intel.com
|
#
30b9d1b3 |
|
25-Feb-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915: add I915_BO_ALLOC_GPU_ONLY If the user doesn't require CPU access for the buffer, then ALLOC_GPU_ONLY should be used, in order to prioritise allocating in the non-mappable portion of LMEM, on devices with small BAR. v2(Thomas): - The BO_ALLOC_TOPDOWN naming here is poor, since this is pure lies on systems that don't even have small BAR. A better name is GPU_ONLY, which is accurate regardless of the configuration. Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Acked-by: Nirmoy Das <nirmoy.das@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220225145502.331818-3-matthew.auld@intel.com
|
#
3312a4ac |
|
25-Feb-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: require mappable by default On devices with non-mappable LMEM ensure we always allocate the pages within the mappable portion. For now we assume that all LMEM buffers will require CPU access, which is also inline with pretty much all current kernel internal users. In the next patch we will introduce a new flag to override this behaviour. Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Acked-by: Nirmoy Das <nirmoy.das@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220225145502.331818-2-matthew.auld@intel.com
|
#
235582ca |
|
25-Feb-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915: add io_size plumbing With small LMEM-BAR we need to be able to differentiate between the total size of LMEM, and how much of it is CPU mappable. The end goal is to be able to utilize the entire range, even if part of is it not CPU accessible. v2: also update intelfb_create Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Acked-by: Nirmoy Das <nirmoy.das@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220225145502.331818-1-matthew.auld@intel.com
|
#
82508de2 |
|
10-Feb-2022 |
Jani Nikula <jani.nikula@intel.com> |
drm/i915: include shmem_fs.h only where needed Don't include shmem_fs.h in i915_drv.h, reducing the build dependencies. Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Cc: Daniel Vetter <daniel.vetter@ffwll.ch> Signed-off-by: Jani Nikula <jani.nikula@intel.com> Acked-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/44eade17f7ba1480d67c584466eeea3553f31506.1644507885.git.jani.nikula@intel.com
|
#
ba2c5d15 |
|
09-Feb-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: tweak priority hint selection For some reason we are selecting PRIO_HAS_PAGES when we don't have mm.pages, and vice versa. v2(Thomas): - Add missing fixes tag Fixes: 213d50927763 ("drm/i915/ttm: Introduce a TTM i915 gem object backend") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220209111652.468762-1-matthew.auld@intel.com
|
#
ab4911b7 |
|
06-Jan-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: ensure we unmap when purging Purging can happen during swapping out, or directly invoked with the madvise ioctl. In such cases this doesn't involve a ttm move, which skips umapping the object. v2(Thomas): - add ttm_truncate helper, and just call into i915_ttm_move_notify() to handle the unmapping step Fixes: 213d50927763 ("drm/i915/ttm: Introduce a TTM i915 gem object backend") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220106174910.280616-4-matthew.auld@intel.com
|
#
903e0387 |
|
06-Jan-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: add unmap_virtual callback Ensure we call ttm_bo_unmap_virtual when releasing the pages. Importantly this should now handle the ttm swapping case, and all other places that already call into i915_ttm_move_notify(). v2: fix up the selftest Fixes: cf3e3e86d779 ("drm/i915: Use ttm mmap handling for ttm bo's.") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220106174910.280616-3-matthew.auld@intel.com
|
#
f3cb4a2d |
|
06-Jan-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: only fault WILLNEED objects Don't attempt to fault and re-populate purged objects. By some fluke this passes the dontneed-after-mmap IGT, but for the wrong reasons. Fixes: cf3e3e86d779 ("drm/i915: Use ttm mmap handling for ttm bo's.") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220106174910.280616-2-matthew.auld@intel.com
|
#
ffa3fe08 |
|
15-Dec-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915: clean up shrinker_release_pages Add some proper flags for the different modes, and shorten the name to something more snappy. Suggested-by: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211215110746.865-2-matthew.auld@intel.com
|
#
0bdc0a06 |
|
09-Feb-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: tweak priority hint selection For some reason we are selecting PRIO_HAS_PAGES when we don't have mm.pages, and vice versa. v2(Thomas): - Add missing fixes tag Fixes: 213d50927763 ("drm/i915/ttm: Introduce a TTM i915 gem object backend") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220209111652.468762-1-matthew.auld@intel.com (cherry picked from commit ba2c5d15022a565da187d90e2fe44768e33e5034) Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
|
#
6ef295e3 |
|
06-Jan-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: ensure we unmap when purging Purging can happen during swapping out, or directly invoked with the madvise ioctl. In such cases this doesn't involve a ttm move, which skips umapping the object. v2(Thomas): - add ttm_truncate helper, and just call into i915_ttm_move_notify() to handle the unmapping step Fixes: 213d50927763 ("drm/i915/ttm: Introduce a TTM i915 gem object backend") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220106174910.280616-4-matthew.auld@intel.com (cherry picked from commit ab4911b7d411ab2ef3b38322178b9138e156c393) Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
|
#
8ee262ba |
|
06-Jan-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: add unmap_virtual callback Ensure we call ttm_bo_unmap_virtual when releasing the pages. Importantly this should now handle the ttm swapping case, and all other places that already call into i915_ttm_move_notify(). v2: fix up the selftest Fixes: cf3e3e86d779 ("drm/i915: Use ttm mmap handling for ttm bo's.") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220106174910.280616-3-matthew.auld@intel.com (cherry picked from commit 903e0387270eef14a711c0feb23b7bf62d2480df) Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
|
#
03ee5956 |
|
06-Jan-2022 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: only fault WILLNEED objects Don't attempt to fault and re-populate purged objects. By some fluke this passes the dontneed-after-mmap IGT, but for the wrong reasons. Fixes: cf3e3e86d779 ("drm/i915: Use ttm mmap handling for ttm bo's.") Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220106174910.280616-2-matthew.auld@intel.com (cherry picked from commit f3cb4a2de5410147b53e53416a3af0ffe26b5f4e) Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
|
#
5719d4fe |
|
10-Dec-2021 |
Robert Beckett <bob.beckett@collabora.com> |
drm/i915/ttm: fix large buffer population trucation ttm->num_pages is uint32_t which was causing very large buffers to only populate a truncated size. This fixes gem_create@create-clear igt test on large memory systems. Fixes: 7ae034590cea ("drm/i915/ttm: add tt shmem backend") Signed-off-by: Robert Beckett <bob.beckett@collabora.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211210195005.2582884-1-bob.beckett@collabora.com
|
#
6385eb7a |
|
22-Nov-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Implement asynchronous TTM moves Don't wait sync while migrating, but rather make the GPU blit await the dependencies and add a moving fence to the object. This also enables asynchronous VRAM management in that on eviction, rather than waiting for the moving fence to expire before freeing VRAM, it is freed immediately and the fence is stored with the VRAM manager and handed out to newly allocated objects to await before clears and swapins, or for kernel objects before setting up gpu vmas or mapping. To collect dependencies before migrating, add a set of utilities that coalesce these to a single dma_fence. What is still missing for fully asynchronous operation is asynchronous vma unbinding, which is still to be implemented. This commit substantially reduces execution time in the gem_lmem_swapping test. v2: - Make a couple of functions static. v4: - Fix some style issues (Matthew Auld) - Audit and add more checks for ghost objects (Matthew Auld) - Add more documentation for the i915_deps utility (Mattew Auld) - Simplify the i915_deps_sync() function v6: - Re-check for fence signaled before returning -EBUSY (Matthew Auld) - Use dma_resv_iter_is_exclusive() (Matthew Auld) - Await all dma-resv fences before a migration blit (Matthew Auld) Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211122214554.371864-6-thomas.hellstrom@linux.intel.com
|
#
004746e4 |
|
22-Nov-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Correctly handle waiting for gpu when shrinking With async migration, the shrinker may end up wanting to release the pages of an object while the migration blit is still running, since the GT migration code doesn't set up VMAs and the shrinker is thus oblivious to the fact that the GPU is still using the pages. Add waiting for gpu in the shrinker_release_pages() op and an argument to that function indicating whether the shrinker expects it to not wait for gpu. In the latter case the shrinker_release_pages() op will return -EBUSY if the object is not idle. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211122214554.371864-5-thomas.hellstrom@linux.intel.com
|
#
8b1f7f92 |
|
22-Nov-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Drop region reference counting There is an interesting refcounting loop: struct intel_memory_region has a struct ttm_resource_manager, ttm_resource_manager->move may hold a reference to i915_request, i915_request may hold a reference to intel_context, intel_context may hold a reference to drm_i915_gem_object, drm_i915_gem_object may hold a reference to intel_memory_region. Break this loop by dropping region reference counting. In addition, Have regions with a manager moving fence make sure that all region objects are released before freeing the region. v6: - Fix a code comment. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211122214554.371864-4-thomas.hellstrom@linux.intel.com
|
#
05d1c761 |
|
22-Nov-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Move the i915_gem_obj_copy_ttm() function Move the i915_gem_obj_copy_ttm() function to i915_gem_ttm_move.h. This will help keep a number of functions static when introducing async moves. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211122214554.371864-3-thomas.hellstrom@linux.intel.com
|
#
be373fad |
|
22-Nov-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: fixup build failure drm-intel-gt-next fails to build with: drivers/gpu/drm/i915/gem/i915_gem_ttm.c: In function ‘vm_fault_ttm’: drivers/gpu/drm/i915/gem/i915_gem_ttm.c:862:23: error: too many arguments to function ‘ttm_bo_vm_fault_reserved’ 862 | ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot, | ^~~~~~~~~~~~~~~~~~~~~~~ Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211123125814.1703220-1-matthew.auld@intel.com
|
#
6164807d |
|
22-Nov-2021 |
Dan Carpenter <dan.carpenter@oracle.com> |
drm/i915/ttm: Fix error code in i915_ttm_eviction_valuable() This function returns a bool type so returning -EBUSY is equivalent to returning true. It should return false instead. Fixes: 7ae034590cea ("drm/i915/ttm: add tt shmem backend") Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211122061438.GA2492@kili
|
#
d3cb30f8 |
|
10-Nov-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Fix illegal addition to shrinker list There's a small window of opportunity during which the adjust_lru() function can be called with a GEM refcount of zero from the TTM eviction code. This results in a kernel BUG(). Ensure that we don't attempt to modify the GEM shrinker lists unless we have a GEM refcount. Fixes: ebd4a8ec7799 ("drm/i915/ttm: move shrinker management into adjust_lru") Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211110085527.1033475-1-thomas.hellstrom@linux.intel.com
|
#
3589fdbd |
|
03-Nov-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Reorganize the ttm move code We are about to introduce failsafe- and asynchronous migration and ttm moves. This will add complexity and code to the TTM move code so it makes sense to split it out to a separate file to make the i915 TTM code easer to digest. Split the i915 TTM move code out and since we will have to change the name of the gpu_binds_iomem() and cpu_maps_iomem() functions anyway, we alter the name of gpu_binds_iomem() to i915_ttm_gtt_binds_lmem() which is more reflecting what it is used for. With this we also add some more documentation. Otherwise there should be no functional change. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211104110718.688420-2-thomas.hellstrom@linux.intel.com
|
#
cad7109a |
|
01-Nov-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915: Introduce refcounted sg-tables As we start to introduce asynchronous failsafe object migration, where we update the object state and then submit asynchronous commands we need to record what memory resources are actually used by various part of the command stream. Initially for three purposes: 1) Error capture. 2) Asynchronous migration error recovery. 3) Asynchronous vma bind. At the time where these happens, the object state may have been updated to be several migrations ahead and object sg-tables discarded. In order to make it possible to keep sg-tables with memory resource information for these operations, introduce refcounted sg-tables that aren't freed until the last user is done with them. The alternative would be to reference information sitting on the corresponding ttm_resources which typically have the same lifetime as these refcountes sg_tables, but that leads to other awkward constructs: Due to the design direction chosen for ttm resource managers that would lead to diamond-style inheritance, the LMEM resources may sometimes be prematurely freed, and finally the subclassed struct ttm_resource would have to bleed into the asynchronous vma bind code. v3: - Address a number of style issues (Matthew Auld) v4: - Dont check for st->sgl being NULL in i915_ttm_tt__shmem_unpopulate(), that should never happen. (Matthew Auld) v5: - Fix a Potential double-free (Matthew Auld) Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211101122444.114607-1-thomas.hellstrom@linux.intel.com
|
#
5d12ffe6 |
|
18-Oct-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: enable shmem tt backend Turn on the shmem tt backend, and enable shrinking. Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211018091055.1998191-8-matthew.auld@intel.com
|
#
2eda4fc6 |
|
18-Oct-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: use cached system pages when evicting lmem This should let us do an accelerated copy directly to the shmem pages when temporarily moving lmem-only objects, where the i915-gem shrinker can later kick in to swap out the pages, if needed. Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211018091055.1998191-7-matthew.auld@intel.com
|
#
ebd4a8ec |
|
18-Oct-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: move shrinker management into adjust_lru We currently just evict lmem objects to system memory when under memory pressure. For this case we might lack the usual object mm.pages, which effectively hides the pages from the i915-gem shrinker, until we actually "attach" the TT to the object, or in the case of lmem-only objects it just gets migrated back to lmem when touched again. For all cases we can just adjust the i915 shrinker LRU each time we also adjust the TTM LRU. The two cases we care about are: 1) When something is moved by TTM, including when initially populating an object. Importantly this covers the case where TTM moves something from lmem <-> smem, outside of the normal get_pages() interface, which should still ensure the shmem pages underneath are reclaimable. 2) When calling into i915_gem_object_unlock(). The unlock should ensure the object is removed from the shinker LRU, if it was indeed swapped out, or just purged, when the shrinker drops the object lock. v2(Thomas): - Handle managing the shrinker LRU in adjust_lru, where it is always safe to touch the object. v3(Thomas): - Pretty much a re-write. This time piggy back off the shrink_pin stuff, which actually seems to fit quite well for what we want here. v4(Thomas): - Just use a simple boolean for tracking ttm_shrinkable. v5: - Ensure we call adjust_lru when faulting the object, to ensure the pages are visible to the shrinker, if needed. - Add back the adjust_lru when in i915_ttm_move (Thomas) v6(Reported-by: kernel test robot <lkp@intel.com>): - Remove unused i915_tt Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> #v4 Link: https://patchwork.freedesktop.org/patch/msgid/20211018091055.1998191-6-matthew.auld@intel.com
|
#
7ae03459 |
|
18-Oct-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: add tt shmem backend For cached objects we can allocate our pages directly in shmem. This should make it possible(in a later patch) to utilise the existing i915-gem shrinker code for such objects. For now this is still disabled. v2(Thomas): - Add optional try_to_writeback hook for objects. Importantly we need to check if the object is even still shrinkable; in between us dropping the shrinker LRU lock and acquiring the object lock it could for example have been moved. Also we need to differentiate between "lazy" shrinking and the immediate writeback mode. Also later we need to handle objects which don't even have mm.pages, so bundling this into put_pages() would require somehow handling that edge case, hence just letting the ttm backend handle everything in try_to_writeback doesn't seem too bad. v3(Thomas): - Likely a bad idea to touch the object from the unpopulate hook, since it's not possible to hold a reference, without also creating circular dependency, so likely this is too fragile. For now just ensure we at least mark the pages as dirty/accessed when called from the shrinker on WILLNEED objects. - s/try_to_writeback/shrinker_release_pages, since this can do more than just writeback. - Get rid of do_backup boolean and just set the SWAPPED flag prior to calling unpopulate. - Keep shmem_tt as lowest priority for the TTM LRU bo_swapout walk, since these just get skipped anyway. We can try to come up with something better later. v4(Thomas): - s/PCI_DMA/DMA/. Also drop NO_KERNEL_MAPPING and NO_WARN, which apparently doesn't do anything with streaming mappings. - Just pass along the error for ->truncate, and assume nothing. Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Cc: Christian König <christian.koenig@amd.com> Cc: Oak Zeng <oak.zeng@intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Acked-by: Oak Zeng <oak.zeng@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20211018091055.1998191-2-matthew.auld@intel.com
|
#
068396bb |
|
30-Sep-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Rework object initialization slightly We may end up in i915_ttm_bo_destroy() in an error path before the object is fully initialized. In that case it's not correct to call __i915_gem_free_object(), because that function a) Assumes the gem object refcount is 0, which it isn't. b) frees the placements which are owned by the caller until the init_object() region ops returns successfully. Fix this by providing a lightweight cleanup function __i915_gem_object_fini() which is also called by __i915_gem_free_object(). While doing this, also make sure we call dma_resv_fini() as part of ordinary object destruction and not from the RCU callback that frees the object. This will help track down bugs where the object is incorrectly locked from an RCU lookup. Finally, make sure the object isn't put on the region list until it's either locked or fully initialized in order to block list processing of partially initialized objects. v2: - The TTM object backend memory was freed before the gem pages were put. Separate this functionality into __i915_gem_object_pages_fini() and call it from the TTM delete_mem_notify() callback. v3: - Include i915_gem_object_free_mmaps() in __i915_gem_object_pages_fini() to make sure we don't inadvertedly introduce a race. Fixes: 48b096126954 ("drm/i915: Move __i915_gem_free_object to ttm_bo_destroy") Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> #v1 Link: https://patchwork.freedesktop.org/patch/msgid/20210930113236.583531-1-thomas.hellstrom@linux.intel.com
|
#
43d46f0b |
|
29-Sep-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/ttm: s/FLAG_SG/FLAG_EXTERNAL/ It covers more than just ttm_bo_type_sg usage, like with say dma-buf, since one other user is userptr in amdgpu, and in the future we might have some more. Hence EXTERNAL is likely a more suitable name. v2(Christian): - Rename these to TTM_TT_FLAGS_* - Fix up all the holes in the flag values Suggested-by: Christian König <christian.koenig@amd.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Cc: Christian König <christian.koenig@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210929132629.353541-1-matthew.auld@intel.com Signed-off-by: Christian König <christian.koenig@amd.com>
|
#
c56ce956 |
|
22-Sep-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915 Implement LMEM backup and restore for suspend / resume Just evict unpinned objects to system. For pinned LMEM objects, make a backup system object and blit the contents to that. Backup is performed in three steps, 1: Opportunistically evict evictable objects using the gpu blitter. 2: After gt idle, evict evictable objects using the gpu blitter. This will be modified in an upcoming patch to backup pinned objects that are not used by the blitter itself. 3: Backup remaining pinned objects using memcpy. Also move uC suspend to after 2) to make sure we have a functional GuC during 2) if using GuC submission. v2: - Major refactor to make sure gem_exec_suspend@hang-SX subtests work, and suspend / resume works with a slightly modified GuC submission enabling patch series. v3: - Fix a potential use-after-free (Matthew Auld) - Use i915_gem_object_create_shmem() instead of i915_gem_object_create_region (Matthew Auld) - Minor simplifications (Matthew Auld) - Fix up kerneldoc for i195_ttm_restore_region(). - Final lmem_suspend() call moved to i915_gem_backup_suspend from i915_gem_suspend_late, since the latter gets called at driver unload and we don't unnecessarily want to run it at that time. v4: - Interface change of ttm- & lmem suspend / resume functions to use flags rather than bools. (Matthew Auld) - Completely drop the i915_gem_backup_suspend change (Matthew Auld) Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210922062527.865433-5-thomas.hellstrom@linux.intel.com
|
#
0d938863 |
|
22-Sep-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Implement a function to copy the contents of two TTM-based objects When backing up or restoring contents of pinned objects at suspend / resume time we need to allocate a new object as the backup. Add a function to facilitate copies between the two. Some data needs to be copied before the migration context is ready for operation, so make sure we can disable accelerated copies. v2: - Fix a missing return value check (Matthew Auld) Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210922062527.865433-2-thomas.hellstrom@linux.intel.com
|
#
48b09612 |
|
30-Aug-2021 |
Maarten Lankhorst <maarten.lankhorst@linux.intel.com> |
drm/i915: Move __i915_gem_free_object to ttm_bo_destroy When we implement delayed destroy, we may have a second call to the delete_mem_notify() handler, while free_object() only should be called once. Move it to bo->destroy(), to ensure it's only called once. This fixes some weird memory corruption issues with delayed destroy when async eviction is used. Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210830121006.2978297-2-maarten.lankhorst@linux.intel.com Fixes: 213d50927763 ("drm/i915/ttm: Introduce a TTM i915 gem object backend") Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
|
#
66907633 |
|
13-Aug-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/ttm, drm/i915: Update ttm_move_memcpy for async use The buffer object argument to ttm_move_memcpy was only used to determine whether the destination memory should be cleared only or whether we should copy data. Replace it with a "clear" bool, and update the callers. The intention here is to be able to use ttm_move_memcpy() async under a dma-fence as a fallback if an accelerated blit fails in a security- critical path where data might leak if the blit is not properly performed. For that purpose the bo is an unsuitable argument since its relevant members might already have changed at call time. Finally, update the ttm_move_memcpy kerneldoc that seems to have ended up with a stale version. Cc: Christian König <christian.koenig@amd.com> Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Reviewed-by: Christian König <christian.koenig@amd.com> Link: https://lore.kernel.org/r/20210813144331.372957-3-thomas.hellstrom@linux.intel.com Link: https://patchwork.freedesktop.org/patch/msgid/20210813144331.372957-3-thomas.hellstrom@linux.intel.com
|
#
d8ac30fd |
|
13-Aug-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Reorganize the ttm move code somewhat In order to make the code a bit more readable and to facilitate async memcpy moves, reorganize the move code a little. Determine at an early stage whether to copy or to clear. v2: - Don't set up the memcpy iterators unless we are actually going to memcpy. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Link: https://lore.kernel.org/r/20210813144331.372957-2-thomas.hellstrom@linux.intel.com Link: https://patchwork.freedesktop.org/patch/msgid/20210813144331.372957-2-thomas.hellstrom@linux.intel.com
|
#
d5f45d1e |
|
28-Jul-2021 |
Christian König <ckoenig.leichtzumerken@gmail.com> |
drm/ttm: remove ttm_tt_destroy_common v2 Move the functionality into ttm_tt_fini and ttm_bo_tt_destroy instead. We don't need this any more since we removed the unbind from the destroy code paths in the drivers. Also add a warning to ttm_tt_fini() if we try to fini a still populated TT object. v2: instead of reverting the patch move the functionality to different places. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210728130552.2074-5-christian.koenig@amd.com
|
#
f9b23c15 |
|
30-Aug-2021 |
Maarten Lankhorst <maarten.lankhorst@linux.intel.com> |
drm/i915: Move __i915_gem_free_object to ttm_bo_destroy When we implement delayed destroy, we may have a second call to the delete_mem_notify() handler, while free_object() only should be called once. Move it to bo->destroy(), to ensure it's only called once. This fixes some weird memory corruption issues with delayed destroy when async eviction is used. Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210830121006.2978297-2-maarten.lankhorst@linux.intel.com Fixes: 213d50927763 ("drm/i915/ttm: Introduce a TTM i915 gem object backend") Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> (cherry picked from commit 48b0961269546716c3232748bf37e64e49fb866c) Signed-off-by: Jani Nikula <jani.nikula@intel.com>
|
#
23852bec |
|
02-Sep-2021 |
Linus Torvalds <torvalds@linux-foundation.org> |
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma Pull rdma updates from Jason Gunthorpe: "This is quite a small cycle, no major series stands out. The HNS and rxe drivers saw the most activity this cycle, with rxe being broken for a good chunk of time. The significant deleted line count is due to a SPDX cleanup series. Summary: - Various cleanup and small features for rtrs - kmap_local_page() conversions - Driver updates and fixes for: efa, rxe, mlx5, hfi1, qed, hns - Cache the IB subnet prefix - Rework how CRC is calcuated in rxe - Clean reference counting in iwpm's netlink - Pull object allocation and lifecycle for user QPs to the uverbs core code - Several small hns features and continued general code cleanups - Fix the scatterlist confusion of orig_nents/nents introduced in an earlier patch creating the append operation" * tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma: (90 commits) RDMA/mlx5: Relax DCS QP creation checks RDMA/hns: Delete unnecessary blank lines. RDMA/hns: Encapsulate the qp db as a function RDMA/hns: Adjust the order in which irq are requested and enabled RDMA/hns: Remove RST2RST error prints for hw v1 RDMA/hns: Remove dqpn filling when modify qp from Init to Init RDMA/hns: Fix QP's resp incomplete assignment RDMA/hns: Fix query destination qpn RDMA/hfi1: Convert to SPDX identifier IB/rdmavt: Convert to SPDX identifier RDMA/hns: Bugfix for incorrect association between dip_idx and dgid RDMA/hns: Bugfix for the missing assignment for dip_idx RDMA/hns: Bugfix for data type of dip_idx RDMA/hns: Fix incorrect lsn field RDMA/irdma: Remove the repeated declaration RDMA/core/sa_query: Retry SA queries RDMA: Use the sg_table directly and remove the opencoded version from umem lib/scatterlist: Fix wrong update of orig_nents lib/scatterlist: Provide a dedicated function to support table append RDMA/hns: Delete unused hns bitmap interface ...
|
#
75e38285 |
|
22-Jul-2021 |
Jason Ekstrand <jason@jlekstrand.net> |
drm/i915/gem/ttm: Only call __i915_gem_object_set_pages if needed __i915_ttm_get_pages does two things. First, it calls ttm_bo_validate() to check the given placement and migrate the BO if needed. Then, it updates the GEM object to match, in case the object was migrated. If no migration occured, however, we might still have pages on the GEM object in which case we don't need to fetch them from TTM and call __i915_gem_object_set_pages. This hasn't been a problem before because the primary user of __i915_ttm_get_pages is __i915_gem_object_get_pages which only calls it if the GEM object doesn't have pages. However, i915_ttm_migrate also uses __i915_ttm_get_pages to do the migration so this meant it was unsafe to call on an already populated object. This patch checks i915_gem_object_has_pages() before trying to __i915_gem_object_set_pages so i915_ttm_migrate is safe to call, even on populated objects. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210723172142.3273510-6-jason@jlekstrand.net |
#
7d6a276e |
|
14-Jul-2021 |
Jason Ekstrand <jason@jlekstrand.net> |
drm/i915: Remove allow_alloc from i915_gem_object_get_sg* This reverts the rest of 0edbb9ba1bfe ("drm/i915: Move cmd parser pinning to execbuffer"). Now that the only user of i915_gem_object_get_sg without allow_alloc has been removed, we can drop the parameter. This portion of the revert was broken into its own patch to aid review. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Reviewed-by: Jon Bloomfield <jon.bloomfield@intel.com> Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: https://patchwork.freedesktop.org/patch/msgid/20210714193419.1459723-4-jason@jlekstrand.net |
#
d22632c8 |
|
25-Jun-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915: support forcing the page size with lmem For some specialised objects we might need something larger than the regions min_page_size due to some hw restriction, and slightly more hairy is needing something smaller with the guarantee that such objects will never be inserted into any GTT, which is the case for the paging structures. This also fixes how we setup the BO page_alignment, if we later migrate the object somewhere else. For example if the placements are {SMEM, LMEM}, then we might get this wrong. Pushing the min_page_size behaviour into the manager should fix this. v2(Thomas): push the default page size behaviour into buddy_man, and let the user override it with the page-alignment, which looks cleaner v3: rebase on ttm sys changes Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210625103824.558481-1-matthew.auld@intel.com |
#
b6e913e1 |
|
29-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/gem: Implement object migration Introduce an interface to migrate objects between regions. This is primarily intended to migrate objects to LMEM for display and to SYSTEM for dma-buf, but might be reused in one form or another for performance-based migration. v2: - Verify that the memory region given as an id really exists. (Reported by Matthew Auld) - Call i915_gem_object_{init,release}_memory_region() when switching region to handle also switching region lists. (Reported by Matthew Auld) v3: - Fix i915_gem_object_can_migrate() to return true if object is already in the correct region, even if the object ops doesn't have a migrate() callback. - Update typo in commit message. - Fix kerneldoc of i915_gem_object_wait_migration(). v4: - Improve documentation (Suggested by Mattew Auld and Michael Ruhl) - Always assume TTM migration hits a TTM move and unsets the pages through move_notify. (Reported by Matthew Auld) - Add a dma_fence_might_wait() annotation to i915_gem_object_wait_migration() (Suggested by Daniel Vetter) v5: - Re-add might_sleep() instead of __dma_fence_might_wait(), Sent v4 with the wrong version, didn't compile and __dma_fence_might_wait() is not exported. - Added an R-B. Reported-by: kernel test robot <lkp@intel.com> Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210629151203.209465-2-thomas.hellstrom@linux.intel.com |
#
32b7cf51 |
|
24-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Use TTM for system memory For discrete, use TTM for both cached and WC system memory. That means we currently rely on the TTM memory accounting / shrinker. For cached system memory we should consider remaining shmem-backed, which can be implemented from our ttm_tt_populate callback. We can then also reuse our own very elaborate shrinker for that memory. If an object is evicted to a gem allowable region, we will now consider the object migrated, and we flip the gem region and move the object to a different region list. Since we are now changing gem regions, we can't any longer rely on the CONTIGUOUS flag being set based on the region min page size, so remove that flag update. If we want to reintroduce it, we need to put it in the mutable flags. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210624084240.270219-4-thomas.hellstrom@linux.intel.com |
#
3c2b8f32 |
|
24-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Adjust gem flags and caching settings after a move After a TTM move or object init we need to update the i915 gem flags and caching settings to reflect the new placement. Currently caching settings are not changed during the lifetime of an object, although that might change moving forward if we run into performance issues or issues with WC system page allocations. Also introduce gpu_binds_iomem() and cpu_maps_iomem() to clean up the various ways we previously used to detect this. Finally, initialize the TTM object reserved to be able to update flags and caching before anyone else gets hold of the object. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210624084240.270219-3-thomas.hellstrom@linux.intel.com |
#
0ff37575 |
|
24-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915: Update object placement flags to be mutable The object ops i915_GEM_OBJECT_HAS_IOMEM and the object I915_BO_ALLOC_STRUCT_PAGE flags are considered immutable by much of our code. Introduce a new mem_flags member to hold these and make sure checks for these flags being set are either done under the object lock or with pages properly pinned. The flags will change during migration under the object lock. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210624084240.270219-2-thomas.hellstrom@linux.intel.com |
#
4bc2d574 |
|
23-Jun-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: fix static warning warning: symbol 'i915_gem_ttm_obj_ops' was not declared. Should it be static? Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210623143411.293630-1-matthew.auld@intel.com |
#
b07a6483 |
|
18-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Fix incorrect assumptions about ttm_bo_validate() semantics We have assumed that if the current placement was not the requested placement, but instead one of the busy placements, a TTM move would have been triggered. That is not the case. So when we initially place LMEM objects in "Limbo", (that is system placement without any pages allocated), to be able to defer clearing objects until first get_pages(), the first get_pages() would happily keep objects in system memory if that is one of the allowed placements. And since we don't yet support i915 GEM system memory from TTM, everything breaks apart. So make sure we try the requested placement first, if no eviction is needed. If that fails, retry with all allowed placements also allowing evictions. Also make sure we handle TTM failure codes correctly. Also temporarily (until we support i915 GEM system on TTM), restrict allowed placements to the requested placement to avoid things falling apart should LMEM be full. Fixes: 38f28c0695c0 ("drm/i915/ttm: Calculate the object placement at get_pages time") Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210618132515.163277-1-thomas.hellstrom@linux.intel.com |
#
50331a7b |
|
17-Jun-2021 |
Ramalingam C <ramalingam.c@intel.com> |
drm/i915/ttm: accelerated move implementation Invokes the pipelined page migration through blt, for i915_ttm_move requests of eviction and also obj clear. Signed-off-by: Ramalingam C <ramalingam.c@intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210617063018.92802-11-thomas.hellstrom@linux.intel.com |
#
13c2ceb6 |
|
16-Jun-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: restore min_page_size behaviour We now have bo->page_alignment which perfectly describes what we need if we have min page size restrictions for lmem. We can also drop the flag here, since this is the default behaviour for all objects. v2(Thomas): - bo->page_alignment is in page units Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210616152501.394518-7-matthew.auld@intel.com |
#
d53ec322 |
|
16-Jun-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: switch over to ttm_buddy_man Move back to the buddy allocator for managing device local memory, and restore the lost mock selftests. Keep around the range manager related bits, since we likely need this for managing stolen at some point. For stolen we also don't need to reserve anything so no need to support a generic reserve interface. v2(Thomas): - bo->page_alignment is in page units, not bytes Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210616152501.394518-6-matthew.auld@intel.com |
#
687c7d0f |
|
16-Jun-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: remove node usage in our naming Now that ttm_resource_manager just returns a generic ttm_resource we don't need to reference the mm_node stuff anymore which mostly only makes sense for drm_mm_node. In the next few patches we want switch over to the ttm_buddy_man which is just another type of ttm_resource so reflect that in the naming. Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210616152501.394518-5-matthew.auld@intel.com |
#
beb6a229 |
|
16-Jun-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: pass along the I915_BO_ALLOC_CONTIGUOUS Currently we just ignore the I915_BO_ALLOC_CONTIGUOUS flag, which is fine since everything is already contiguous with the ttm range manager. However in the next patch we want to switch over to the ttm buddy manager, where allocations are by default not contiguous. v2(Thomas): - Forward ALLOC_CONTIG for all regions Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210616152501.394518-4-matthew.auld@intel.com |
#
38f28c06 |
|
16-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Calculate the object placement at get_pages time Instead of relying on a static placement, calculate at get_pages() time. This should work for LMEM regions and system for now. For stolen we need to take preallocated range into account. That will if needed be added later. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210616152501.394518-3-matthew.auld@intel.com |
#
c865204e |
|
15-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Fix memory leaks Fix two memory leaks introduced with the ttm backend. Fixes: 213d50927763 ("drm/i915/ttm: Introduce a TTM i915 gem object backend") Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210615122408.32347-1-thomas.hellstrom@linux.intel.com |
#
cf3e3e86d |
|
10-Jun-2021 |
Maarten Lankhorst <maarten.lankhorst@linux.intel.com> |
drm/i915: Use ttm mmap handling for ttm bo's. Use the ttm handlers for servicing page faults, and vm_access. We do our own validation of read-only access, otherwise use the ttm handlers as much as possible. Because the ttm handlers expect the vma_node at vma->base, we slightly need to massage the mmap handlers to look at vma_node->driver_private to fetch the bo, if it's NULL, we assume i915's normal mmap_offset uapi is used. This is the easiest way to achieve compatibility without changing ttm's semantics. Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210610070152.572423-5-thomas.hellstrom@linux.intel.com |
#
213d5092 |
|
10-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Introduce a TTM i915 gem object backend Most logical place to introduce TTM buffer objects is as an i915 gem object backend. We need to add some ops to account for added functionality like delayed delete and LRU list manipulation. Initially we support only LMEM and SYSTEM memory, but SYSTEM (which in this case means evicted LMEM objects) is not visible to i915 GEM yet. The plan is to move the i915 gem system region over to the TTM system memory type in upcoming patches. We set up GPU bindings directly both from LMEM and from the system region, as there is no need to use the legacy TTM_TT memory type. We reserve that for future porting of GGTT bindings to TTM. Remove the old lmem backend. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210610070152.572423-2-thomas.hellstrom@linux.intel.com |
#
75e38285 |
|
22-Jul-2021 |
Jason Ekstrand <jason@jlekstrand.net> |
drm/i915/gem/ttm: Only call __i915_gem_object_set_pages if needed __i915_ttm_get_pages does two things. First, it calls ttm_bo_validate() to check the given placement and migrate the BO if needed. Then, it updates the GEM object to match, in case the object was migrated. If no migration occured, however, we might still have pages on the GEM object in which case we don't need to fetch them from TTM and call __i915_gem_object_set_pages. This hasn't been a problem before because the primary user of __i915_ttm_get_pages is __i915_gem_object_get_pages which only calls it if the GEM object doesn't have pages. However, i915_ttm_migrate also uses __i915_ttm_get_pages to do the migration so this meant it was unsafe to call on an already populated object. This patch checks i915_gem_object_has_pages() before trying to __i915_gem_object_set_pages so i915_ttm_migrate is safe to call, even on populated objects. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210723172142.3273510-6-jason@jlekstrand.net
|
#
7d6a276e |
|
14-Jul-2021 |
Jason Ekstrand <jason@jlekstrand.net> |
drm/i915: Remove allow_alloc from i915_gem_object_get_sg* This reverts the rest of 0edbb9ba1bfe ("drm/i915: Move cmd parser pinning to execbuffer"). Now that the only user of i915_gem_object_get_sg without allow_alloc has been removed, we can drop the parameter. This portion of the revert was broken into its own patch to aid review. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Reviewed-by: Jon Bloomfield <jon.bloomfield@intel.com> Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch> Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: https://patchwork.freedesktop.org/patch/msgid/20210714193419.1459723-4-jason@jlekstrand.net
|
#
d22632c8 |
|
25-Jun-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915: support forcing the page size with lmem For some specialised objects we might need something larger than the regions min_page_size due to some hw restriction, and slightly more hairy is needing something smaller with the guarantee that such objects will never be inserted into any GTT, which is the case for the paging structures. This also fixes how we setup the BO page_alignment, if we later migrate the object somewhere else. For example if the placements are {SMEM, LMEM}, then we might get this wrong. Pushing the min_page_size behaviour into the manager should fix this. v2(Thomas): push the default page size behaviour into buddy_man, and let the user override it with the page-alignment, which looks cleaner v3: rebase on ttm sys changes Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210625103824.558481-1-matthew.auld@intel.com
|
#
b6e913e1 |
|
29-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/gem: Implement object migration Introduce an interface to migrate objects between regions. This is primarily intended to migrate objects to LMEM for display and to SYSTEM for dma-buf, but might be reused in one form or another for performance-based migration. v2: - Verify that the memory region given as an id really exists. (Reported by Matthew Auld) - Call i915_gem_object_{init,release}_memory_region() when switching region to handle also switching region lists. (Reported by Matthew Auld) v3: - Fix i915_gem_object_can_migrate() to return true if object is already in the correct region, even if the object ops doesn't have a migrate() callback. - Update typo in commit message. - Fix kerneldoc of i915_gem_object_wait_migration(). v4: - Improve documentation (Suggested by Mattew Auld and Michael Ruhl) - Always assume TTM migration hits a TTM move and unsets the pages through move_notify. (Reported by Matthew Auld) - Add a dma_fence_might_wait() annotation to i915_gem_object_wait_migration() (Suggested by Daniel Vetter) v5: - Re-add might_sleep() instead of __dma_fence_might_wait(), Sent v4 with the wrong version, didn't compile and __dma_fence_might_wait() is not exported. - Added an R-B. Reported-by: kernel test robot <lkp@intel.com> Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210629151203.209465-2-thomas.hellstrom@linux.intel.com
|
#
32b7cf51 |
|
24-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Use TTM for system memory For discrete, use TTM for both cached and WC system memory. That means we currently rely on the TTM memory accounting / shrinker. For cached system memory we should consider remaining shmem-backed, which can be implemented from our ttm_tt_populate callback. We can then also reuse our own very elaborate shrinker for that memory. If an object is evicted to a gem allowable region, we will now consider the object migrated, and we flip the gem region and move the object to a different region list. Since we are now changing gem regions, we can't any longer rely on the CONTIGUOUS flag being set based on the region min page size, so remove that flag update. If we want to reintroduce it, we need to put it in the mutable flags. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210624084240.270219-4-thomas.hellstrom@linux.intel.com
|
#
3c2b8f32 |
|
24-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Adjust gem flags and caching settings after a move After a TTM move or object init we need to update the i915 gem flags and caching settings to reflect the new placement. Currently caching settings are not changed during the lifetime of an object, although that might change moving forward if we run into performance issues or issues with WC system page allocations. Also introduce gpu_binds_iomem() and cpu_maps_iomem() to clean up the various ways we previously used to detect this. Finally, initialize the TTM object reserved to be able to update flags and caching before anyone else gets hold of the object. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210624084240.270219-3-thomas.hellstrom@linux.intel.com
|
#
0ff37575 |
|
24-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915: Update object placement flags to be mutable The object ops i915_GEM_OBJECT_HAS_IOMEM and the object I915_BO_ALLOC_STRUCT_PAGE flags are considered immutable by much of our code. Introduce a new mem_flags member to hold these and make sure checks for these flags being set are either done under the object lock or with pages properly pinned. The flags will change during migration under the object lock. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210624084240.270219-2-thomas.hellstrom@linux.intel.com
|
#
4bc2d574 |
|
23-Jun-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: fix static warning warning: symbol 'i915_gem_ttm_obj_ops' was not declared. Should it be static? Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210623143411.293630-1-matthew.auld@intel.com
|
#
b07a6483 |
|
18-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Fix incorrect assumptions about ttm_bo_validate() semantics We have assumed that if the current placement was not the requested placement, but instead one of the busy placements, a TTM move would have been triggered. That is not the case. So when we initially place LMEM objects in "Limbo", (that is system placement without any pages allocated), to be able to defer clearing objects until first get_pages(), the first get_pages() would happily keep objects in system memory if that is one of the allowed placements. And since we don't yet support i915 GEM system memory from TTM, everything breaks apart. So make sure we try the requested placement first, if no eviction is needed. If that fails, retry with all allowed placements also allowing evictions. Also make sure we handle TTM failure codes correctly. Also temporarily (until we support i915 GEM system on TTM), restrict allowed placements to the requested placement to avoid things falling apart should LMEM be full. Fixes: 38f28c0695c0 ("drm/i915/ttm: Calculate the object placement at get_pages time") Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210618132515.163277-1-thomas.hellstrom@linux.intel.com
|
#
50331a7b |
|
17-Jun-2021 |
Ramalingam C <ramalingam.c@intel.com> |
drm/i915/ttm: accelerated move implementation Invokes the pipelined page migration through blt, for i915_ttm_move requests of eviction and also obj clear. Signed-off-by: Ramalingam C <ramalingam.c@intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210617063018.92802-11-thomas.hellstrom@linux.intel.com
|
#
13c2ceb6 |
|
16-Jun-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: restore min_page_size behaviour We now have bo->page_alignment which perfectly describes what we need if we have min page size restrictions for lmem. We can also drop the flag here, since this is the default behaviour for all objects. v2(Thomas): - bo->page_alignment is in page units Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210616152501.394518-7-matthew.auld@intel.com
|
#
d53ec322 |
|
16-Jun-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: switch over to ttm_buddy_man Move back to the buddy allocator for managing device local memory, and restore the lost mock selftests. Keep around the range manager related bits, since we likely need this for managing stolen at some point. For stolen we also don't need to reserve anything so no need to support a generic reserve interface. v2(Thomas): - bo->page_alignment is in page units, not bytes Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210616152501.394518-6-matthew.auld@intel.com
|
#
687c7d0f |
|
16-Jun-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: remove node usage in our naming Now that ttm_resource_manager just returns a generic ttm_resource we don't need to reference the mm_node stuff anymore which mostly only makes sense for drm_mm_node. In the next few patches we want switch over to the ttm_buddy_man which is just another type of ttm_resource so reflect that in the naming. Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210616152501.394518-5-matthew.auld@intel.com
|
#
beb6a229 |
|
16-Jun-2021 |
Matthew Auld <matthew.auld@intel.com> |
drm/i915/ttm: pass along the I915_BO_ALLOC_CONTIGUOUS Currently we just ignore the I915_BO_ALLOC_CONTIGUOUS flag, which is fine since everything is already contiguous with the ttm range manager. However in the next patch we want to switch over to the ttm buddy manager, where allocations are by default not contiguous. v2(Thomas): - Forward ALLOC_CONTIG for all regions Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210616152501.394518-4-matthew.auld@intel.com
|
#
38f28c06 |
|
16-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Calculate the object placement at get_pages time Instead of relying on a static placement, calculate at get_pages() time. This should work for LMEM regions and system for now. For stolen we need to take preallocated range into account. That will if needed be added later. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210616152501.394518-3-matthew.auld@intel.com
|
#
c865204e |
|
15-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Fix memory leaks Fix two memory leaks introduced with the ttm backend. Fixes: 213d50927763 ("drm/i915/ttm: Introduce a TTM i915 gem object backend") Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210615122408.32347-1-thomas.hellstrom@linux.intel.com
|
#
cf3e3e86d |
|
10-Jun-2021 |
Maarten Lankhorst <maarten.lankhorst@linux.intel.com> |
drm/i915: Use ttm mmap handling for ttm bo's. Use the ttm handlers for servicing page faults, and vm_access. We do our own validation of read-only access, otherwise use the ttm handlers as much as possible. Because the ttm handlers expect the vma_node at vma->base, we slightly need to massage the mmap handlers to look at vma_node->driver_private to fetch the bo, if it's NULL, we assume i915's normal mmap_offset uapi is used. This is the easiest way to achieve compatibility without changing ttm's semantics. Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210610070152.572423-5-thomas.hellstrom@linux.intel.com
|
#
213d5092 |
|
10-Jun-2021 |
Thomas Hellström <thomas.hellstrom@linux.intel.com> |
drm/i915/ttm: Introduce a TTM i915 gem object backend Most logical place to introduce TTM buffer objects is as an i915 gem object backend. We need to add some ops to account for added functionality like delayed delete and LRU list manipulation. Initially we support only LMEM and SYSTEM memory, but SYSTEM (which in this case means evicted LMEM objects) is not visible to i915 GEM yet. The plan is to move the i915 gem system region over to the TTM system memory type in upcoming patches. We set up GPU bindings directly both from LMEM and from the system region, as there is no need to use the legacy TTM_TT memory type. We reserve that for future porting of GGTT bindings to TTM. Remove the old lmem backend. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20210610070152.572423-2-thomas.hellstrom@linux.intel.com
|