drm/msm/gem: Demote userspace errors to DRM_UT_DRIVER

Error messages resulting from incorrect usage of the kernel uabi should
not spam dmesg by default. But it is useful to enable them to debug
userspace. So demote to DRM_UT_DRIVER.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/564189/

drm/msm/gem: Add metadata

The EXT_external_objects extension is a bit awkward as it doesn't pass
explicit modifiers, leaving the importer to guess with incomplete
information. In the case of vk (turnip) exporting and gl (freedreno)
importing, the "OPTIMAL_TILING_EXT" layout depends on VkImageCreateInfo
flags (among other things), which the importer does not know. Which
unfortunately leaves us with the need for a metadata back-channel.

The contents of the metadata are defined by userspace. The
EXT_external_objects extension is only required to work between
compatible versions of gl and vk drivers, as defined by device and
driver UUIDs.

v2: add missing metadata kfree
v3: Rework to move copy_from/to_user out from under gem obj lock
to avoid angering lockdep about deadlocks against fs-reclaim

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/566157/

drm/msm: Remove vma use tracking

This was not strictly necessary, as page unpinning (ie. shrinker) only
cares about the resv. It did give us some extra sanity checking for
userspace controlled iova, and was useful to catch issues on kernel and
userspace side when enabling userspace iova. But if userspace screws
this up, it just corrupts it's own gpu buffers and/or gets iova faults.
So we can just let userspace shoot it's own foot and drop the extra per-
buffer SUBMIT overhead.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Patchwork: https://patchwork.freedesktop.org/patch/551023/

drm/msm: Take lru lock once per submit_pin_objects()

Split out pin_count incrementing and lru updating into a separate loop
so we can take the lru lock only once for all objs. Since we are still
holding the obj lock, it is safe to split this up.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/551025/

drm/msm: Take lru lock once per job_run

Rather than acquiring it and dropping it for each individual obj.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/551019/

drm/msm: Initialize mmap offset after constructing the buffer object

Only the msm driver provides its own implementation of gem_prime_mmap
from struct drm_driver. All other drivers use the drm_gem_prime_mmap()
helper.

Initialize the mmap offset when constructing the buffer object in msm
and reduce the gem_prime_mmap code to the generic helper. Prepares
msm for the removal of struct drm_driver.gem_prime_mmap.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230613150441.17720-2-tzimmermann@suse.de

drm/msm: Add memory stats to fdinfo

Use the new helper to export stats about memory usage.

v2: Drop unintended hunk
v3: Rebase

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Neil Armstrong <neil.armstrong@linaro.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20230524155956.382440-7-robdclark@gmail.com

drm/msm: Fix vmap madv warning

Commit d6ae7d1cd58e ("drm/msm/gem: Simplify vmap vs LRU tracking")
introduced a splat in the pin_pages_locked() path for buffers that
had been MADV_DONTNEED.

------------[ cut here ]------------
msm_obj->madv != 0
WARNING: CPU: 1 PID: 144 at drivers/gpu/drm/msm/msm_gem.c:230 msm_gem_pin_pages_locked+0x9c/0xd4
Modules linked in: lzo_rle cros_ec_lid_angle cros_ec_sensors cros_ec_sensors_core venus_dec venus_enc videobuf2_dma_contig cdc_ether usbnet mii uvcvideo videobuf2_vmalloc hci_uart btqca qcom_spmi_adc5 uvc qcom_spmi_temp_alarm qcom_vadc_common cros_ec_sensorhub videobuf2_memops cros_ec_typec sx9324 sx_common typec joydev bluetooth industrialio_triggered_buffer ecdh_generic kfifo_buf ecc venus_core qcom_stats v4l2_mem2mem videobuf2_v4l2 videobuf2_common ath11k_ahb ath11k mac80211 cfg80211 fuse zram zsmalloc
CPU: 1 PID: 144 Comm: ring0 Tainted: G W 6.3.0-rc2-debug+ #622
Hardware name: Google Villager (rev1+) with LTE (DT)
pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : msm_gem_pin_pages_locked+0x9c/0xd4
lr : msm_gem_pin_pages_locked+0x9c/0xd4
sp : ffffffc009ffbab0
x29: ffffffc009ffbab0 x28: ffffffee8da75008 x27: ffffff80a10274d0
x26: ffffff8087fe3bf8 x25: ffffff8087fe3c08 x24: 0000000000000001
x23: ffffff80891d5800 x22: ffffff809d0de480 x21: ffffff8081e5a080
x20: 0000000000000002 x19: ffffff80a3564c00 x18: 0000000000000000
x17: 0000000000000000 x16: 0000000000000000 x15: 00000000000a9620
x14: 0000000000000000 x13: 2d2d2d2d2d2d2d2d x12: 2d2d2d2d5d206572
x11: 656820747563205b x10: 2d2d2d2d2d2d2d2d x9 : ffffffee8c705dfc
x8 : ffffffee8da75000 x7 : ffffffee8d34e6d0 x6 : 0000000000000000
x5 : 00000000000affa8 x4 : 000000000000000d x3 : ffffffee8da75008
x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffffff8088048040
Call trace:
msm_gem_pin_pages_locked+0x9c/0xd4
get_vaddr+0xb0/0x150
msm_gem_get_vaddr_active+0x1c/0x28
snapshot_buf+0x90/0x10c
msm_rd_dump_submit+0x30c/0x380
msm_gpu_submit+0x88/0x174
msm_job_run+0x68/0x118
drm_sched_main+0x2b8/0x3a0
kthread+0xf0/0x100
ret_from_fork+0x10/0x20
irq event stamp: 3358
hardirqs last enabled at (3357): [<ffffffee8c7051f4>] __up_console_sem+0x7c/0x80
hardirqs last disabled at (3358): [<ffffffee8d3480b0>] el1_dbg+0x24/0x80
softirqs last enabled at (3330): [<ffffffee8c610420>] __do_softirq+0x21c/0x4bc
softirqs last disabled at (3325): [<ffffffee8c616708>] ____do_softirq+0x18/0x24
---[ end trace 0000000000000000 ]---

But, as with msm_gem_get_vaddr_active(), this is a special case
because we know that the buffer won't be purged evicted until it's
fence is signaled. We just forgot to propagate the logic get_vaddr()
to pin_pages_locked().

Fixes: d6ae7d1cd58e ("drm/msm/gem: Simplify vmap vs LRU tracking")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/532616/
Link: https://lore.kernel.org/r/20230417225504.494934-1-robdclark@gmail.com

drm/msm: Add wait-boost support

Add a way for various userspace waits to signal urgency.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/525817/
Link: https://lore.kernel.org/r/20230308155322.344664-14-robdclark@gmail.com

drm/msm/gem: Avoid obj lock in job_run()

Now that everything that controls which LRU an obj lives in *except* the
backing pages is protected by the LRU lock, add a special path to unpin
in the job_run() path, where we are assured that we already have backing
pages and will not be racing against eviction (because the GEM object's
dma_resv contains the fence that will be signaled when the submit/job
completes).

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527845/
Link: https://lore.kernel.org/r/20230320144356.803762-10-robdclark@gmail.com

drm/msm/gem: Protect pin_count/madv by LRU lock

Since the LRU lock is already acquired when moving an obj between LRUs,
we can use it to protect pin_count and madv, without any significant
change in locking (ie. it just expands the scope of the lock by a hand-
ful of instructions). This prepares the way to decrement the pin_count
in the job_run() path without needing to hold the obj lock, to avoid a
potential deadlock (or rather stall) caused by the fence-signaling path
(job_run()) blocking on shrinker/reclaim. (Only a stall because the
wait for fence signaling wait_for_idle() is not infinite.)

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527843/
Link: https://lore.kernel.org/r/20230320144356.803762-9-robdclark@gmail.com

drm/msm/gem: Move update_lru()

Just code-motion.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527841/
Link: https://lore.kernel.org/r/20230320144356.803762-8-robdclark@gmail.com

drm/msm/gem: Simplify vmap vs LRU tracking

vmap'ing is just pinning in disguise. So treat it as such and simplify
the LRU tracking.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527837/
Link: https://lore.kernel.org/r/20230320144356.803762-6-robdclark@gmail.com

drm/msm/gem: Tidy up VMA API

Stop open coding VMA construction, which will be needed in the next
commit. And since the VMA already has a ptr to the adress space, stop
passing that around everywhere. (Also, an aspace always has an mmu so
we can drop a couple pointless NULL checks.)

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/527833/
Link: https://lore.kernel.org/r/20230320144356.803762-4-robdclark@gmail.com

mm: replace vma->vm_flags direct modifications with modifier calls

Replace direct modifications to vma->vm_flags with calls to modifier
functions to be able to track flag changes and to keep vma locking
correctness.

[akpm@linux-foundation.org: fix drivers/misc/open-dice.c, per Hyeonggon Yoo]
Link: https://lkml.kernel.org/r/20230126193752.297968-5-surenb@google.com
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Mike Rapoport (IBM) <rppt@kernel.org>
Acked-by: Sebastian Reichel <sebastian.reichel@collabora.com>
Reviewed-by: Liam R. Howlett <Liam.Howlett@Oracle.com>
Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arjun Roy <arjunroy@google.com>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: David Rientjes <rientjes@google.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Greg Thelen <gthelen@google.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jann Horn <jannh@google.com>
Cc: Joel Fernandes <joelaf@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Laurent Dufour <ldufour@linux.ibm.com>
Cc: Lorenzo Stoakes <lstoakes@gmail.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Minchan Kim <minchan@google.com>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Peter Oskolkov <posk@google.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Punit Agrawal <punit.agrawal@bytedance.com>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Shakeel Butt <shakeelb@google.com>
Cc: Soheil Hassas Yeganeh <soheil@google.com>
Cc: Song Liu <songliubraving@fb.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

drm/msm/gem: Add msm_gem_assert_locked()

All use of msm_gem_is_locked() is just for WARN_ON()s, so extract out
into an msm_gem_assert_locked() patch.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/496136/
Link: https://lore.kernel.org/r/20220802155152.1727594-15-robdclark@gmail.com

drm/msm/gem: Convert to using drm_gem_lru

This converts over to use the shared GEM LRU/shrinker helpers. Note
that it means we are no longer tracking purgeable or willneed buffers
that are active separately. But the most recently pinned buffers should
be at the tail of the various LRUs, and the shrinker is already prepared
to encounter objects which are still active.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/496131/
Link: https://lore.kernel.org/r/20220802155152.1727594-11-robdclark@gmail.com

drm/msm/gem: Remove active refcnt

At this point the pinned refcnt is sufficient, and the shrinker is
already prepared to encounter objects which are still active according
to fences attached to the resv.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/496122/
Link: https://lore.kernel.org/r/20220802155152.1727594-9-robdclark@gmail.com

drm/msm/gem: Consolidate pin/unpin paths

Avoid having multiple spots where we increment/decrement pin_count (and
associated LRU updating)

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/496130/
Link: https://lore.kernel.org/r/20220802155152.1727594-8-robdclark@gmail.com

drm/msm/gem: Rename to pin/unpin_pages

Since that is what these fxns actually do.. they are getting *pinned*
pages (as opposed to cases where we need pages, but don't need them
pinned, like CPU mappings).

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/496121/
Link: https://lore.kernel.org/r/20220802155152.1727594-7-robdclark@gmail.com

drm/msm/gem: Rename update_inactive

Really what this is doing is updating various LRU lists.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/496115/
Link: https://lore.kernel.org/r/20220802155152.1727594-6-robdclark@gmail.com

drm/msm/gem: Check for active in shrinker path

Currently in our shrinker path we shouldn't be encountering anything
that is active, but this will change in subsequent patches. So check
if there are unsignaled fences.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/496117/
Link: https://lore.kernel.org/r/20220802155152.1727594-5-robdclark@gmail.com

drm/msm: Switch to pfn mappings

I'm not entirely sure why we were using VM_MIXEDMAP. These are never
CoW mappings. Let's switch to be more consistent with what other
drivers and the GEM shmem helpers do.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/491218/
Link: https://lore.kernel.org/r/20220625225454.81039-2-robdclark@gmail.com

drm/msm: Deprecate MSM_BO_UNCACHED harder

Handle the demotion to MSM_BO_WC at the userspace ABI level, and fix
the remaining internal MSM_BO_UNCACHED user.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/489339/
Link: https://lore.kernel.org/r/20220613194623.2588353-1-robdclark@gmail.com

drm/msm/gem: Drop obj lock in msm_gem_free_object()

The only reason we grabbed the lock was to satisfy a bunch of places
that WARN_ON() if called without the lock held. But this angers lockdep
which doesn't realize no one else can be holding the lock by the time we
end up destroying the object (and sees what would otherwise be a locking
inversion between reservation_ww_class_mutex and fs_reclaim).

Closes: https://gitlab.freedesktop.org/drm/msm/-/issues/14
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/489364/
Link: https://lore.kernel.org/r/20220613205032.2652374-1-robdclark@gmail.com

drm/msm: Make msm_gem_free_object() static

Misc small cleanup I noticed. Not called from another object file since
commit 3c9edd9c85f5 ("drm/msm: Introduce GEM object funcs")

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Abhinav Kumar <quic_abhinavk@quicinc.com>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Patchwork: https://patchwork.freedesktop.org/patch/489362/
Link: https://lore.kernel.org/r/20220613204910.2651747-1-robdclark@gmail.com
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>

drm/msm/gem: Separate object and vma unpin

Previously the BO_PINNED state in the submit was tracking two related
but different things: (1) that the buffer object was pinned, and (2)
that the vma (mapping within a set of pagetables) was pinned. But with
fenced vma unpin (needed so that userspace couldn't race with retire
path for releasing a vma) these two were decoupled. The fact that the
BO_PINNED flag was already cleared meant that we leaked the bo pin count
which should have been dropped when the submit was retired.

So split this state into BO_OBJ_PINNED and BO_VMA_PINNED, so they can be
dropped independently.

Fixes: 95d1deb02a9c ("drm/msm/gem: Add fenced vma unpin")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/487559/
Link: https://lore.kernel.org/r/20220527172341.2151005-1-robdclark@gmail.com

drm/msm: Add a way for userspace to allocate GPU iova

The motivation at this point is mainly native userspace mesa driver in a
VM guest. The one remaining synchronous "hotpath" is buffer allocation,
because guest needs to wait to know the bo's iova before it can start
emitting cmdstream/state that references the new bo. By allocating the
iova in the guest userspace, we no longer need to wait for a response
from the host, but can just rely on the allocation request being
processed before the cmdstream submission. Allocation failures (OoM,
etc) would just be treated as context-lost (ie. GL_GUILTY_CONTEXT_RESET)
or subsequent allocations (or readpix, etc) can raise GL_OUT_OF_MEMORY.

v2: Fix inuse check
v3: Change mismatched iova case to -EBUSY

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Reviewed-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Link: https://lore.kernel.org/r/20220411215849.297838-11-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm/gem: Add fenced vma unpin

With userspace allocated iova (next patch), we can have a race condition
where userspace observes the fence completion and deletes the vma before
retire_submit() gets around to unpinning the vma. To handle this, add a
fenced unpin which drops the refcount but tracks the fence, and update
msm_gem_vma_inuse() to check any previously unsignaled fences.

v2: Fix inuse underflow (duplicate unpin)
v3: Fix msm_job_run() vs submit_cleanup() race condition

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20220411215849.297838-10-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm/gem: Split vma lookup and pin

This way we only lookup vma once per object per submit, for both the
submit and retire path.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20220411215849.297838-9-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm/gem: Rework vma lookup and pin

Combines duplicate vma lookup in the get_and_pin path.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Link: https://lore.kernel.org/r/20220411215849.297838-8-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Drop msm_gem_iova()

There was only a single user, which could just as easily stash the iova
when pinning.

v2: fix prepare->prepare->cleanup->cleanup sequences

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Link: https://lore.kernel.org/r/20220411215849.297838-7-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm/gem: Drop PAGE_SHIFT for address space mm

Get rid of all the unnecessary conversion between address/size and page
offsets. It just confuses things.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Link: https://lore.kernel.org/r/20220411215849.297838-6-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm/gem: Split out inuse helper

Prep for a following patch, where it gets a bit more complicated.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Link: https://lore.kernel.org/r/20220411215849.297838-5-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

dma-buf: add enum dma_resv_usage v4

This change adds the dma_resv_usage enum and allows us to specify why a
dma_resv object is queried for its containing fences.

Additional to that a dma_resv_usage_rw() helper function is added to aid
retrieving the fences for a read or write userspace submission.

This is then deployed to the different query functions of the dma_resv
object and all of their users. When the write paratermer was previously
true we now use DMA_RESV_USAGE_WRITE and DMA_RESV_USAGE_READ otherwise.

v2: add KERNEL/OTHER in separate patch
v3: some kerneldoc suggestions by Daniel
v4: some more kerneldoc suggestions by Daniel, fix missing cases lost in
the rebase pointed out by Bas.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20220407085946.744568-2-christian.koenig@amd.com

drm/msm: Add missing put_task_struct() in debugfs path

Fixes: 25faf2f2e065 ("drm/msm: Show process names in gem_describe")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20220317184550.227991-1-robdclark@gmail.com

drm/msm: use the new dma_resv_describe

Instead of hand rolling pretty much the same code.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211103081231.18578-3-christian.koenig@amd.com

drm/msm: use new iterator in msm_gem_describe

Simplifying the code a bit. Also drop the RCU read side lock since the
object is locked anyway.

Untested since I can't get the driver to compile on !ARM.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20211005113742.1101-15-christian.koenig@amd.com

drm/msm: Demote debug message

Mesa attempts to allocate a cached-coherent buffer in order to determine
if cached-coherent is supported. Resulting in seeing this error message
once per process with newer mesa. But no reason for this to be more
than a debug msg.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20211111230214.765476-1-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Fix mmap to include VM_IO and VM_DONTDUMP

In commit 510410bfc034 ("drm/msm: Implement mmap as GEM object
function") we switched to a new/cleaner method of doing things. That's
good, but we missed a little bit.

Before that commit, we used to _first_ run through the
drm_gem_mmap_obj() case where `obj->funcs->mmap()` was NULL. That meant
that we ran:

vma->vm_flags |= VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP;
vma->vm_page_prot = pgprot_writecombine(vm_get_page_prot(vma->vm_flags));
vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);

...and _then_ we modified those mappings with our own. Now that
`obj->funcs->mmap()` is no longer NULL we don't run the default
code. It looks like the fact that the vm_flags got VM_IO / VM_DONTDUMP
was important because we're now getting crashes on Chromebooks that
use ARC++ while logging out. Specifically a crash that looks like this
(this is on a 5.10 kernel w/ relevant backports but also seen on a
5.15 kernel):

Unable to handle kernel paging request at virtual address ffffffc008000000
Mem abort info:
ESR = 0x96000006
EC = 0x25: DABT (current EL), IL = 32 bits
SET = 0, FnV = 0
EA = 0, S1PTW = 0
Data abort info:
ISV = 0, ISS = 0x00000006
CM = 0, WnR = 0
swapper pgtable: 4k pages, 39-bit VAs, pgdp=000000008293d000
[ffffffc008000000] pgd=00000001002b3003, p4d=00000001002b3003,
pud=00000001002b3003, pmd=0000000000000000
Internal error: Oops: 96000006 [#1] PREEMPT SMP
[...]
CPU: 7 PID: 15734 Comm: crash_dump64 Tainted: G W 5.10.67 #1 [...]
Hardware name: Qualcomm Technologies, Inc. sc7280 IDP SKU2 platform (DT)
pstate: 80400009 (Nzcv daif +PAN -UAO -TCO BTYPE=--)
pc : __arch_copy_to_user+0xc0/0x30c
lr : copyout+0xac/0x14c
[...]
Call trace:
__arch_copy_to_user+0xc0/0x30c
copy_page_to_iter+0x1a0/0x294
process_vm_rw_core+0x240/0x408
process_vm_rw+0x110/0x16c
__arm64_sys_process_vm_readv+0x30/0x3c
el0_svc_common+0xf8/0x250
do_el0_svc+0x30/0x80
el0_svc+0x10/0x1c
el0_sync_handler+0x78/0x108
el0_sync+0x184/0x1c0
Code: f8408423 f80008c3 910020c6 36100082 (b8404423)

Let's add the two flags back in.

While we're at it, the fact that we aren't running the default means
that we _don't_ need to clear out VM_PFNMAP, so remove that and save
an instruction.

NOTE: it was confirmed that VM_IO was the important flag to fix the
problem I was seeing, but adding back VM_DONTDUMP seems like a sane
thing to do so I'm doing that too.

Fixes: 510410bfc034 ("drm/msm: Implement mmap as GEM object function")
Reported-by: Stephen Boyd <swboyd@chromium.org>
Signed-off-by: Douglas Anderson <dianders@chromium.org>
Reviewed-by: Stephen Boyd <swboyd@chromium.org>
Tested-by: Stephen Boyd <swboyd@chromium.org>
Link: https://lore.kernel.org/r/20211110113334.1.I1687e716adb2df746da58b508db3f25423c40b27@changeid
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Fix missing include files in msm_gem.c

Include linux/vmalloc.h to fix below errors:
error: implicit declaration of function 'vmap'

Signed-off-by: Yanteng Si <siyanteng@loongson.cn>
Link: https://lore.kernel.org/r/15f30165e94574e4cd7c4da9f9c6fd1e320d4d8e.1634200323.git.siyanteng@loongson.cn
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: uninitialized variable in msm_gem_import()

The msm_gem_new_impl() function cleans up after itself so there is no
need to call drm_gem_object_put(). Conceptually, it does not make sense
to call a kref_put() function until after the reference counting has
been initialized which happens immediately after this call in the
drm_gem_(private_)object_init() functions.

In the msm_gem_import() function the "obj" pointer is uninitialized, so
it will lead to a crash.

Fixes: 05b849111c07 ("drm/msm: prime support")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Link: https://lore.kernel.org/r/20211013081315.GG6010@kili
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: fix potential NULL dereference in cleanup

The "msm_obj->node" list needs to be initialized earlier so that the
list_del() in msm_gem_free_object() doesn't experience a NULL pointer
dereference.

Fixes: 6ed0897cd800 ("drm/msm: Fix debugfs deadlock")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Link: https://lore.kernel.org/r/20211013081133.GF6010@kili
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: allow compile_test on !ARM

MSM is one of the few drivers which won't even compile
test on !ARM platforms.

Looking into this a bit more it turned out that there is
actually not that much missing to at least let the driver
compile on x86 as well.

So this patch replaces the use of phys_to_page() with the
open coded version and provides a dummy for of_drm_find_bridge().

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Rob Clark <robdclark@gmail.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210924071759.22659-2-christian.koenig@amd.com

drm/msm: Use list_move_tail instead of list_del/list_add_tail in msm_gem.c

Using list_move_tail() instead of list_del() + list_add_tail() in msm_gem.c.

Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Baokun Li <libaokun1@huawei.com>
Link: https://lore.kernel.org/r/20210609072838.1369371-1-libaokun1@huawei.com
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Implement mmap as GEM object function

Moving the driver-specific mmap code into a GEM object function allows
for using DRM helpers for various mmap callbacks.

The respective msm functions are being removed. The file_operations
structure fops is now being created by the helper macro
DEFINE_DRM_GEM_FOPS().

v2:
* rebase onto latest upstream
* remove declaration of msm_gem_mmap_obj() from msm_fbdev.c

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Link: https://lore.kernel.org/r/20210706084753.8194-1-tzimmermann@suse.de
[squash in missing VM_DONTEXPAND flag]
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm/gem: Mark active before pinning

Mark all the bos in the submit as active, before pinning, to prevent
evicting a buffer in the same submit to make room for a buffer earlier
in the table.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20210728010632.2633470-14-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Drop submit bo_list

This was only used to detect userspace including the same bo multiple
times in a submit. But ww_mutex can already tell us this.

When we drop struct_mutex around the submit ioctl, we'd otherwise need
to lock the bo before adding it to the bo_list. But since ww_mutex can
already tell us this, it is simpler just to remove the bo_list.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20210728010632.2633470-11-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Conversion to drm scheduler

For existing adrenos, there is one or more ringbuffer, depending on
whether preemption is supported. When preemption is supported, each
ringbuffer has it's own priority. A submitqueue (which maps to a
gl context or vk queue in userspace) is mapped to a specific ring-
buffer at creation time, based on the submitqueue's priority.

Each ringbuffer has it's own drm_gpu_scheduler. Each submitqueue
maps to a drm_sched_entity. And each submit maps to a drm_sched_job.

Closes: https://gitlab.freedesktop.org/drm/msm/-/issues/4
Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Christian König <christian.koenig@amd.com>
Link: https://lore.kernel.org/r/20210728010632.2633470-10-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: drop drm_gem_object_put_locked()

No idea why we were still using this. It certainly hasn't been needed
for some time. So drop the pointless twin codepaths.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Christian König <christian.koenig@amd.com>
Link: https://lore.kernel.org/r/20210728010632.2633470-4-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: always wait for the exclusive fence

Drivers also need to to sync to the exclusive fence when
a shared one is present.

Completely untested since the driver won't even compile on !ARM.

Signed-off-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210702111642.17259-5-christian.koenig@amd.com
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>

drm/msm: Fix fall-through warning in msm_gem_new_impl()

Fix the following fall-through warning:

drivers/gpu/drm/msm/msm_gem.c: In function 'msm_gem_new_impl':
drivers/gpu/drm/msm/msm_gem.c:1170:6: warning: this statement may fall through [-Wimplicit-fallthrough=]
1170 | if (priv->has_cached_coherent)
| ^
drivers/gpu/drm/msm/msm_gem.c:1173:2: note: here
1173 | default:
| ^~~~~~~

by replacing the /* fallthrough */ comment with fallthrough;

Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>

drm/msm/msm_gem: Demote kernel-doc abuses

Fixes the following W=1 kernel build warning(s):

drivers/gpu/drm/msm/msm_gem.c:364: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst
drivers/gpu/drm/msm/msm_gem.c:763: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst

Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: linux-arm-msm@vger.kernel.org
Cc: dri-devel@lists.freedesktop.org
Cc: freedreno@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Signed-off-by: Lee Jones <lee.jones@linaro.org>
Link: https://lore.kernel.org/r/20210602143300.2330146-13-lee.jones@linaro.org
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: deprecate MSM_BO_UNCACHED (map as writecombine instead)

There shouldn't be any reason to ever use uncached over writecombine,
so just use writecombine for MSM_BO_UNCACHED.

Note: userspace never used MSM_BO_UNCACHED anyway

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Acked-by: Jordan Crouse <jordan@cosmicpenguin.net>
Link: https://lore.kernel.org/r/20210423190833.25319-6-jonathan@marek.ca
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: add MSM_BO_CACHED_COHERENT

Add a new cache mode for creating coherent host-cached BOs.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Reviewed-by: Jordan Crouse <jcrouse@codeaurora.org>
Link: https://lore.kernel.org/r/20210423190833.25319-5-jonathan@marek.ca
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: use the right pgprot when mapping BOs in the kernel

Use the same logic as the userspace mapping.

This fixes msm_rd with cached BOs.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Acked-by: Jordan Crouse <jordan@cosmicpenguin.net>
Link: https://lore.kernel.org/r/20210423190833.25319-4-jonathan@marek.ca
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: remove unnecessary mmap logic for cached BOs

No one knows what this is for anymore, so just remove it.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Acked-by: Jordan Crouse <jordan@cosmicpenguin.net>
Link: https://lore.kernel.org/r/20210423190833.25319-2-jonathan@marek.ca
Signed-off-by: Rob Clark <robdclark@chromium.org>

dma-buf: drop the _rcu postfix on function names v3

The functions can be called both in _rcu context as well
as while holding the lock.

v2: add some kerneldoc as suggested by Daniel
v3: fix indentation

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210602111714.212426-7-christian.koenig@amd.com

dma-buf: rename and cleanup dma_resv_get_list v2

When the comment needs to state explicitly that this is doesn't get a reference
to the object then the function is named rather badly.

Rename the function and use it in even more places.

v2: use dma_resv_shared_list as new name

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210602111714.212426-5-christian.koenig@amd.com

dma-buf: rename and cleanup dma_resv_get_excl v3

When the comment needs to state explicitly that this
doesn't get a reference to the object then the function
is named rather badly.

Rename the function and use rcu_dereference_check(), this
way it can be used from both rcu as well as lock protected
critical sections.

v2: improve kerneldoc as suggested by Daniel
v3: use dma_resv_excl_fence as function name

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Link: https://patchwork.freedesktop.org/patch/msgid/20210602111714.212426-4-christian.koenig@amd.com

drm/msm: Init mm_list before accessing it for use_vram path

Fix NULL pointer dereference caused by update_inactive()
trying to list_del() an uninitialized mm_list who's
prev/next pointers are NULL.

Fixes: 64fcbde772c7 ("drm/msm: Track potentially evictable objects")
Signed-off-by: Alexey Minnekhanov <alexeymin@postmarketos.org>
Link: https://lore.kernel.org/r/20210518102624.1193955-1-alexeymin@postmarketos.org
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Do not unpin/evict exported dma-buf's

Our initial logic for excluding dma-bufs was not quite right. In
particular we want msm_gem_get/put_pages() path used for exported
dma-bufs to increment/decrement the pin-count.

Also, in case the importer is vmap'ing the dma-buf, we need to be
sure to update the object's status, because it is now no longer
potentially evictable.

Fixes: 63f17ef83428 drm/msm: Support evicting GEM objects to swap
Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20210426235326.1230125-1-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Support evicting GEM objects to swap

Now that tracking is wired up for potentially evictable GEM objects,
wire up shrinker and the remaining GEM bits for unpinning backing pages
of inactive objects.

Disabled by default for now, with an 'enable_eviction' module param to
enable so that we can get some more testing on the range of generations
(and iommu pairings) supported.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20210405174532.1441497-9-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Small msm_gem_purge() fix

Shoot down any mmap's *first* before put_pages(). Also add a WARN_ON
that the object is locked (to make it clear that this doesn't race with
msm_gem_fault()) and remove a redundant WARN_ON (since is_purgable()
already covers that case).

Fixes: 68209390f116 ("drm/msm: shrinker support")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20210405174532.1441497-8-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Track potentially evictable objects

Objects that are potential for swapping out are (1) willneed (ie. if
they are purgable/MADV_WONTNEED we can just free the pages without them
having to land in swap), (2) not on an active list, (3) not dma-buf
imported or exported, and (4) not vmap'd. This repurposes the purged
list for objects that do not have backing pages (either because they
have not been pinned for the first time yet, or in a later patch because
they have been unpinned/evicted.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20210405174532.1441497-7-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Add $debugfs/gem stats on resident objects

Currently nearly everything, other than newly allocated objects which
are not yet backed by pages, is pinned and resident in RAM. But it will
be nice to have some stats on what is unpinned once that is supported.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20210405174532.1441497-6-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Split iova purge and close

Currently these always go together, either when we purge MADV_WONTNEED
objects or when the object is freed. But for unpin, we want to be able
to purge (unmap from iommu) the vma, while keeping the iova range
allocated (so we can remap back to the same GPU virtual address when the
object is re-pinned.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20210405174532.1441497-5-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Clear msm_obj->sgt in put_pages()

Currently this doesn't matter since we keep the pages pinned until the
object is destroyed. But when we start unpinning pages to allow objects
to be evicted to swap, it will.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20210405174532.1441497-4-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: ratelimit GEM related WARN_ON()s

If you mess something up, you don't really need to see the same warn on
splat 4000 times pumped out a slow debug UART port..

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20210405174532.1441497-2-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Fix spelling "purgable" -> "purgeable"

The previous patch fixes the user visible spelling. This one fixes the
code. Oops.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20210406151816.1515329-1-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Fix spelling mistake "Purgable" -> "Purgeable"

There is a spelling mistake in debugfs gem stats. Fix it. Also
re-align output to cater for the extra 1 character.

Signed-off-by: Colin Ian King <colin.king@canonical.com>
Link: https://lore.kernel.org/r/20210406133939.425987-1-colin.king@canonical.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Drop mm_lock in scan loop

lock_stat + mmm_donut[1] say that this reduces contention on mm_lock
significantly (~350x lower waittime-max, and ~100x lower waittime-avg)

[1] https://chromium.googlesource.com/chromiumos/platform/microbenchmarks/+/refs/heads/main/mmm_donut.py

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Link: https://lore.kernel.org/r/20210402211226.875726-1-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Improved debugfs gem stats

The last patch lost the breakdown of active vs inactive GEM objects in
$debugfs/gem. But we can add some better stats to summarize not just
active vs inactive, but also purgable/purged to make up for that.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Tested-by: Douglas Anderson <dianders@chromium.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Link: https://lore.kernel.org/r/20210401012722.527712-5-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Fix debugfs deadlock

In normal cases the gem obj lock is acquired first before mm_lock. The
exception is iterating the various object lists. In the shrinker path,
deadlock is avoided by using msm_gem_trylock() and skipping over objects
that cannot be locked. But for debugfs the straightforward thing is to
split things out into a separate list of all objects protected by it's
own lock.

Fixes: d984457b31c4 ("drm/msm: Add priv->mm_lock to protect active/inactive lists")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Tested-by: Douglas Anderson <dianders@chromium.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Link: https://lore.kernel.org/r/20210401012722.527712-4-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Avoid mutex in shrinker_count()

When the system is under heavy memory pressure, we can end up with lots
of concurrent calls into the shrinker. Keeping a running tab on what we
can shrink avoids grabbing a lock in shrinker->count(), and avoids
shrinker->scan() getting called when not profitable.

Also, we can keep purged objects in their own list to avoid re-traversing
them to help cut down time in the critical section further.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Tested-by: Douglas Anderson <dianders@chromium.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Link: https://lore.kernel.org/r/20210401012722.527712-3-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: remove redundant NULL check

Fix below warnings reported by coccicheck:
./drivers/gpu/drm/msm/msm_gem.c:991:3-9: WARNING: NULL check before some
freeing functions is not needed.

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Signed-off-by: Jiapeng Zhong <abaci-bugfix@linux.alibaba.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/prime: split array import functions v4

Mapping the imported pages of a DMA-buf into an userspace process
doesn't work as expected.

But we have reoccurring requests on this approach, so split the
functions for this and document that dma_buf_mmap() needs to be used
instead.

v2: split it into two functions
v3: rebased on latest changes
v4: update commit message a bit

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/403838/

drm/msm: Ensure get_pages is called when locked

get_pages is only called in a locked context. Add a WARN_ON to make sure
it stays that way.

Signed-off-by: Iskren Chernev <iskren.chernev@gmail.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Fix null dereference in _msm_gem_new

The crash was caused by locking an uninitialized lock during init of
drm_gem_object. The lock changed in the breaking commit, but the init
was not moved accordingly.

8<--- cut here ---
Unable to handle kernel NULL pointer dereference at virtual address 00000000
pgd = (ptrval)
[00000000] *pgd=00000000
Internal error: Oops: 5 [#1] PREEMPT SMP ARM
Modules linked in: msm(+) qcom_spmi_vadc qcom_vadc_common dm_mod usb_f_rndis rmi_i2c rmi_core qnoc_msm8974 icc_smd_rpm pm8941_pwrkey
CPU: 2 PID: 1020 Comm: udevd Not tainted 5.10.0-postmarketos-qcom-msm8974 #8
Hardware name: Generic DT based system
PC is at ww_mutex_lock+0x20/0xb0
LR is at _msm_gem_new+0x13c/0x298 [msm]
pc : [<c0be31e8>] lr : [<bf0b3404>] psr: 20000013
sp : c36e7ad0 ip : c3b3d800 fp : 00000000
r10: 00000001 r9 : c3b22800 r8 : 00000000
r7 : c3b23000 r6 : c3b3d600 r5 : c3b3d600 r4 : 00000000
r3 : c34b4780 r2 : c3b3d6f4 r1 : 00000000 r0 : 00000000
Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
Control: 10c5787d Table: 03ae406a DAC: 00000051
Process udevd (pid: 1020, stack limit = 0x(ptrval))
Stack: (0xc36e7ad0 to 0xc36e8000)
[...]
[<c0be31e8>] (ww_mutex_lock) from [<bf0b3404>] (_msm_gem_new+0x13c/0x298 [msm])
[<bf0b3404>] (_msm_gem_new [msm]) from [<bf0b3aa8>] (_msm_gem_kernel_new+0x20/0x190 [msm])
[<bf0b3aa8>] (_msm_gem_kernel_new [msm]) from [<bf0b4a30>] (msm_gem_kernel_new+0x24/0x2c [msm])
[<bf0b4a30>] (msm_gem_kernel_new [msm]) from [<bf0b8e2c>] (msm_gpu_init+0x308/0x548 [msm])
[<bf0b8e2c>] (msm_gpu_init [msm]) from [<bf060a90>] (adreno_gpu_init+0x13c/0x240 [msm])
[<bf060a90>] (adreno_gpu_init [msm]) from [<bf062b1c>] (a3xx_gpu_init+0x78/0x1dc [msm])
[<bf062b1c>] (a3xx_gpu_init [msm]) from [<bf05f394>] (adreno_bind+0x1cc/0x274 [msm])
[<bf05f394>] (adreno_bind [msm]) from [<c087a254>] (component_bind_all+0x11c/0x278)
[<c087a254>] (component_bind_all) from [<bf0b11d4>] (msm_drm_bind+0x18c/0x5b4 [msm])
[<bf0b11d4>] (msm_drm_bind [msm]) from [<c0879ea0>] (try_to_bring_up_master+0x200/0x2c8)
[<c0879ea0>] (try_to_bring_up_master) from [<c087a648>] (component_master_add_with_match+0xc8/0xfc)
[<c087a648>] (component_master_add_with_match) from [<bf0b0c3c>] (msm_pdev_probe+0x288/0x2c4 [msm])
[<bf0b0c3c>] (msm_pdev_probe [msm]) from [<c08844cc>] (platform_drv_probe+0x48/0x98)
[<c08844cc>] (platform_drv_probe) from [<c0881cc4>] (really_probe+0x108/0x528)
[<c0881cc4>] (really_probe) from [<c0882480>] (driver_probe_device+0x78/0x1d4)
[<c0882480>] (driver_probe_device) from [<c08828dc>] (device_driver_attach+0xa8/0xb0)
[<c08828dc>] (device_driver_attach) from [<c0882998>] (__driver_attach+0xb4/0x154)
[<c0882998>] (__driver_attach) from [<c087fa1c>] (bus_for_each_dev+0x78/0xb8)
[<c087fa1c>] (bus_for_each_dev) from [<c0880e98>] (bus_add_driver+0x10c/0x208)
[<c0880e98>] (bus_add_driver) from [<c0883504>] (driver_register+0x88/0x118)
[<c0883504>] (driver_register) from [<c0302098>] (do_one_initcall+0x50/0x2b0)
[<c0302098>] (do_one_initcall) from [<c03bace4>] (do_init_module+0x60/0x288)
[<c03bace4>] (do_init_module) from [<c03bdf1c>] (sys_finit_module+0xd4/0x120)
[<c03bdf1c>] (sys_finit_module) from [<c0300060>] (ret_fast_syscall+0x0/0x54)
Exception stack(0xc36e7fa8 to 0xc36e7ff0)
7fa0: 00020000 00000000 00000007 b6edd5b0 00000000 b6f2ff20
7fc0: 00020000 00000000 0000017b 0000017b b6eef980 bedc3a54 00473c99 00000000
7fe0: b6edd5b0 bedc3918 b6ed8a5f b6f6a8b0
Code: e3c3303f e593300c e1a04000 f590f000 (e1940f9f)
---[ end trace 277e2a3da40bbb76 ]---

Fixes: 6c0e3ea250476 ("drm/msm/gem: Switch over to obj->resv for locking")
Signed-off-by: Iskren Chernev <iskren.chernev@gmail.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Fix WARN_ON() splat in _free_object()

[ 192.062000] ------------[ cut here ]------------
[ 192.062498] WARNING: CPU: 3 PID: 2039 at drivers/gpu/drm/msm/msm_gem.c:381 put_iova_vmas+0x94/0xa0 [msm]
[ 192.062870] Modules linked in: snd_hrtimer snd_seq snd_seq_device rfcomm algif_hash algif_skcipher af_alg bnep xt_CHECKSUM nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_counter xt_tcpudp nft_compat cpufreq_powersave cpufreq_conservative q6asm_dai q6routing q6afe_dai q6adm bridge q6afe q6asm q6dsp_common q6core stp llc nf_tables libcrc32c nfnetlink snd_soc_wsa881x regmap_sdw soundwire_qcom gpio_wcd934x snd_soc_wcd934x wcd934x regmap_slimbus venus_enc venus_dec apr videobuf2_dma_sg qrtr_smd uvcvideo videobuf2_vmalloc videobuf2_memops ath10k_snoc ath10k_core hci_uart btqca btbcm mac80211 bluetooth snd_soc_sdm845 ath snd_soc_rt5663 snd_soc_qcom_common snd_soc_rl6231 soundwire_bus ecdh_generic ecc qcom_spmi_adc5 venus_core qcom_pon qcom_spmi_temp_alarm qcom_vadc_common v4l2_mem2mem videobuf2_v4l2 cfg80211 videobuf2_common hid_multitouch reset_qcom_pdc qcrypto qcom_rng rfkill qcom_q6v5_mss libarc4 libdes qrtr ns qcom_wdt socinfo slim_qcom_ngd_ctrl
[ 192.065739] pdr_interface qcom_q6v5_pas slimbus qcom_pil_info qcom_q6v5 qcom_sysmon qcom_common qcom_glink_smem qmi_helpers rmtfs_mem tcp_bbr sch_fq fuse ip_tables x_tables ipv6 crc_ccitt ti_sn65dsi86 i2c_hid msm mdt_loader llcc_qcom rtc_pm8xxx ocmem drm_kms_helper crct10dif_ce phy_qcom_qusb2 i2c_qcom_geni panel_simple drm pwm_bl
[ 192.066066] CPU: 3 PID: 2039 Comm: gnome-shell Tainted: G W 5.10.0-rc7-next-20201208 #1
[ 192.066068] Hardware name: LENOVO 81JL/LNVNB161216, BIOS 9UCN33WW(V2.06) 06/ 4/2019
[ 192.066072] pstate: 40400005 (nZcv daif +PAN -UAO -TCO BTYPE=--)
[ 192.066099] pc : put_iova_vmas+0x94/0xa0 [msm]
[ 192.066262] lr : put_iova_vmas+0x1c/0xa0 [msm]
[ 192.066403] sp : ffff800019efbbb0
[ 192.066405] x29: ffff800019efbbb0 x28: ffff800019efbd88
[ 192.066411] x27: 0000000000000000 x26: ffff109582efa400
[ 192.066417] x25: 0000000000000009 x24: 000000000000012b
[ 192.066422] x23: ffff109582efa438 x22: ffff109582efa450
[ 192.066427] x21: ffff109582efa528 x20: ffff1095cbd4f200
[ 192.066432] x19: ffff1095cbd4f200 x18: 0000000000000000
[ 192.066438] x17: 0000000000000000 x16: ffffc26c200ca750
[ 192.066727] x15: 0000000000000000 x14: 0000000000000000
[ 192.066741] x13: ffff1096fb8c9100 x12: 0000000000000002
[ 192.066754] x11: ffffffffffffffff x10: 0000000000000002
[ 192.067046] x9 : 0000000000000001 x8 : 0000000000000a36
[ 192.067060] x7 : ffff4e2ad9f11000 x6 : ffffc26c216d4000
[ 192.067212] x5 : ffffc26c2022661c x4 : ffff1095c2b98000
[ 192.067367] x3 : ffff1095cbd4f300 x2 : 0000000000000000
[ 192.067380] x1 : ffff1095c2b98000 x0 : 0000000000000000
[ 192.067667] Call trace:
[ 192.067734] put_iova_vmas+0x94/0xa0 [msm]
[ 192.068078] msm_gem_free_object+0xb4/0x110 [msm]
[ 192.068399] drm_gem_object_free+0x1c/0x30 [drm]
[ 192.068717] drm_gem_object_handle_put_unlocked+0xf0/0xf8 [drm]
[ 192.069032] drm_gem_object_release_handle+0x6c/0x88 [drm]
[ 192.069349] drm_gem_handle_delete+0x68/0xc0 [drm]
[ 192.069666] drm_gem_close_ioctl+0x30/0x48 [drm]
[ 192.069984] drm_ioctl_kernel+0xc0/0x110 [drm]
[ 192.070303] drm_ioctl+0x210/0x440 [drm]
[ 192.070588] __arm64_sys_ioctl+0xa8/0xf0
[ 192.070599] el0_svc_common.constprop.0+0x74/0x190
[ 192.070608] do_el0_svc+0x24/0x90
[ 192.070618] el0_svc+0x14/0x20
[ 192.070903] el0_sync_handler+0xb0/0xb8
[ 192.070911] el0_sync+0x174/0x180
[ 192.070918] ---[ end trace bee6b12a899001a3 ]---
[ 192.072140] ------------[ cut here ]------------

Fixes: 9b73bde39cf2 ("drm/msm: Fix use-after-free in msm_gem with carveout")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Iskren Chernev <iskren.chernev@gmail.com>

drm/msm: Fix use-after-free in msm_gem with carveout

When using gem with vram carveout the page allocation is managed via
drm_mm. The necessary drm_mm_node is allocated in add_vma, but it is
referenced in msm_gem_object as well. It is freed before the drm_mm_node
has been deallocated leading to use-after-free on every single vram
allocation.

Currently put_iova is called before put_pages in both
msm_gem_free_object and msm_gem_purge:

put_iova -> del_vma -> kfree(vma) // vma holds drm_mm_node
/* later */
put_pages -> put_pages_vram -> drm_mm_remove_node(
msm_obj->vram_node)
// vram_node is a ref to
// drm_mm_node; in _msm_gem_new

It looks like del_vma does nothing else other than freeing the vma
object and removing it from it's list, so delaying the deletion should
be harmless.

This patch splits put_iova in put_iova_spaces and put_iova_vmas, so the
vma can be freed after the mm_node has been deallocated with the mm.

Note: The breaking commit separated the vma allocation from within
msm_gem_object to outside, so the vram_node reference became outside the
msm_gem_object allocation, and freeing order was therefore overlooked.

Fixes: 4b85f7f5cf7 ("drm/msm: support for an arbitrary number of address spaces")
Signed-off-by: Iskren Chernev <iskren.chernev@gmail.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm/shrinker: Only iterate dontneed objs

In situations where the GPU is mostly idle, all or nearly all buffer
objects will be in the inactive list. But if the system is under memory
pressure (from something other than GPU), we could still get a lot of
shrinker calls. Which results in traversing a list of thousands of objs
and in the end finding nothing to shrink. Which isn't so efficient.

Instead split the inactive_list into two lists, one inactive objs which
are shrinkable, and a second one for those that are not. This way we
can avoid traversing objs which we know are not shrinker candidates.

v2: Fix inverted logic think-o

Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Protect obj->active_count under obj lock

Previously we only held obj lock in the _active_get() path, and relied
on atomic_dec_return() to not be racy in the _active_put() path where
obj lock was not held.

But this is a false sense of security. Unlike obj lifetime refcnt,
where you do not expect to *increase* the refcnt after the last put
(which would mean that something has gone horribly wrong with the
object liveness reference counting), the active_count can increase
again from zero. Racing _active_put()s and _active_get()s could leave
the obj on the wrong mm list.

But in the retire path, immediately after the _active_put(), the
_unpin_iova() would acquire obj lock. So just move the locking earlier
and rely on that to protect obj->active_count.

Fixes: c5c1643cef7a ("drm/msm: Drop struct_mutex from the retire path")
Signed-off-by: Rob Clark <robdclark@chromium.org>

mm: introduce vma_set_file function v5

Add the new vma_set_file() function to allow changing
vma->vm_file with the necessary refcount dance.

v2: add more users of this.
v3: add missing EXPORT_SYMBOL, rebase on mmap cleanup,
add comments why we drop the reference on two occasions.
v4: make it clear that changing an anonymous vma is illegal.
v5: move vma_set_file to mm/util.c

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> (v2)
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Andrew Morton <akpm@linux-foundation.org>
Link: https://patchwork.freedesktop.org/patch/399360/

Revert "mm: introduce vma_set_file function v4"

The kernel test robot is not happy with that.

This reverts commit 2b5b95b1ff3d70a95013a45e3b5b90f1daf42348.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/394773/

mm: introduce vma_set_file function v4

Add the new vma_set_file() function to allow changing
vma->vm_file with the necessary refcount dance.

v2: add more users of this.
v3: add missing EXPORT_SYMBOL, rebase on mmap cleanup,
add comments why we drop the reference on two occasions.
v4: make it clear that changing an anonymous vma is illegal.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> (v2)
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Link: https://patchwork.freedesktop.org/patch/394773/

drm/msm: Drop struct_mutex in shrinker path

Now that the inactive_list is protected by mm_lock, and everything
else on per-obj basis is protected by obj->resv, we no longer depend
on struct_mutex.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Drop struct_mutex in madvise path

The obj->lock is sufficient for what we need.

This *does* have the implication that userspace can try to shoot
themselves in the foot by racing madvise(DONTNEED) with submit. But
the result will be about the same if they did madvise(DONTNEED) before
the submit ioctl, ie. they might not get want they want if they race
with shrinker. But iova fault handling is robust enough, so userspace
is only shooting it's own foot.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Remove msm_gem_free_work

Now that we don't need struct_mutex in the free path, we can get rid of
the asynchronous free altogether.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Drop struct_mutex in free_object() path

Now that active_list/inactive_list is protected by mm_lock, we no longer
need dev->struct_mutex in the free_object() path.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Remove obj->gpu

It cannot be atomically updated with obj->active_count, and the only
purpose is a useless WARN_ON() (which becomes a buggy WARN_ON() once
retire_submits() is not serialized with incoming submits via
struct_mutex)

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Add priv->mm_lock to protect active/inactive lists

Rather than relying on the big dev->struct_mutex hammer, introduce a
more specific lock for protecting the bo lists.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Jordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Use correct drm_gem_object_put() in fail case

We only want to use the _unlocked() variant in the unlocked case.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm/gem: Switch over to obj->resv for locking

This also converts the special msm_gem_get_vaddr_active() to expect the
lock to already be held. There are two call-sites for this, one already
has the lock held, so it is more straightforward to just open-code the
locking for the other caller.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm/gem: Move locking in shrinker path

Move grabbing the bo lock into shrinker, with a msm_gem_trylock() to
skip over bo's that are already locked. This gets rid of the nested
lock classes.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm/gem: Add some _locked() helpers

When we cut-over to using dma_resv_lock/etc instead of msm_obj->lock,
we'll need these for the submit path (where resv->lock is already held).

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm/gem: Rename internal get_iova_locked helper

We'll need to introduce a _locked() version of msm_gem_get_iova(), so we
need to make that name available.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm/gem: Add obj->lock wrappers

This will make it easier to transition over to obj->resv locking for
everything that is per-bo locking.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

dma-mapping: split <linux/dma-mapping.h>

Split out all the bits that are purely for dma_map_ops implementations
and related code into a new <linux/dma-map-ops.h> header so that they
don't get pulled into all the drivers. That also means the architecture
specific <asm/dma-mapping.h> is not pulled in by <linux/dma-mapping.h>
any more, which leads to a missing includes that were pulled in by the
x86 or arm versions in a few not overly portable drivers.

Signed-off-by: Christoph Hellwig <hch@lst.de>

drm/msm: Introduce GEM object funcs

GEM object functions deprecate several similar callback interfaces in
struct drm_driver. This patch replaces the per-driver callbacks with
per-instance callbacks in msm. The only exception is gem_prime_mmap,
which is non-trivial to convert.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200923102159.24084-10-tzimmermann@suse.de

drm/msm: Fix premature purging of BO

In the case where we have a back-to-back submission that shares the same
BO, this BO will be prematurely moved to inactive_list while retiring the
first submit. But it will be still part of the second submit which is
being processed by the GPU. Now, if the shrinker happens to be triggered at
this point, it will result in a premature purging of this BO.

To fix this, we need to refcount BO while doing submit and retire. Then,
it should be moved to inactive list when this refcount becomes 0.

Signed-off-by: Akhil P Oommen <akhilpo@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Show process names in gem_describe

In $debugfs/gem we already show any vma(s) associated with an object.
Also show process names if the vma's address space is a per-process
address space.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Jordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: Bjorn Andersson <bjorn.andersson@linaro.org>

drm: msm: fix common struct sg_table related issues

The Documentation/DMA-API-HOWTO.txt states that the dma_map_sg() function
returns the number of the created entries in the DMA address space.
However the subsequent calls to the dma_sync_sg_for_{device,cpu}() and
dma_unmap_sg must be called with the original number of the entries
passed to the dma_map_sg().

struct sg_table is a common structure used for describing a non-contiguous
memory buffer, used commonly in the DRM and graphics subsystems. It
consists of a scatterlist with memory pages and DMA addresses (sgl entry),
as well as the number of scatterlist entries: CPU pages (orig_nents entry)
and DMA mapped pages (nents entry).

It turned out that it was a common mistake to misuse nents and orig_nents
entries, calling DMA-mapping functions with a wrong number of entries or
ignoring the number of mapped entries returned by the dma_map_sg()
function.

To avoid such issues, lets use a common dma-mapping wrappers operating
directly on the struct sg_table objects and use scatterlist page
iterators where possible. This, almost always, hides references to the
nents and orig_nents entries, making the code robust, easier to follow
and copy/paste safe.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Acked-by: Rob Clark <robdclark@gmail.com>

drm: allow limiting the scatter list size.

Add drm_device argument to drm_prime_pages_to_sg(), so we can
call dma_max_mapping_size() to figure the segment size limit
and call into __sg_alloc_table_from_pages() with the correct
limit.

This fixes virtio-gpu with sev. Possibly it'll fix other bugs
too given that drm seems to totaly ignore segment size limits
so far ...

v2: place max_segment in drm driver not gem object.
v3: move max_segment next to the other gem fields.
v4: just use dma_max_mapping_size().

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20200907112425.15610-2-kraxel@redhat.com

drm/msm: drop cache sync hack

Now that it isn't causing problems to use dma_map/unmap, we can drop the
hack of using dma_sync in certain cases.

Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Fix a null pointer access in msm_gem_shrinker_count()

Adding an msm_gem_object object to the inactive_list before completing
its initialization is a bad idea because shrinker may pick it up from the
inactive_list. Fix this by making sure that the initialization is complete
before moving the msm_obj object to the inactive list.

This patch fixes the below error:
[10027.553044] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000068
[10027.573305] Mem abort info:
[10027.590160] ESR = 0x96000006
[10027.597905] EC = 0x25: DABT (current EL), IL = 32 bits
[10027.614430] SET = 0, FnV = 0
[10027.624427] EA = 0, S1PTW = 0
[10027.632722] Data abort info:
[10027.638039] ISV = 0, ISS = 0x00000006
[10027.647459] CM = 0, WnR = 0
[10027.654345] user pgtable: 4k pages, 39-bit VAs, pgdp=00000001e3a6a000
[10027.672681] [0000000000000068] pgd=0000000198c31003, pud=0000000198c31003, pmd=0000000000000000
[10027.693900] Internal error: Oops: 96000006 [#1] PREEMPT SMP
[10027.738261] CPU: 3 PID: 214 Comm: kswapd0 Tainted: G S 5.4.40 #1
[10027.745766] Hardware name: Qualcomm Technologies, Inc. SC7180 IDP (DT)
[10027.752472] pstate: 80c00009 (Nzcv daif +PAN +UAO)
[10027.757409] pc : mutex_is_locked+0x14/0x2c
[10027.761626] lr : msm_gem_shrinker_count+0x70/0xec
[10027.766454] sp : ffffffc011323ad0
[10027.769867] x29: ffffffc011323ad0 x28: ffffffe677e4b878
[10027.775324] x27: 0000000000000cc0 x26: 0000000000000000
[10027.780783] x25: ffffff817114a708 x24: 0000000000000008
[10027.786242] x23: ffffff8023ab7170 x22: 0000000000000001
[10027.791701] x21: ffffff817114a080 x20: 0000000000000119
[10027.797160] x19: 0000000000000068 x18: 00000000000003bc
[10027.802621] x17: 0000000004a34210 x16: 00000000000000c0
[10027.808083] x15: 0000000000000000 x14: 0000000000000000
[10027.813542] x13: ffffffe677e0a3c0 x12: 0000000000000000
[10027.819000] x11: 0000000000000000 x10: ffffff8174b94340
[10027.824461] x9 : 0000000000000000 x8 : 0000000000000000
[10027.829919] x7 : 00000000000001fc x6 : ffffffc011323c88
[10027.835373] x5 : 0000000000000001 x4 : ffffffc011323d80
[10027.840832] x3 : ffffffff0477b348 x2 : 0000000000000000
[10027.846290] x1 : ffffffc011323b68 x0 : 0000000000000068
[10027.851748] Call trace:
[10027.854264] mutex_is_locked+0x14/0x2c
[10027.858121] msm_gem_shrinker_count+0x70/0xec
[10027.862603] shrink_slab+0xc0/0x4b4
[10027.866187] shrink_node+0x4a8/0x818
[10027.869860] kswapd+0x624/0x890
[10027.873097] kthread+0x11c/0x12c
[10027.876424] ret_from_fork+0x10/0x18
[10027.880102] Code: f9000bf3 910003fd aa0003f3 d503201f (f9400268)
[10027.886362] ---[ end trace df5849a1a3543251 ]---
[10027.891518] Kernel panic - not syncing: Fatal exception

Signed-off-by: Akhil P Oommen <akhilpo@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Don't call dma_buf_vunmap without _vmap

I honestly don't exactly understand what's going on here, but the
current code is wrong for sure: It calls dma_buf_vunmap without ever
calling dma_buf_vmap.

What I'm not sure about is whether the WARN_ON is correct:
- msm imports dma-buf using drm_prime_sg_to_page_addr_arrays. Which is
a pretty neat layering violation of how you shouldn't peek behind
the curtain of the dma-buf exporter, but par for course. Note that
all the nice new helpers don't (and we should probably have a bit a
warning about this in the kerneldoc).

- but then in the get_vaddr() in msm_gem.c, we seems to happily wrap a
vmap() around any object with ->pages set (so including imported
dma-buf).

- I'm not seeing any guarantees that userspace can't use an imported
dma-buf for e.g. MSM_SUBMIT_CMD_BUF in a5xx_submit_in_rb, so no
guarantees that an imported dma-buf won't end up with a ->vaddr set.

But even if that WARN_ON is wrong, cleaning up a vmap() done by msm by
calling dma_buf_vunmap is the wrong thing to do.

v2: Rob said in review that we do indeed have a gap in get_vaddr() that
needs to be plugged. But the users I've found aren't legit users on
imported dma-buf, so we can just reject that.

Reviewed-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
Link: https://patchwork.freedesktop.org/patch/msgid/20200514201117.465146-1-daniel.vetter@ffwll.ch

drm/msm: remove _unlocked suffix in drm_gem_object_put_unlocked

Spelling out _unlocked for each and every driver is a annoying.
Especially if we consider how many drivers, do not know (or need to)
about the horror stories involving struct_mutex.

Just drop the suffix. It makes the API cleaner.

Done via the following script:

__from=drm_gem_object_put_unlocked
__to=drm_gem_object_put
for __file in $(git grep --name-only $__from); do
sed -i "s/$__from/$__to/g" $__file;
done

Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: David Airlie <airlied@linux.ie>
Signed-off-by: Emil Velikov <emil.velikov@collabora.com>
Acked-by: Sam Ravnborg <sam@ravnborg.org>
Acked-by: Thomas Zimmermann <tzimmermann@suse.de>
Link: https://patchwork.freedesktop.org/patch/msgid/20200515095118.2743122-25-emil.l.velikov@gmail.com

drm/gem: add _locked suffix to drm_gem_object_put

Vast majority of DRM (core and drivers) are struct_mutex free.

As such we have only a handful of cases where the locked helper should
be used. Make that stand out a little bit better.

Done via the following script:

__from=drm_gem_object_put
__to=drm_gem_object_put_locked

for __file in $(git grep --name-only --word-regexp $__from); do
sed -i "s/\<$__from\>/$__to/g" $__file;
done

Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: linux-arm-msm@vger.kernel.org
Signed-off-by: Emil Velikov <emil.velikov@collabora.com>
Acked-by: Sam Ravnborg <sam@ravnborg.org>
Reviewed-by: Steven Price <steven.price@arm.com>
Acked-by: Thomas Zimmermann <tzimmermann@suse.de>
Link: https://patchwork.freedesktop.org/patch/msgid/20200515095118.2743122-12-emil.l.velikov@gmail.com

drm/msm: add internal MSM_BO_MAP_PRIV flag

This flag sets IOMMU_PRIV, which is required for some a6xx GMU objects.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Reviewed-by: Jordan Crouse <jcrouse@codeauorora.org>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: add msm_gem_get_and_pin_iova_range

This function allows pinning iova to a specific page range (for a6xx GMU).

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Reviewed-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Use the correct dma_sync calls harder

Looks like the dma_sync calls don't do what we want on armv7 either.
Fixes:

Unable to handle kernel paging request at virtual address 50001000
pgd = (ptrval)
[50001000] *pgd=00000000
Internal error: Oops: 805 [#1] SMP ARM
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.3.0-rc6-00271-g9f159ae07f07 #4
Hardware name: Freescale i.MX53 (Device Tree Support)
PC is at v7_dma_clean_range+0x20/0x38
LR is at __dma_page_cpu_to_dev+0x28/0x90
pc : [<c011c76c>] lr : [<c01181c4>] psr: 20000013
sp : d80b5a88 ip : de96c000 fp : d840ce6c
r10: 00000000 r9 : 00000001 r8 : d843e010
r7 : 00000000 r6 : 00008000 r5 : ddb6c000 r4 : 00000000
r3 : 0000003f r2 : 00000040 r1 : 50008000 r0 : 50001000
Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
Control: 10c5387d Table: 70004019 DAC: 00000051
Process swapper/0 (pid: 1, stack limit = 0x(ptrval))

Signed-off-by: Rob Clark <robdclark@chromium.org>
Fixes: 3de433c5b38a ("drm/msm: Use the correct dma_sync calls in msm_gem")
Tested-by: Fabio Estevam <festevam@gmail.com>

drm/msm: drop use of drmP.h

Drop the deprecated drmP.h header file, and trim msm_drv.h
to the relevant include files.

This resulted in a suprisingly many edits as many files relied
on headers included via msm_drv.h.
But msm_drv.h is not supposed to carry include files it do not need, so
the individual files have to include what extra they needs.

v2:
- Rebased on top of https://gitlab.freedesktop.org/drm/msm.git msm-next

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Jordan Crouse <jcrouse@codeaurora.org>
Cc: Jeykumar Sankaran <jsanka@codeaurora.org>
Cc: Bruce Wang <bzwang@chromium.org>
Cc: Shayenne Moura <shayenneluzmoura@gmail.com>
Cc: Mamta Shukla <mamtashukla555@gmail.com>
Cc: Jonathan Marek <jonathan@marek.ca>
Cc: Carsten Behling <carsten.behling@googlemail.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Maxime Ripard <maxime.ripard@bootlin.com>
Cc: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Cc: Sibi Sankar <sibis@codeaurora.org>
Cc: Todor Tomov <todor.tomov@linaro.org>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
Signed-off-by: Sean Paul <seanpaul@chromium.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20190804065551.GA5211@ravnborg.org

dma-buf: rename reservation_object to dma_resv

Be more consistent with the naming of the other DMA-buf objects.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/323401/

drm/msm: Use the correct dma_sync calls in msm_gem

[subject was: drm/msm: shake fist angrily at dma-mapping]

So, using dma_sync_* for our cache needs works out w/ dma iommu ops, but
it falls appart with dma direct ops. The problem is that, depending on
display generation, we can have either set of dma ops (mdp4 and dpu have
iommu wired to mdss node, which maps to toplevel drm device, but mdp5
has iommu wired up to the mdp sub-node within mdss).

Fixes this splat on mdp5 devices:

Unable to handle kernel paging request at virtual address ffffffff80000000
Mem abort info:
ESR = 0x96000144
Exception class = DABT (current EL), IL = 32 bits
SET = 0, FnV = 0
EA = 0, S1PTW = 0
Data abort info:
ISV = 0, ISS = 0x00000144
CM = 1, WnR = 1
swapper pgtable: 4k pages, 48-bit VAs, pgdp=00000000810e4000
[ffffffff80000000] pgd=0000000000000000
Internal error: Oops: 96000144 [#1] SMP
Modules linked in: btqcomsmd btqca bluetooth cfg80211 ecdh_generic ecc rfkill libarc4 panel_simple msm wcnss_ctrl qrtr_smd drm_kms_helper venus_enc venus_dec videobuf2_dma_sg videobuf2_memops drm venus_core ipv6 qrtr qcom_wcnss_pil v4l2_mem2mem qcom_sysmon videobuf2_v4l2 qmi_helpers videobuf2_common crct10dif_ce mdt_loader qcom_common videodev qcom_glink_smem remoteproc bmc150_accel_i2c bmc150_magn_i2c bmc150_accel_core bmc150_magn snd_soc_lpass_apq8016 snd_soc_msm8916_analog mms114 mc nf_defrag_ipv6 snd_soc_lpass_cpu snd_soc_apq8016_sbc industrialio_triggered_buffer kfifo_buf snd_soc_lpass_platform snd_soc_msm8916_digital drm_panel_orientation_quirks
CPU: 2 PID: 33 Comm: kworker/2:1 Not tainted 5.3.0-rc2 #1
Hardware name: Samsung Galaxy A5U (EUR) (DT)
Workqueue: events deferred_probe_work_func
pstate: 80000005 (Nzcv daif -PAN -UAO)
pc : __clean_dcache_area_poc+0x20/0x38
lr : arch_sync_dma_for_device+0x28/0x30
sp : ffff0000115736a0
x29: ffff0000115736a0 x28: 0000000000000001
x27: ffff800074830800 x26: ffff000011478000
x25: 0000000000000000 x24: 0000000000000001
x23: ffff000011478a98 x22: ffff800009fd1c10
x21: 0000000000000001 x20: ffff800075ad0a00
x19: 0000000000000000 x18: ffff0000112b2000
x17: 0000000000000000 x16: 0000000000000000
x15: 00000000fffffff0 x14: ffff000011455d70
x13: 0000000000000000 x12: 0000000000000028
x11: 0000000000000001 x10: ffff00001106c000
x9 : ffff7e0001d6b380 x8 : 0000000000001000
x7 : ffff7e0001d6b380 x6 : ffff7e0001d6b382
x5 : 0000000000000000 x4 : 0000000000001000
x3 : 000000000000003f x2 : 0000000000000040
x1 : ffffffff80001000 x0 : ffffffff80000000
Call trace:
__clean_dcache_area_poc+0x20/0x38
dma_direct_sync_sg_for_device+0xb8/0xe8
get_pages+0x22c/0x250 [msm]
msm_gem_get_and_pin_iova+0xdc/0x168 [msm]
...

Fixes the combination of two patches:

Fixes: 0036bc73ccbe (drm/msm: stop abusing dma_map/unmap for cache)
Fixes: 449fa54d6815 (dma-direct: correct the physical addr in dma_direct_sync_sg_for_cpu/device)
Tested-by: Stephan Gerhold <stephan@gerhold.net>
Signed-off-by: Rob Clark <robdclark@chromium.org>
[seanpaul changed subject to something more desriptive]
Signed-off-by: Sean Paul <seanpaul@chromium.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20190730214633.17820-1-robdclark@gmail.com

drm/msm: stop abusing dma_map/unmap for cache

Recently splats like this started showing up:

WARNING: CPU: 4 PID: 251 at drivers/iommu/dma-iommu.c:451 __iommu_dma_unmap+0xb8/0xc0
Modules linked in: ath10k_snoc ath10k_core fuse msm ath mac80211 uvcvideo cfg80211 videobuf2_vmalloc videobuf2_memops vide
CPU: 4 PID: 251 Comm: kworker/u16:4 Tainted: G W 5.2.0-rc5-next-20190619+ #2317
Hardware name: LENOVO 81JL/LNVNB161216, BIOS 9UCN23WW(V1.06) 10/25/2018
Workqueue: msm msm_gem_free_work [msm]
pstate: 80c00005 (Nzcv daif +PAN +UAO)
pc : __iommu_dma_unmap+0xb8/0xc0
lr : __iommu_dma_unmap+0x54/0xc0
sp : ffff0000119abce0
x29: ffff0000119abce0 x28: 0000000000000000
x27: ffff8001f9946648 x26: ffff8001ec271068
x25: 0000000000000000 x24: ffff8001ea3580a8
x23: ffff8001f95ba010 x22: ffff80018e83ba88
x21: ffff8001e548f000 x20: fffffffffffff000
x19: 0000000000001000 x18: 00000000c00001fe
x17: 0000000000000000 x16: 0000000000000000
x15: ffff000015b70068 x14: 0000000000000005
x13: 0003142cc1be1768 x12: 0000000000000001
x11: ffff8001f6de9100 x10: 0000000000000009
x9 : ffff000015b78000 x8 : 0000000000000000
x7 : 0000000000000001 x6 : fffffffffffff000
x5 : 0000000000000fff x4 : ffff00001065dbc8
x3 : 000000000000000d x2 : 0000000000001000
x1 : fffffffffffff000 x0 : 0000000000000000
Call trace:
__iommu_dma_unmap+0xb8/0xc0
iommu_dma_unmap_sg+0x98/0xb8
put_pages+0x5c/0xf0 [msm]
msm_gem_free_work+0x10c/0x150 [msm]
process_one_work+0x1e0/0x330
worker_thread+0x40/0x438
kthread+0x12c/0x130
ret_from_fork+0x10/0x18
---[ end trace afc0dc5ab81a06bf ]---

Not quite sure what triggered that, but we really shouldn't be abusing
dma_{map,unmap}_sg() for cache maint.

Cc: Stephen Boyd <sboyd@kernel.org>
Tested-by: Stephen Boyd <swboyd@chromium.org>
Reviewed-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@chromium.org>
Signed-off-by: Sean Paul <seanpaul@chromium.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20190630124735.27786-1-robdclark@gmail.com

drm/msm: Drop robj from msm_gem_new_impl

Only user was the prime import, and drm_prime.c takes care of that
now.

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Acked-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
Link: https://patchwork.freedesktop.org/patch/msgid/20190614203615.12639-46-daniel.vetter@ffwll.ch

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234

Based on 1 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license version 2 as
published by the free software foundation this program is
distributed in the hope that it will be useful but without any
warranty without even the implied warranty of merchantability or
fitness for a particular purpose see the gnu general public license
for more details you should have received a copy of the gnu general
public license along with this program if not see http www gnu org
licenses

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-only

has been chosen to replace the boilerplate/reference in 503 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Alexios Zavras <alexios.zavras@intel.com>
Reviewed-by: Allison Randal <allison@lohutok.net>
Reviewed-by: Enrico Weigelt <info@metux.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190602204653.811534538@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

drm/msm: correct attempted NULL pointer dereference in put_iova

put_iova() would attempt to dereference a NULL pointer via the
address space pointer when no IOMMU is present. Correct this by adding
the appropriate check.

Signed-off-by: Brian Masney <masneyb@onstation.org>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: correct attempted NULL pointer dereference in debugfs

msm_gem_describe() would attempt to dereference a NULL pointer via the
address space pointer when no IOMMU is present. Correct this by adding
the appropriate check.

Signed-off-by: Brian Masney <masneyb@onstation.org>
Fixes: 575f0485508b ("drm/msm: Clean up and enhance the output of the 'gem' debugfs node")
Signed-off-by: Sean Paul <seanpaul@chromium.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20190513234105.7531-2-masneyb@onstation.org

drm/msm: Implement .gem_free_object_unlocked

We use a llist and a worker to delay the object cleanup. This avoids
taking mmap_sem and struct_mutex in the wrong order when calling
drm_gem_object_put_unlocked() from drm_gem_mmap().

Fixes lockdep problem with copy_from_user() in msm_ioctl_gem_submit().

Signed-off-by: Kristian H. Kristensen <hoegsberg@chromium.org>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: don't allocate pages from the MOVABLE zone

The pages backing the GEM objects are kept pinned in place as
long as they are alive, so they must not be allocated from the
MOVABLE zone. Blocking page migration for too long will cause
the VM subsystem headaches and will outright break CMA, as a
few pinned pages in CMA will lead to failure to find the
required large contiguous regions.

Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm: msm: Switch to use drm_gem_object reservation_object

Now that the base struct drm_gem_object has a reservation_object, use it
and remove the private BO one.

We can't use the drm_gem_reservation_object_wait() helper for MSM
because (in theory) msm_gem_cpu_prep() will also do some cache
maintenance on the GEM object.

Cc: David Airlie <airlied@linux.ie>
Cc: linux-arm-msm@vger.kernel.org
Cc: dri-devel@lists.freedesktop.org
Cc: freedreno@lists.freedesktop.org
Signed-off-by: Rob Herring <robh@kernel.org>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: Rob Clark <robdclark@gmail.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190202154158.10443-4-robh@kernel.org
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>

drm/msm: honor GPU_READONLY flag

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: honor GPU_READONLY flag

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: fix build warning for 64-bit seqno

Fixes: b312d8ca3a7c ("dma-buf: make fence sequence numbers 64 bit v2")
Acked-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/msm: implement a2xx mmu

A2XX has its own very simple MMU.

Added a msm_use_mmu() function because we can't rely on iommu_present to
decide to use MMU or not.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: use contiguous vram for MSM_BO_SCANOUT when possible

Makes it possible to have MMU for GPU but not display.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Add a name field for gem objects

For debugging purposes it is useful to assign descriptions
to buffers so that we know what they are used for. Add
a field to the buffer object and use that to name the various
kernel side allocations which ends up looking like like this
in /d/dri/X/gem:

flags id ref offset kaddr size madv name
00040000: I 0 ( 1) 00000000 0000000070b79eca 00004096 memptrs
vmas: [gpu: 01000000,mapped,inuse=1]
00020000: I 0 ( 1) 00000000 0000000031ed4074 00032768 ring0

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Count how many times iova memory is pinned

Add a reference count to track how many times a particular
chunk of iova memory is pinned (mapped) in the iomu and
add msm_gem_unpin_iova to give up references.

It is important to note that msm_gem_unpin_iova replaces
msm_gem_put_iova because the new implicit behavior
that an assigned iova in a given vma is now valid for the
life of the buffer and what we are really focusing on is
the use of that iova.

For now the unmappings are lazy; once the reference counts
go to zero they *COULD* be unmapped dynamically but that
will require an outside force such as a shrinker or
mm_notifiers. For now, we're just focusing on getting
the counting right and setting ourselves up to be ready
for the future.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Add msm_gem_get_and_pin_iova()

Add a new function to get and pin the iova memory in one
step (basically renaming the old msm_gem_get_iova function)
and switch msm_gem_get_iova() to only allocate an iova but
not map it in the IOMMU. This is only currently used by
msm_ioctl_gem_info() since all other users of of the iova
expect that the memory be immediately available.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Clean up and enhance the output of the 'gem' debugfs node

Add headers for the 'gem' debugfs file to make it easier to remember
what all the values mean and move the list of virtual address regions
to the next line and add the name and map status to make it clearer
what we are looking at.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Split msm_gem_get_iova into two steps

Split the operation of msm_gem_get_iova into two operations:
1) allocate an iova and 2) map (pin) the backing memory int the
iommu. This is the first step toward allowing memory pinning
to occur independently of the iova management.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Remove sgt from the mmu unmap function

The scatter gather table doesn't need to be passed in for the
MMU unmap function.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Add a common function to free kernel buffer objects

Buffer objects allocated with msm_gem_kernel_new() are mostly
freed the same way so we can save a few lines of code with a
common function.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Gracefully handle failure in _msm_gem_kernel_new

If any of the function calls in _msm_gem_kernel_new fail we need
to make sure to dereference the GEM object with the appropriate
function for the current locking state.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm: msm: Use DRM_DEV_* instead of dev_*

Use DRM_DEV_INFO/ERROR/WARN instead of dev_info/err/debug to generate
drm-formatted specific log messages so that it will be easy to
differentiate in case of multiple instances of driver.

Signed-off-by: Mamta Shukla <mamtashukla555@gmail.com>
Signed-off-by: Rob Clark <robdclark@gmail.com>

gpu: drm: msm: Change return type to vm_fault_t

Use new return type vm_fault_t for fault handler. For
now, this is just documenting that the function returns
a VM_FAULT value rather than an errno. Once all instances
are converted, vm_fault_t will become a distinct type.

Ref- commit 1c8f422059ae ("mm: change return type to vm_fault_t")

Previously vm_insert_mixed() returns err which driver
mapped into VM_FAULT_* type. The new function
vmf_insert_mixed() will replace this inefficiency by
returning VM_FAULT_* type.

vmf_error() is the newly introduce inline function
in 4.17-rc6.

Signed-off-by: Souptick Joarder <jrdr.linux@gmail.com>
Reviewed-by: Matthew Wilcox <mawilcox@microsoft.com>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Fix possible null dereference on failure of get_pages()

Commit 62e3a3e342af changed get_pages() to initialise
msm_gem_object::pages before trying to initialise msm_gem_object::sgt,
so that put_pages() would properly clean up pages in the failure
case.

However, this means that put_pages() now needs to check that
msm_gem_object::sgt is not null before trying to clean it up, and
this check was only applied to part of the cleanup code. Move
it all into the conditional block. (Strictly speaking we don't
need to make the kfree() conditional, but since we can't avoid
checking for null ourselves we may as well do so.)

Fixes: 62e3a3e342af ("drm/msm: fix leak in failed get_pages")
Signed-off-by: Ben Hutchings <ben.hutchings@codethink.co.uk>
Reviewed-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Fix possible null dereference on failure of get_pages()

Commit 62e3a3e342af changed get_pages() to initialise
msm_gem_object::pages before trying to initialise msm_gem_object::sgt,
so that put_pages() would properly clean up pages in the failure
case.

However, this means that put_pages() now needs to check that
msm_gem_object::sgt is not null before trying to clean it up, and
this check was only applied to part of the cleanup code. Move
it all into the conditional block. (Strictly speaking we don't
need to make the kfree() conditional, but since we can't avoid
checking for null ourselves we may as well do so.)

Fixes: 62e3a3e342af ("drm/msm: fix leak in failed get_pages")
Signed-off-by: Ben Hutchings <ben.hutchings@codethink.co.uk>
Reviewed-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm/dsi: fix direct caller of msm_gem_free_object()

This should be using drm_gem_object_put(). Also since this is done only
in driver unload path, we don't need to synchronize setting tx_gem_obj
to NULL, so juse use the _unlocked() variant.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Replace gem_object deprecated functions

drm_gem_object_{reference,unreference,unreference_unlocked} are
deprecated functions, and merely alias to the get/put functions.
Switch to the new names.

Signed-off-by: Steve Kowalik <steven@wedontsleep.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: fix leak in failed get_pages

get_pages doesn't keep a reference of the pages allocated
when it fails later in the code path. This can lead to
a memory leak. Keep reference of the allocated pages so
that it can be freed when msm_gem_free_object gets called
later during cleanup.

Signed-off-by: Prakash Kamliya <pkamliya@codeaurora.org>
Signed-off-by: Sharat Masetty <smasetty@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: add special _get_vaddr_active() for cmdstream dumps

Prep work for adding a debugfs file that dumps just submits which
trigger hangs/faults. In this case the bo may already be in the
MADV_DONTNEED state, but will be still on the active list (since
the submit hasn't completed yet). So the normal check that the
bo is in the WILLNEED state does not apply. (But of course the bo
should definitely not be in the PURGED state!)

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: fix _NO_IMPLICIT fencing case

We need to call reservation_object_reserve_shared() in both cases, but
this wasn't happening in the _NO_IMPLICIT submit case.

Fixes: f0a42bb ("drm/msm: submit support for in-fences")
Reported-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: fix _NO_IMPLICIT fencing case

We need to call reservation_object_reserve_shared() in both cases, but
this wasn't happening in the _NO_IMPLICIT submit case.

Fixes: f0a42bb ("drm/msm: submit support for in-fences")
Reported-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: fix return value check in _msm_gem_kernel_new()

In case of error, the function msm_gem_get_vaddr() returns ERR_PTR()
and never returns NULL. The NULL test in the return value check should
be replaced with IS_ERR().

Fixes: 8223286d62e2 ("drm/msm: Add a helper function for in-kernel
buffer allocations")
Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Add a helper function for in-kernel buffer allocations

Nearly all of the buffer allocations for kernel allocate an buffer object,
virtual address and GPU iova at the same time. Make a helper function to
handle the details.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
[dropped msm_fbdev conversion to new helper, since it interferes with
display-handover work, where we want to separate allocation and mapping]
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: fix WARN_ON in add_vma() with no iommu

While I was testing the upcoming adv7533 CEC support with my Dragonboard c410
I encountered this warning several times during boot:

[ 4.408309] WARNING: CPU: 3 PID: 1347 at drivers/gpu/drm/msm/msm_gem.c:312 add_vma+0x78/0x88 [msm]
[ 4.412951] Modules linked in: snd_soc_hdmi_codec adv7511 cec qcom_wcnss_pil msm mdt_loader drm_kms_helper msm_rng rng_core drm
[ 4.421728] CPU: 3 PID: 1347 Comm: kworker/3:3 Not tainted 4.13.0-rc1-dragonboard #111
[ 4.433090] Hardware name: Qualcomm Technologies, Inc. APQ 8016 SBC (DT)
[ 4.441081] Workqueue: events deferred_probe_work_func
[ 4.447929] task: ffff800031243600 task.stack: ffff800003394000
[ 4.453023] PC is at add_vma+0x78/0x88 [msm]
[ 4.458823] LR is at _msm_gem_new+0xd4/0x188 [msm]
[ 4.463207] pc : [<ffff000000ac01f8>] lr : [<ffff000000ac06b4>] pstate: 40000145
[ 4.467811] sp : ffff8000033978a0
[ 4.475357] x29: ffff8000033978a0 x28: ffff8000031dea18
[ 4.478572] x27: ffff800003933a00 x26: ffff800003b39800
[ 4.483953] x25: ffff8000338ff800 x24: 0000000000000001
[ 4.489249] x23: 0000000000000000 x22: ffff800003b39800
[ 4.494544] x21: ffff8000338ff800 x20: 0000000000000000
[ 4.499839] x19: ffff800003932600 x18: 0000000000000001
[ 4.505135] x17: 0000ffff8969e9e0 x16: ffff7e00000ce7a0
[ 4.510429] x15: ffffffffffffffff x14: ffff8000833977ef
[ 4.515724] x13: ffff8000033977f3 x12: 0000000000000038
[ 4.521020] x11: 0101010101010101 x10: ffffff7f7fff7f7f
[ 4.526315] x9 : 0000000000000000 x8 : ffff800003932800
[ 4.531633] x7 : 0000000000000000 x6 : 000000000000003f
[ 4.531644] x5 : 0000000000000040 x4 : 0000000000000000
[ 4.531650] x3 : ffff800031243600 x2 : 0000000000000000
[ 4.531655] x1 : 0000000000000000 x0 : 0000000000000000
[ 4.531670] Call trace:
[ 4.531676] Exception stack(0xffff8000033976c0 to 0xffff8000033977f0)
[ 4.531683] 76c0: ffff800003932600 0001000000000000 ffff8000033978a0 ffff000000ac01f8
[ 4.531688] 76e0: 0000000000000140 0000000000000000 ffff800003932550 ffff800003397780
[ 4.531694] 7700: ffff800003397730 ffff000008261ce8 0000000000000000 ffff8000031d2f80
[ 4.531699] 7720: ffff800003397800 ffff0000081d671c 0000000000000140 0000000000000000
[ 4.531705] 7740: ffff000000ac04c0 0000000000004003 ffff800003397908 00000000014080c0
[ 4.531710] 7760: 0000000000000000 ffff800003b39800 0000000000000000 0000000000000000
[ 4.531716] 7780: 0000000000000000 ffff800031243600 0000000000000000 0000000000000040
[ 4.531721] 77a0: 000000000000003f 0000000000000000 ffff800003932800 0000000000000000
[ 4.531726] 77c0: ffffff7f7fff7f7f 0101010101010101 0000000000000038 ffff8000033977f3
[ 4.531730] 77e0: ffff8000833977ef ffffffffffffffff
[ 4.531881] [<ffff000000ac01f8>] add_vma+0x78/0x88 [msm]
[ 4.532011] [<ffff000000ac06b4>] _msm_gem_new+0xd4/0x188 [msm]
[ 4.532134] [<ffff000000ac1900>] msm_gem_new+0x10/0x18 [msm]
[ 4.532260] [<ffff000000acb274>] msm_dsi_host_modeset_init+0x17c/0x268 [msm]
[ 4.532384] [<ffff000000ac9024>] msm_dsi_modeset_init+0x34/0x1b8 [msm]
[ 4.532504] [<ffff000000ab6168>] modeset_init+0x408/0x488 [msm]
[ 4.532623] [<ffff000000ab6c4c>] mdp5_kms_init+0x2b4/0x338 [msm]
[ 4.532745] [<ffff000000abeff8>] msm_drm_bind+0x218/0x4e8 [msm]
[ 4.532755] [<ffff00000855d744>] try_to_bring_up_master+0x1f4/0x318
[ 4.532762] [<ffff00000855d900>] component_add+0x98/0x180
[ 4.532887] [<ffff000000ac8da0>] dsi_dev_probe+0x18/0x28 [msm]
[ 4.532895] [<ffff000008565fe8>] platform_drv_probe+0x58/0xc0
[ 4.532901] [<ffff00000856410c>] driver_probe_device+0x324/0x458
[ 4.532907] [<ffff00000856440c>] __device_attach_driver+0xac/0x170
[ 4.532913] [<ffff000008561ef4>] bus_for_each_drv+0x4c/0x98
[ 4.532918] [<ffff000008563c38>] __device_attach+0xc0/0x160
[ 4.532924] [<ffff000008564530>] device_initial_probe+0x10/0x18
[ 4.532929] [<ffff000008562f84>] bus_probe_device+0x94/0xa0
[ 4.532934] [<ffff0000085635d4>] deferred_probe_work_func+0x8c/0xe8
[ 4.532941] [<ffff0000080d79bc>] process_one_work+0x1d4/0x330
[ 4.532946] [<ffff0000080d7b60>] worker_thread+0x48/0x468
[ 4.532952] [<ffff0000080ddae4>] kthread+0x12c/0x130
[ 4.532958] [<ffff000008082f10>] ret_from_fork+0x10/0x40
[ 4.532962] ---[ end trace b1ac6888ec40b0bb ]---

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: unlock on error in msm_gem_get_iova()

We recently added locking to this function but there was a direct return
that was overlooked where we need to unlock.

Fixes: 0e08270a1f01 ("drm/msm: Separate locking of buffer resources from struct_mutex")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Separate locking of buffer resources from struct_mutex

Buffer object specific resources like pages, domains, sg list
need not be protected with struct_mutex. They can be protected
with a buffer object level lock. This simplifies locking and
makes it easier to avoid potential recursive locking scenarios
for SVM involving mmap_sem and struct_mutex. This also removes
unnecessary serialization when creating buffer objects, and also
between buffer object creation and GPU command submission.

Signed-off-by: Sushmita Susheelendra <ssusheel@codeaurora.org>
[robclark: squash in handling new locking for shrinker]
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: support for an arbitrary number of address spaces

It means we have to do a list traversal where we once had an index into
a table. But the list will normally have one or two entries.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: refactor how we handle vram carveout buffers

Pull some of the logic out into msm_gem_new() (since we don't need to
care about the imported-bo case), and don't defer allocating pages. The
latter is generally a good idea, since if we are using VRAM carveout to
allocate contiguous buffers (ie. no IOMMU), the allocation is more
likely to fail. So failing at allocation time is a more sane option.
Plus this simplifies things in the next patch.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: pass address-space to _get_iova() and friends

No functional change, that will come later. But this will make it
easier to deal with dynamically created address spaces (ie. per-
process pagetables for gpu).

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: fix locking inconsistency for gpu->hw_init()

Most, but not all, paths where calling the with struct_mutex held. The
fast-path in msm_gem_get_iova() (plus some sub-code-paths that only run
the first time) was masking this issue.

So lets just always hold struct_mutex for hw_init(). And sprinkle some
WARN_ON()'s and might_lock() to avoid this sort of problem in the
future.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Take the mutex before calling msm_gem_new_impl

Amongst its other duties, msm_gem_new_impl adds the newly created
GEM object to the shared inactive list which may also be actively
modifiying the list during submission. All the paths to modify
the list are protected by the mutex except for the one through
msm_gem_import which can end up causing list corruption.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
[add extra WARN_ON(!mutex_is_locked(&dev->struct_mutex))]
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm: drop drm_[cm]alloc* helpers

Now that drm_[cm]alloc* helpers are simple one line wrappers around
kvmalloc_array and drm_free_large is just kvfree alias we can drop
them and replace by their native forms.

This shouldn't introduce any functional change.

Changes since v1
- fix typo in drivers/gpu//drm/etnaviv/etnaviv_gem.c - noticed by 0day
build robot

Suggested-by: Daniel Vetter <daniel@ffwll.ch>
Signed-off-by: Michal Hocko <mhocko@suse.com>drm: drop drm_[cm]alloc* helpers
[danvet: Fixup vgem which grew another user very recently.]
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: Christian König <christian.koenig@amd.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170517122312.GK18247@dhcp22.suse.cz

drm/msm: Don't allow zero sized buffer objects

Zero sized buffer objects tend to make various bits of the GEM
infrastructure complain:

WARNING: CPU: 1 PID: 2323 at drivers/gpu/drm/drm_mm.c:389 drm_mm_insert_node_generic+0x258/0x2f0
Modules linked in:

CPU: 1 PID: 2323 Comm: drm-api-test Tainted: G W 4.9.0-rc4-00906-g693af44 #213
Hardware name: Qualcomm Technologies, Inc. DB820c (DT)
task: ffff8000d7353400 task.stack: ffff8000d7720000
PC is at drm_mm_insert_node_generic+0x258/0x2f0
LR is at drm_vma_offset_add+0x4c/0x70

Zero sized buffers serve no appreciable value to the user so disallow
them at create time.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Don't allow zero sized buffer objects

Zero sized buffer objects tend to make various bits of the GEM
infrastructure complain:

WARNING: CPU: 1 PID: 2323 at drivers/gpu/drm/drm_mm.c:389 drm_mm_insert_node_generic+0x258/0x2f0
Modules linked in:

CPU: 1 PID: 2323 Comm: drm-api-test Tainted: G W 4.9.0-rc4-00906-g693af44 #213
Hardware name: Qualcomm Technologies, Inc. DB820c (DT)
task: ffff8000d7353400 task.stack: ffff8000d7720000
PC is at drm_mm_insert_node_generic+0x258/0x2f0
LR is at drm_vma_offset_add+0x4c/0x70

Zero sized buffers serve no appreciable value to the user so disallow
them at create time.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

mm, fs: reduce fault, page_mkwrite, and pfn_mkwrite to take only vmf

->fault(), ->page_mkwrite(), and ->pfn_mkwrite() calls do not need to
take a vma and vmf parameter when the vma already resides in vmf.

Remove the vma parameter to simplify things.

[arnd@arndb.de: fix ARM build]
Link: http://lkml.kernel.org/r/20170125223558.1451224-1-arnd@arndb.de
Link: http://lkml.kernel.org/r/148521301778.19116.10840599906674778980.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Darrick J. Wong <darrick.wong@oracle.com>
Cc: Matthew Wilcox <mawilcox@microsoft.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Jan Kara <jack@suse.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

drm: Improve drm_mm search (and fix topdown allocation) with rbtrees

The drm_mm range manager claimed to support top-down insertion, but it
was neither searching for the top-most hole that could fit the
allocation request nor fitting the request to the hole correctly.

In order to search the range efficiently, we create a secondary index
for the holes using either their size or their address. This index
allows us to find the smallest hole or the hole at the bottom or top of
the range efficiently, whilst keeping the hole stack to rapidly service
evictions.

v2: Search for holes both high and low. Rename flags to mode.
v3: Discover rb_entry_safe() and use it!
v4: Kerneldoc for enum drm_mm_insert_mode.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Russell King <rmk+kernel@armlinux.org.uk>
Cc: Daniel Vetter <daniel.vetter@intel.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Sean Paul <seanpaul@chromium.org>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Stephen Warren <swarren@wwwdotorg.org>
Cc: Alexandre Courbot <gnurou@gmail.com>
Cc: Eric Anholt <eric@anholt.net>
Cc: Sinclair Yeh <syeh@vmware.com>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Sinclair Yeh <syeh@vmware.com> # vmwgfx
Reviewed-by: Lucas Stach <l.stach@pengutronix.de> #etnaviv
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20170202210438.28702-1-chris@chris-wilson.co.uk

locking/atomic, kref: Add kref_read()

Since we need to change the implementation, stop exposing internals.

Provide kref_read() to read the current reference count; typically
used for debug messages.

Kills two anti-patterns:

atomic_read(&kref->refcount)
kref->refcount.counter

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>

drm/msm: fix potential null ptr issue in non-iommu case

Fixes: 9cb07b099fb ("drm/msm: support multiple address spaces")
Reported-by: Riku Voipio <riku.voipio@linaro.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

mm: use vmf->address instead of of vmf->virtual_address

Every single user of vmf->virtual_address typed that entry to unsigned
long before doing anything with it so the type of virtual_address does
not really provide us any additional safety. Just use masked
vmf->address which already has the appropriate type.

Link: http://lkml.kernel.org/r/1479460644-25076-3-git-send-email-jack@suse.cz
Signed-off-by: Jan Kara <jack@suse.cz>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

drm/msm: convert iova to 64b

For a5xx the gpu is 64b so we need to change iova to 64b everywhere. On
the display side, iova is still 32b so it can ignore the upper bits.
(Although all the armv8 devices have an iommu that can map 64b pa to 32b
iova.)

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: support multiple address spaces

We can have various combinations of 64b and 32b address space, ie. 64b
CPU but 32b display and gpu, or 64b CPU and GPU but 32b display. So
best to decouple the device iova's from mmap offset.

Signed-off-by: Rob Clark <robdclark@gmail.com>

dma-buf: Rename struct fence to dma_fence

I plan to usurp the short name of struct fence for a core kernel struct,
and so I need to rename the specialised fence/timeline for DMA
operations to make room.

A consensus was reached in
https://lists.freedesktop.org/archives/dri-devel/2016-July/113083.html
that making clear this fence applies to DMA operations was a good thing.
Since then the patch has grown a bit as usage increases, so hopefully it
remains a good thing!

(v2...: rebase, rerun spatch)
v3: Compile on msm, spotted a manual fixup that I broke.
v4: Try again for msm, sorry Daniel

coccinelle script:
@@

@@
- struct fence
+ struct dma_fence
@@

@@
- struct fence_ops
+ struct dma_fence_ops
@@

@@
- struct fence_cb
+ struct dma_fence_cb
@@

@@
- struct fence_array
+ struct dma_fence_array
@@

@@
- enum fence_flag_bits
+ enum dma_fence_flag_bits
@@

@@
(
- fence_init
+ dma_fence_init
|
- fence_release
+ dma_fence_release
|
- fence_free
+ dma_fence_free
|
- fence_get
+ dma_fence_get
|
- fence_get_rcu
+ dma_fence_get_rcu
|
- fence_put
+ dma_fence_put
|
- fence_signal
+ dma_fence_signal
|
- fence_signal_locked
+ dma_fence_signal_locked
|
- fence_default_wait
+ dma_fence_default_wait
|
- fence_add_callback
+ dma_fence_add_callback
|
- fence_remove_callback
+ dma_fence_remove_callback
|
- fence_enable_sw_signaling
+ dma_fence_enable_sw_signaling
|
- fence_is_signaled_locked
+ dma_fence_is_signaled_locked
|
- fence_is_signaled
+ dma_fence_is_signaled
|
- fence_is_later
+ dma_fence_is_later
|
- fence_later
+ dma_fence_later
|
- fence_wait_timeout
+ dma_fence_wait_timeout
|
- fence_wait_any_timeout
+ dma_fence_wait_any_timeout
|
- fence_wait
+ dma_fence_wait
|
- fence_context_alloc
+ dma_fence_context_alloc
|
- fence_array_create
+ dma_fence_array_create
|
- to_fence_array
+ to_dma_fence_array
|
- fence_is_array
+ dma_fence_is_array
|
- trace_fence_emit
+ trace_dma_fence_emit
|
- FENCE_TRACE
+ DMA_FENCE_TRACE
|
- FENCE_WARN
+ DMA_FENCE_WARN
|
- FENCE_ERR
+ DMA_FENCE_ERR
)
(
...
)

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Gustavo Padovan <gustavo.padovan@collabora.co.uk>
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20161025120045.28839-1-chris@chris-wilson.co.uk

drm/msm: Remove call to reservation_object_test_signaled_rcu before wait

Since fence_wait_timeout_reservation_object_wait_timeout_rcu() with a
timeout of 0 becomes reservation_object_test_signaled_rcu(), we do not
need to handle such conversion in the caller. The only challenge are
those callers that wish to differentiate the error code between the
nonblocking busy check and potentially blocking wait.

v2: 9 is only 0 in German.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Rob Clark <robdclark@gmail.com>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: protect against faults from copy_from_user() in submit ioctl

An evil userspace could try to cause deadlock by passing an unfaulted-in
GEM bo as submit->bos (or submit->cmds) table. Which will trigger
msm_gem_fault() while we already hold struct_mutex. See:

https://github.com/freedreno/msmtest/blob/master/evilsubmittest.c

Cc: stable@vger.kernel.org
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Delete an unnecessary check before drm_gem_object_unreference()

The drm_gem_object_unreference() function tests whether its argument
is NULL and then returns immediately.
Thus the test around the call is not needed.

This issue was detected by using the Coccinelle software.

Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Delete unnecessary checks before drm_gem_object_unreference_unlocked()

The drm_gem_object_unreference_unlocked() function tests whether
its argument is NULL and then returns immediately.
Thus the test around the calls is not needed.

This issue was detected by using the Coccinelle software.

Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: wire up vmap shrinker

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: change gem->vmap() to get/put

Before we can add vmap shrinking, we really need to know which vmap'ings
are currently being used. So switch to get/put interface. Stubbed put
fxns for now.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: shrinker support

For a first step, only purge obj->madv==DONTNEED objects. We could be
more agressive and next try unpinning inactive objects.. but that is
only useful if you have swap.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: add put_iova() helper

We'll need this too for shrinker/purging.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: add madvise ioctl

Doesn't do anything too interesting until we wire up shrinker. Pretty
much lifted from i915.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: deal with exhausted vmap space better

Some, but not all, callers of obj->vmap() would check if return
IS_ERR(). So let's actually return an error if vmap() fails. And fixup
the call-sites that were not handling this properly.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm: Remove unused drm_device from drm_gem_object_lookup()

drm_gem_object_lookup() has never required the drm_device for its file
local translation of the user handle to the GEM object. Let's remove the
unused parameter and save some space.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: dri-devel@lists.freedesktop.org
Cc: Dave Airlie <airlied@redhat.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
[danvet: Fixup kerneldoc too.]
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

drm/msm: 'struct fence' conversion

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: remove fence_cbs

This was only used for atomic commit these days. So instead just give
atomic it's own work-queue where we can do a block on each bo in turn.
Simplifies things a whole bunch and makes the 'struct fence' conversion
easier.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: introduce msm_fence_context

Better encapsulate the per-timeline stuff into fence-context. For now
there is just a single fence-context, but eventually we'll also have one
per-CRTC to enable fully explicit fencing.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: use imported dmabuf's reservation object

This was always the intention, but somehow it was never wired up
properly.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: move fence code to it's own file

Signed-off-by: Rob Clark <robdclark@gmail.com>

mm, dax, gpu: convert vm_insert_mixed to pfn_t

Convert the raw unsigned long 'pfn' argument to pfn_t for the purpose of
evaluating the PFN_MAP and PFN_DEV flags. When both are set it triggers
_PAGE_DEVMAP to be set in the resulting pte.

There are no functional changes to the gpu drivers as a result of this
conversion.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Cc: Dave Hansen <dave@sr71.net>
Cc: David Airlie <airlied@linux.ie>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

drm/msm: change to uninterruptible wait in atomic commit

The atomic commit cannot easily undo and return an error once the
state is swapped. Change to uninterruptible wait, and ignore the
timeout error.

Signed-off-by: Wentao Xu <wentaox@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: change to uninterruptible wait in atomic commit

The atomic commit cannot easily undo and return an error once the
state is swapped. Change to uninterruptible wait, and ignore the
timeout error.

Signed-off-by: Wentao Xu <wentaox@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: fix timeout calculation

The 'timeout' value comes from userspace (CLOCK_MONOTONIC), but
converting this directly to jiffies doesn't take into account the
initial jiffies count at boot, which may differ from the base time
of CLOCK_MONOTONIC.

TODO: add ktime_delta_jiffies() when rebasing on 4.1 and use that
instead of ktime_sub/ktime_to_timespec/timespec_to_jiffies combo (as
suggested by Arnd)

v2: switch over from 'struct timespec' to ktime_t throughout, since
'struct timespec' will be deprecated (as suggested by Arnd)
v3: minor cosmetic tweaks

Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Call drm_prime_gem_destroy to clean up imported GEM object

If the GEM object is imported, drm_prime_gem_destroy needs to be
called to clean up dma buffer related information.

Signed-off-by: Jilai Wang <jilaiw@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Fix a couple of 64-bit build warnings

Avoid casts from pointers to fixed-size integers to prevent the compiler
from warning. Print virtual memory addresses using %p instead. Also turn
a couple of %d/%x specifiers into %zu/%zd/%zx to avoid further warnings
due to mismatched format strings.

Signed-off-by: Thierry Reding <treding@nvidia.com>
Reviewed-by: Rob Clark <robdclark@gmail.com>

drm/msm: add support for "stolen" mem

Add support to use the VRAM carveout (if specified in dtb) for fbdev
scanout buffer. This allows drm/msm to take over a bootloader splash-
screen, and avoids corruption on screen that results if the kernel uses
memory that is still being scanned out for itself.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Deletion of unnecessary checks before two function calls

The functions framebuffer_release() and vunmap() perform also input
parameter validation. Thus the test around the call is not needed.

This issue was detected by using the Coccinelle software.

Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
Reviewed-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: small mmap offset cleanups

Use pre-computed iova when unmapping, to reduce the places we assume iova
and mmap offset are (at the moment) the same. And get rid of an extra
drm_gem_free_mmap_offset() call (since it is already called from
drm_gem_object_release())

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: fb prepare/cleanup

Atomic wants to split the prepare/pin from where we actually program the
scanout address (so that any part that can fail is done synchronously).
Add some fb/gem apis to make this easier to use from the kms parts.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: small fence cleanup

Give ourselves a way to wait for certain fence #.. makes it easier to
wait on a set of bo's, which we'll need for atomic.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: no mmu is only error if not using vram carveout

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/gem: remove misleading gfp parameter to get_pages()

drm_gem_get_pages() currently allows passing a 'gfp' parameter that is
passed to shmem combined with mapping_gfp_mask(). Given that the default
mapping_gfp_mask() is GFP_HIGHUSER, it is _very_ unlikely that anyone will
ever make use of that parameter. In fact, all drivers currently pass
redundant flags or 0.

This patch removes the 'gfp' parameter. The only reason to keep it is to
remove flags like __GFP_WAIT. But in its current form, it can only be used
to add flags. So to remove __GFP_WAIT, you'd have to drop it from the
mapping_gfp_mask, which again is stupid as this mask is used by shmem-core
for other allocations, too.

If any driver ever requires that parameter, we can introduce a new helper
that takes the raw 'gfp' parameter. The caller'd be responsible to combine
it with mapping_gfp_mask() in a suitable way. The current
drm_gem_get_pages() helper would then simply use mapping_gfp_mask() and
call the new helper. This is what shmem_read_mapping_pages{_gfp,} does
right now.

Moreover, the gfp-zone flag-usage is not obvious: If you pass a modified
zone, shmem core will WARN() or even BUG(). In other words, the following
must be true for 'gfp' passed to shmem_read_mapping_pages_gfp():
gfp_zone(mapping_gfp_mask(mapping)) == gfp_zone(gfp)
Add a comment to drm_gem_read_pages() explaining that constraint.

Signed-off-by: David Herrmann <dh.herrmann@gmail.com>

drm/msm: fix IOMMU cleanup for -EPROBE_DEFER

If probe fails after IOMMU is attached, we need to detach in order to
clean up properly. Before this change, IOMMU faults would occur if the
probe failed (-EPROBE_DEFER).

Signed-off-by: Stephane Viau <sviau@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: fix memory leak

Signed-off-by: Micah Richert <richert@braincorporation.com>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: fix deadlock in bo create fail path

We already hold struct_mutex here.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: add support for non-IOMMU systems

Add a VRAM carveout that is used for systems which do not have an IOMMU.

The VRAM carveout uses CMA. The arch code must setup a CMA pool for the
device (preferrably in highmem.. a 256m-512m VRAM pool in lowmem is not
cool). The user can configure the VRAM pool size using msm.vram module
param.

Technically, the abstraction of IOMMU behind msm_mmu is not strictly
needed, but it simplifies the GEM code a bit, and will be useful later
when I add support for a2xx devices with GPUMMU, so I decided to keep
this part.

It appears to be possible to configure the GPU to restrict access to
addresses within the VRAM pool, but this is not done yet. So for now
the GPU will refuse to load if there is no sort of mmu. Once address
based limits are supported and tested to confirm that we aren't giving
the GPU access to arbitrary memory, this restriction can be lifted

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: EBUSY status handling in msm_gem_fault()

Subsequent threads returning EBUSY from vm_insert_pfn() was not
handled correctly. As a result concurrent access from new threads
to mmapped data caused SIGBUS.

See e79e0fe3

Signed-off-by: Rob Clark <robdclark@gmail.com>
Acked-by: David Brown <davidb@codeaurora.org>

drm/msm: rework inactive-work

Re-arrange things a bit so that we can get work requested after a bo
fence passes, like pageflip, done before retiring bo's. Without any
sort of bo cache in userspace, some games can trigger hundred's of
transient bo's, which can cause retire to take a long time (5-10ms).
Obviously we want a bo cache.. but this cleanup will make things a
bit easier for atomic as well and makes things a bit cleaner.

Signed-off-by: Rob Clark <robdclark@gmail.com>
Acked-by: David Brown <davidb@codeaurora.org>

drm/msm: prime support

Signed-off-by: Rob Clark <robdclark@gmail.com>
Acked-by: David Brown <davidb@codeaurora.org>

drm/msm: use drm_gem_dumb_destroy helper

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: drop unnecessary set_need_resched()

This was inherited from i915/udl, and not actually needed.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: return -EBUSY if bo still active

When we CPU_PREP a bo with NOSYNC flag (for example, to implement
PIPE_TRANSFER_DISCARD_WHOLE_RESOURCE), an -EBUSY return indicates to
userspace that the bo is still busy. Previously it was incorrectly
returning 0 in this case.

And while we're in there throw in an bit of extra sanity checking in
case userspace tries to wait for a bogus fence.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: fix return value check in ERR_PTR()

In case of error, the function drm_prime_pages_to_sg() returns ERR_PTR()
and never returns NULL. The NULL test in the return value check should
be replaced with IS_ERR().

Signed-off-by: Wei Yongjun <yongjun_wei@trendmicro.com.cn>

drm/msm: handle read vs write fences

The userspace API already had everything needed to handle read vs write
synchronization. This patch actually bothers to hook it up properly, so
that we don't need to (for example) stall on userspace read access to a
buffer that gpu is also still reading.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: add a3xx gpu support

Add initial support for a3xx 3d core.

So far, with hardware that I've seen to date, we can have:
+ zero, one, or two z180 2d cores
+ a3xx or a2xx 3d core, which share a common CP (the firmware
for the CP seems to implement some different PM4 packet types
but the basics of cmdstream submission are the same)

Which means that the eventual complete "class" hierarchy, once
support for all past and present hw is in place, becomes:
+ msm_gpu
+ adreno_gpu
+ a3xx_gpu
+ a2xx_gpu
+ z180_gpu

This commit splits out the parts that will eventually be common
between a2xx/a3xx into adreno_gpu, and the parts that are even
common to z180 into msm_gpu.

Note that there is no cmdstream validation required. All memory access
from the GPU is via IOMMU/MMU. So as long as you don't map silly things
to the GPU, there isn't much damage that the GPU can do.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: basic KMS driver for snapdragon

The snapdragon chips have multiple different display controllers,
depending on which chip variant/version. (As far as I can tell, current
devices have either MDP3 or MDP4, and upcoming devices have MDSS.) And
then external to the display controller are HDMI, DSI, etc. blocks which
may be shared across devices which have different display controller
blocks.

To more easily add support for different display controller blocks, the
display controller specific bits are split out into a "kms" module,
which provides the kms plane/crtc/encoder objects.

The external HDMI, DSI, etc. blocks are part encoder, and part connector
currently. But I think I will pull in the drm_bridge patches from
chromeos tree, and split them into a bridge+connector, with the
registers that need to be set in modeset handled by the bridge. This
would remove the 'msm_connector' base class. But some things need to be
double checked to make sure I could get the correct ON/OFF sequencing..

This patch adds support for mdp4 crtc (including hw cursor), dtv encoder
(part of MDP4 block), and hdmi.

Signed-off-by: Rob Clark <robdclark@gmail.com>