drm/nouveau: fix stale locked mutex in nouveau_gem_ioctl_pushbuf

If VM_BIND is enabled on the client the legacy submission ioctl can't be
used, however if a client tries to do so regardless it will return an
error. In this case the clients mutex remained unlocked leading to a
deadlock inside nouveau_drm_postclose or any other nouveau ioctl call.

Fixes: b88baab82871 ("drm/nouveau: implement new VM_BIND uAPI")
Cc: Danilo Krummrich <dakr@redhat.com>
Cc: <stable@vger.kernel.org> # v6.6+
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Reviewed-by: Danilo Krummrich <dakr@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20240305133853.2214268-1-kherbst@redhat.com

drm/nouveau: make use of the GPUVM's shared dma-resv

DRM GEM objects private to a single GPUVM can use a shared dma-resv.
Make use of the shared dma-resv of GPUVM rather than a driver specific
one.

The shared dma-resv originates from a "root" GEM object serving as
container for the dma-resv to make it compatible with drm_exec.

In order to make sure the object proving the shared dma-resv can't be
freed up before the objects making use of it, let every such GEM object
take a reference on it.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231108001259.15123-7-dakr@redhat.com

drm/nouveau: uapi: don't pass NO_PREFETCH flag implicitly

Currently, NO_PREFETCH is passed implicitly through
drm_nouveau_gem_pushbuf_push::length and drm_nouveau_exec_push::va_len.

Since this is a direct representation of how the HW is programmed it
isn't really future proof for a uAPI. Hence, fix this up for the new
uAPI and split up the va_len field of struct drm_nouveau_exec_push,
such that we keep 32bit for va_len and 32bit for flags.

For drm_nouveau_gem_pushbuf_push::length at least provide
NOUVEAU_GEM_PUSHBUF_NO_PREFETCH to indicate the bit shift.

While at it, fix up nv50_dma_push() as well, such that the caller
doesn't need to encode the NO_PREFETCH flag into the length parameter.

Signed-off-by: Danilo Krummrich <dakr@redhat.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230823181746.3446-1-dakr@redhat.com

drm/nouveau: fence: fix undefined fence state after emit

nouveau_fence_emit() can fail before and after initializing the
dma-fence and hence before and after initializing the dma-fence' kref.

In order to avoid nouveau_fence_emit() potentially failing before
dma-fence initialization pass the channel to nouveau_fence_new() already
and perform the required check before even allocating the fence.

While at it, restore the original behavior of nouveau_fence_new() and
add nouveau_fence_create() for separate (pre-)allocation instead. Always
splitting up allocation end emit wasn't a good idea in the first place.
Hence, limit it to the places where we actually need to pre-allocate.

Fixes: 7f2a0b50b2b2 ("drm/nouveau: fence: separate fence alloc and emit")
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230829223847.4406-1-dakr@redhat.com

drm/nouveau: uapi: don't pass NO_PREFETCH flag implicitly

Currently, NO_PREFETCH is passed implicitly through
drm_nouveau_gem_pushbuf_push::length and drm_nouveau_exec_push::va_len.

Since this is a direct representation of how the HW is programmed it
isn't really future proof for a uAPI. Hence, fix this up for the new
uAPI and split up the va_len field of struct drm_nouveau_exec_push,
such that we keep 32bit for va_len and 32bit for flags.

For drm_nouveau_gem_pushbuf_push::length at least provide
NOUVEAU_GEM_PUSHBUF_NO_PREFETCH to indicate the bit shift.

While at it, fix up nv50_dma_push() as well, such that the caller
doesn't need to encode the NO_PREFETCH flag into the length parameter.

Signed-off-by: Danilo Krummrich <dakr@redhat.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230823181746.3446-1-dakr@redhat.com

drm/nouveau: implement new VM_BIND uAPI

This commit provides the implementation for the new uapi motivated by the
Vulkan API. It allows user mode drivers (UMDs) to:

1) Initialize a GPU virtual address (VA) space via the new
DRM_IOCTL_NOUVEAU_VM_INIT ioctl for UMDs to specify the portion of VA
space managed by the kernel and userspace, respectively.

2) Allocate and free a VA space region as well as bind and unbind memory
to the GPUs VA space via the new DRM_IOCTL_NOUVEAU_VM_BIND ioctl.
UMDs can request the named operations to be processed either
synchronously or asynchronously. It supports DRM syncobjs
(incl. timelines) as synchronization mechanism. The management of the
GPU VA mappings is implemented with the DRM GPU VA manager.

3) Execute push buffers with the new DRM_IOCTL_NOUVEAU_EXEC ioctl. The
execution happens asynchronously. It supports DRM syncobj (incl.
timelines) as synchronization mechanism. DRM GEM object locking is
handled with drm_exec.

Both, DRM_IOCTL_NOUVEAU_VM_BIND and DRM_IOCTL_NOUVEAU_EXEC, use the DRM
GPU scheduler for the asynchronous paths.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230804182406.5222-12-dakr@redhat.com

drm/nouveau: fence: separate fence alloc and emit

The new (VM_BIND) UAPI exports DMA fences through DRM syncobjs. Hence,
in order to emit fences within DMA fence signalling critical sections
(e.g. as typically done in the DRM GPU schedulers run_job() callback) we
need to separate fence allocation and fence emitting.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230804182406.5222-8-dakr@redhat.com

drm/nouveau: move usercopy helpers to nouveau_drv.h

Move the usercopy helpers to a common driver header file to make it
usable for the new API added in subsequent commits.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230804182406.5222-7-dakr@redhat.com

drm/nouveau: get vmm via nouveau_cli_vmm()

Provide a getter function for the client's current vmm context. Since
we'll add a new (u)vmm context for UMD bindings in subsequent commits,
this will keep the code clean.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230804182406.5222-5-dakr@redhat.com

drm/nouveau: fix incorrect conversion to dma_resv_wait_timeout()

Commit 41d351f29528 ("drm/nouveau: stop using ttm_bo_wait")
converted from ttm_bo_wait_ctx() to dma_resv_wait_timeout().
However, dma_resv_wait_timeout() returns greater than zero on
success as opposed to ttm_bo_wait_ctx(). As a result, relocs
will fail and log errors even when it was a success.

Change the return code handling to match that of
nouveau_gem_ioctl_cpu_prep(), which was already using
dma_resv_wait_timeout() correctly.

Fixes: 41d351f29528 ("drm/nouveau: stop using ttm_bo_wait")
Reported-by: Tanmay Bhushan <007047221b@gmail.com>
Link: https://lore.kernel.org/lkml/20230119225351.71657-1-007047221b@gmail.com
Signed-off-by: John Ogness <john.ogness@linutronix.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/87edolaomt.fsf@jogness.linutronix.de

drm/nouveau: stop using ttm_bo_wait

TTM is just wrapping core DMA functionality here, remove the mid-layer.
No functional change.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221125102137.1801-5-christian.koenig@amd.com

drm/ttm: rework on ttm_resource to use size_t type

Change ttm_resource structure from num_pages to size_t size in bytes.
v1 -> v2: change PFN_UP(dst_mem->size) to ttm->num_pages
v1 -> v2: change bo->resource->size to bo->base.size at some places
v1 -> v2: remove the local variable
v1 -> v2: cleanup cmp_size_smaller_first()
v2 -> v3: adding missing PFN_UP in ttm_bo_vm_fault_reserved

Signed-off-by: Somalapuram Amaranath <Amaranath.Somalapuram@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221027091237.983582-1-Amaranath.Somalapuram@amd.com
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>

dma-buf: add enum dma_resv_usage v4

This change adds the dma_resv_usage enum and allows us to specify why a
dma_resv object is queried for its containing fences.

Additional to that a dma_resv_usage_rw() helper function is added to aid
retrieving the fences for a read or write userspace submission.

This is then deployed to the different query functions of the dma_resv
object and all of their users. When the write paratermer was previously
true we now use DMA_RESV_USAGE_WRITE and DMA_RESV_USAGE_READ otherwise.

v2: add KERNEL/OTHER in separate patch
v3: some kerneldoc suggestions by Daniel
v4: some more kerneldoc suggestions by Daniel, fix missing cases lost in
the rebase pointed out by Bas.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20220407085946.744568-2-christian.koenig@amd.com

drm/ttm: remove ttm_bo_vm_insert_huge()

The huge page functionality in TTM does not work safely because PUD and
PMD entries do not have a special bit.

get_user_pages_fast() considers any page that passed pmd_huge() as
usable:

if (unlikely(pmd_trans_huge(pmd) || pmd_huge(pmd) ||
pmd_devmap(pmd))) {

And vmf_insert_pfn_pmd_prot() unconditionally sets

entry = pmd_mkhuge(pfn_t_pmd(pfn, prot));

eg on x86 the page will be _PAGE_PRESENT | PAGE_PSE.

As such gup_huge_pmd() will try to deref a struct page:

head = try_grab_compound_head(pmd_page(orig), refs, flags);

and thus crash.

Thomas further notices that the drivers are not expecting the struct page
to be used by anything - in particular the refcount incr above will cause
them to malfunction.

Thus everything about this is not able to fully work correctly considering
GUP_fast. Delete it entirely. It can return someday along with a proper
PMD/PUD_SPECIAL bit in the page table itself to gate GUP_fast.

Fixes: 314b6580adc5 ("drm/ttm, drm/vmwgfx: Support huge TTM pagefaults")
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Thomas Hellström <thomas.helllstrom@linux.intel.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
[danvet: Update subject per Thomas' &Christian's review]
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/0-v2-a44694790652+4ac-ttm_pmd_jgg@nvidia.com

drm/nouveau/gem: remove redundant semi-colon

Signed-off-by: Yang Yingliang <yangyingliang@huawei.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210401141731.1684236-1-yangyingliang@huawei.com

drm/nouveau: avoid a use-after-free when BO init fails

nouveau_bo_init() is backed by ttm_bo_init() and ferries its return code
back to the caller. On failures, ttm_bo_init() invokes the provided
destructor which should de-initialize and free the memory.

Thus, when nouveau_bo_init() returns an error the gem object has already
been released and the memory freed by nouveau_bo_del_ttm().

Fixes: 019cbd4a4feb ("drm/nouveau: Initialize GEM object before TTM object")
Cc: Thierry Reding <treding@nvidia.com>
Signed-off-by: Jeremy Cline <jcline@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201203000220.18238-1-jcline@redhat.com
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>

dma-buf: drop the _rcu postfix on function names v3

The functions can be called both in _rcu context as well
as while holding the lock.

v2: add some kerneldoc as suggested by Daniel
v3: fix indentation

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210602111714.212426-7-christian.koenig@amd.com

drm/ttm: rename bo->mem and make it a pointer

When we want to decouble resource management from buffer management we need to
be able to handle resources separately.

Add a resource pointer and rename bo->mem so that all code needs to
change to access the pointer instead.

No functional change.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210430092508.60710-4-christian.koenig@amd.com

drm/nouveau: Implement mmap as GEM object function

Moving the driver-specific mmap code into a GEM object function allows
for using DRM helpers for various mmap callbacks.

The GEM object function is provided by GEM TTM helpers. Nouveau's
implementation of verify_access is unused and has been removed. Access
permissions are validated by the DRM helpers.

As a side effect, nouveau_ttm_vm_ops and nouveau_ttm_fault() are now
implemented in nouveau's GEM code.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210525151055.8174-5-tzimmermann@suse.de

drm/nouveau: use bo->base.size instead of mem->num_pages

Change a couple of cases where it makes more sense to use the base size
instead of the number of pages in the resource.

Signed-off-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210413135248.1266-1-christian.koenig@amd.com
Reviewed-by: Matthew Auld <matthew.auld@intel.com>

drm/nouveau: fix relocations applying logic and a double-free

Commit 03e0d26fcf79 ("drm/nouveau: slowpath for pushbuf ioctl") included
a logic-bug which results in the relocations not actually getting
applied at all as the call to nouveau_gem_pushbuf_reloc_apply() is
never reached. This causes a regression with graphical corruption,
triggered when relocations need to be done (for example after a
suspend/resume cycle.)

Fix by setting *apply_relocs value only if there were more than 0
relocations.

Additionally, the never reached code had a leftover u_free() call,
which, after fixing the logic, now got called and resulted in a
double-free. Fix by removing one u_free(), moving the other
and adding check for errors.

Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: nouveau@lists.freedesktop.org
Cc: dri-devel@lists.freedesktop.org
Signed-off-by: Matti Hamalainen <ccr@tnsp.org>
Fixes: 03e0d26fcf79 ("drm/nouveau: slowpath for pushbuf ioctl")
References: https://gitlab.freedesktop.org/drm/nouveau/-/issues/11
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20201120152338.1203257-1-ccr@tnsp.org

drm/gem: Use struct dma_buf_map in GEM vmap ops and convert GEM backends

This patch replaces the vmap/vunmap's use of raw pointers in GEM object
functions with instances of struct dma_buf_map. GEM backends are
converted as well. For most of them, this simply changes the returned type.

TTM-based drivers now return information about the location of the memory,
either system or I/O memory. GEM VRAM helpers and qxl now use ttm_bo_vmap()
et al. Amdgpu, nouveau and radeon use drm_gem_ttm_vmap() et al instead of
implementing their own vmap callbacks.

v7:
* init QXL cursor to mapped BO buffer (kernel test robot)
v5:
* update vkms after switch to shmem
v4:
* use ttm_bo_vmap(), drm_gem_ttm_vmap(), et al. (Daniel, Christian)
* fix a trailing { in drm_gem_vmap()
* remove several empty functions instead of converting them (Daniel)
* comment uses of raw pointers with a TODO (Daniel)
* TODO list: convert more helpers to use struct dma_buf_map

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Acked-by: Christian König <christian.koenig@amd.com>
Tested-by: Sam Ravnborg <sam@ravnborg.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20201103093015.1063-7-tzimmermann@suse.de

drm/nouveau/gem: fix "refcount_t: underflow; use-after-free"

we can't use nouveau_bo_ref here as no ttm object was allocated and
nouveau_bo_ref mainly deals with that. Simply deallocate the object.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Introduce GEM object functions

GEM object functions deprecate several similar callback interfaces in
struct drm_driver. This patch replaces the per-driver callbacks with
per-instance callbacks in nouveau.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200923102159.24084-11-tzimmermann@suse.de

drm/nouveau: stop using persistent_swap_storage

According to Ben this is most likely just a leftover.

Signed-off-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/391078/?series=81804&rev=1
Reviewed-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: stop using TTM placement flags

Those are going to be removed, stop using them here.

Instead use the GEM flags from the UAPI.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/389825/?series=81551&rev=1

drm/nouveau/gem: convert indirect pushbufs to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau: fix multiple instances of reference count leaks

On calling pm_runtime_get_sync() the reference count of the device
is incremented. In case of failure, decrement the
ref count before returning the error.

Signed-off-by: Aditya Pakki <pakki001@umn.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: don't use ttm bo->offset v3

Store ttm bo->offset in struct nouveau_bo instead.

Signed-off-by: Nirmoy Das <nirmoy.das@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/372932/
Signed-off-by: Christian König <christian.koenig@amd.com>

drm/nouveau: fix runtime pm imbalance on error

pm_runtime_get_sync() increments the runtime PM usage counter even
the call returns an error code. Thus a pairing decrement is needed
on the error handling path to keep the counter balanced.

Signed-off-by: Dinghao Liu <dinghao.liu@zju.edu.cn>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fix runtime pm imbalance on error

pm_runtime_get_sync() increments the runtime PM usage counter even
the call returns an error code. Thus a pairing decrement is needed
on the error handling path to keep the counter balanced.

Signed-off-by: Dinghao Liu <dinghao.liu@zju.edu.cn>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: remove _unlocked suffix in drm_gem_object_put_unlocked

Spelling out _unlocked for each and every driver is a annoying.
Especially if we consider how many drivers, do not know (or need to)
about the horror stories involving struct_mutex.

Just drop the suffix. It makes the API cleaner.

Done via the following script:

__from=drm_gem_object_put_unlocked
__to=drm_gem_object_put
for __file in $(git grep --name-only $__from); do
sed -i "s/$__from/$__to/g" $__file;
done

Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Signed-off-by: Emil Velikov <emil.velikov@collabora.com>
Acked-by: Sam Ravnborg <sam@ravnborg.org>
Acked-by: Thomas Zimmermann <tzimmermann@suse.de>
Link: https://patchwork.freedesktop.org/patch/msgid/20200515095118.2743122-26-emil.l.velikov@gmail.com

drm/nouveau: support synchronous pushbuf submission

This is useful for debugging GPU hangs.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: reject attempts to submit to dead channels

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: slowpath for pushbuf ioctl

We can't copy_*_user while holding reservations, that will (soon even
for nouveau) lead to deadlocks. And it breaks the cross-driver
contract around dma_resv.

Fix this by adding a slowpath for when we need relocations, and by
pushing the writeback of the new presumed offsets to the very end.

Aside from "it compiles" entirely untested unfortunately.

Cc: Ilia Mirkin <imirkin@alum.mit.edu>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: nouveau@lists.freedesktop.org
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Acked-by: Dave Airlie <airlied@gmail.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20191104173801.2972-2-daniel.vetter@ffwll.ch

drm/nouveau: Fix ordering between TTM and GEM release

When the last reference to a TTM BO is dropped, ttm_bo_release() will
acquire the DMA reservation object's wound/wait mutex while trying to
clean up (ttm_bo_cleanup_refs_or_queue() via ttm_bo_release()). It is
therefore essential that drm_gem_object_release() be called after the
TTM BO has been uninitialized, otherwise drm_gem_object_release() has
already destroyed the wound/wait mutex (via dma_resv_fini()).

Signed-off-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Fix fallout from reservation object rework

Commit 019cbd4a4feb ("drm/nouveau: Initialize GEM object before TTM
object") introduced a subtle change in how the buffer allocation size is
handled. Prior to that change, the size would get aligned to at least a
page, whereas after that change a non-page-aligned size would get passed
through unmodified. This ultimately causes a BUG_ON() to trigger in
drm_gem_private_object_init() and crashes the system.

Fix this by restoring the code that align the allocation size.

Fixes: 019cbd4a4feb ("drm/nouveau: Initialize GEM object before TTM object")
Reported-by: Ilia Mirkin <imirkin@alum.mit.edu>
Signed-off-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Initialize GEM object before TTM object

TTM assumes that drivers initialize the embedded GEM object before
calling the ttm_bo_init() function. This is not currently the case
in the Nouveau driver. Fix this by splitting up nouveau_bo_new()
into nouveau_bo_alloc() and nouveau_bo_init() so that the GEM can
be initialized before TTM BO initialization when necessary.

Fixes: b96f3e7c8069 ("drm/ttm: use gem vma_node")
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190814093524.GA31345@ulmo

dma-buf: rename reservation_object to dma_resv

Be more consistent with the naming of the other DMA-buf objects.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/323401/

drm/nouveau: switch driver from bo->resv to bo->base.resv

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20190805140119.7337-15-kraxel@redhat.com

drm/ttm: use gem vma_node

Drop vma_node from ttm_buffer_object, use the gem struct
(base.vma_node) instead.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20190805140119.7337-9-kraxel@redhat.com

drm/nouveau: use embedded gem object

Drop drm_gem_object from nouveau_bo, use the
ttm_buffer_object.base instead.

Build tested only.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Christian König <christian.koenig@amd.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20190805140119.7337-7-kraxel@redhat.com

drm/nouveau: prepare for enabling svm with existing userspace interfaces

For a channel to make use of SVM features, it requires a different GPU MMU
configuration than we would normally use, which is not desirable to switch
to unless a client is actively going to use SVM.

In order to supporting SVM without more extensive changes to the userspace
interfaces, the SVM_INIT ioctl needs to replace the previous configuration
safely.

The only way we can currently do this safely, accounting for some unlikely
failure conditions, is to allocate the new VMM without destroying the last
one, and prioritising the SVM-enabled configuration in the code that cares.

This will get cleaned up again further down the track.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Replace ttm_bo_unref with ttm_bo_put

The function ttm_bo_put releases a reference to a TTM buffer object. The
function's name is more aligned to the Linux kernel convention of naming
ref-counting function _get and _put.

A call to ttm_bo_unref takes the address of the TTM BO object's pointer and
clears the pointer's value to NULL. This is not necessary in most cases and
sometimes even worked around by the calling code. A call to ttm_bo_put only
releases the reference without clearing the pointer.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/nouveau: Replace drm_gem_object_unreference_unlocked with put function

This patch unifies the naming of DRM functions for reference counting
of struct drm_gem_object. The resulting code is more aligned with the
rest of the Linux kernel interfaces.

Signed-off-by: Thomas Zimmermann <tdz@users.sourceforge.net>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gem: off by one bugs in nouveau_gem_pushbuf_reloc_apply()

The bo array has req->nr_buffers elements so the > should be >= so we
don't read beyond the end of the array.

Fixes: a1606a9596e5 ("drm/nouveau: new gem pushbuf interface, bump to 0.0.16")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gem: tie deferred unmapping of buffers to VMA fence completion

As VMAs are per-client, unlike buffers, this allows us to avoid referencing
foreign fences (those that belong to another client/driver) from the client
deferred work handler, and prevent some not-fun race conditions that can be
triggered when a fence stalls.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gem: attach fences to VMAs to track GPU usage

An upcoming patch will use these to fix issues related to the deferred
unmapping of GEM objects.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gem: lookup VMAs for buffers referenced by pushbuf ioctl

We previously only did this for push buffers, but an upcoming patch will
need to attach fences to all VMAs to resolve another issue.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/ttm: remove ttm_bo_unreserve_ticket

Just another alias for ttm_bo_unreserve.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-and-Tested-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/nouveau/mmu: remove old vmm frontend

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: switch over to new memory and vmm interfaces

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: queue delayed unmapping of VMAs on client workqueue

VMAs are about to not take references on the VMM they belong to, which
means more care is required when handling delayed unmapping.

Queuing it on the client workqueue ensures all pending VMA unmaps will
have completed before the VMM is destroyed.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: allocate vmm object for every client

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: separate constant-va tracking from nvkm vma structure

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: split various bo flags out into their own members

It's far more convenient to deal with like this.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: store nouveau_drm in nouveau_cli, as opposed to drm_device

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

treewide: use kv[mz]alloc* rather than opencoded variants

There are many code paths opencoding kvmalloc. Let's use the helper
instead. The main difference to kvmalloc is that those users are
usually not considering all the aspects of the memory allocator. E.g.
allocation requests <= 32kB (with 4kB pages) are basically never failing
and invoke OOM killer to satisfy the allocation. This sounds too
disruptive for something that has a reasonable fallback - the vmalloc.
On the other hand those requests might fallback to vmalloc even when the
memory allocator would succeed after several more reclaim/compaction
attempts previously. There is no guarantee something like that happens
though.

This patch converts many of those places to kv[mz]alloc* helpers because
they are more conservative.

Link: http://lkml.kernel.org/r/20170306103327.2766-2-mhocko@kernel.org
Signed-off-by: Michal Hocko <mhocko@suse.com>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> # Xen bits
Acked-by: Kees Cook <keescook@chromium.org>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Andreas Dilger <andreas.dilger@intel.com> # Lustre
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com> # KVM/s390
Acked-by: Dan Williams <dan.j.williams@intel.com> # nvdim
Acked-by: David Sterba <dsterba@suse.com> # btrfs
Acked-by: Ilya Dryomov <idryomov@gmail.com> # Ceph
Acked-by: Tariq Toukan <tariqt@mellanox.com> # mlx4
Acked-by: Leon Romanovsky <leonro@mellanox.com> # mlx5
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Anton Vorontsov <anton@enomsg.org>
Cc: Colin Cross <ccross@android.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: Kent Overstreet <kent.overstreet@gmail.com>
Cc: Santosh Raspatur <santosh@chelsio.com>
Cc: Hariprasad S <hariprasad@chelsio.com>
Cc: Yishai Hadas <yishaih@mellanox.com>
Cc: Oleg Drokin <oleg.drokin@intel.com>
Cc: "Yan, Zheng" <zyan@redhat.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: David Miller <davem@davemloft.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

drm/nouveau: size is u64 everywhere

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: allocate device object for every client

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: pass nvif_client to nouveau_gem_new() instead of drm_device

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: pass nvif_client to nouveau_bo_new() instead of drm_device

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: hide gcc-4.9 -Wmaybe-uninitialized

gcc-4.9 notices that the validate_init() function returns unintialized
data when called with a zero 'nr_buffers' argument, when called with the
-Wmaybe-uninitialized flag:

drivers/gpu/drm/nouveau/nouveau_gem.c: In function ‘validate_init.isra.6’:
drivers/gpu/drm/nouveau/nouveau_gem.c:457:5: error: ‘ret’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

However, the only caller of this function always passes a nonzero
argument, and gcc-6 is clever enough to take this into account and
not warn about it any more.

Adding an explicit initialization to -EINVAL here is correct even if
the caller changed, and it avoids the warning on gcc-4.9 as well.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-By: Karol Herbst <karolherbst@gmail.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

dma-buf: Rename struct fence to dma_fence

I plan to usurp the short name of struct fence for a core kernel struct,
and so I need to rename the specialised fence/timeline for DMA
operations to make room.

A consensus was reached in
https://lists.freedesktop.org/archives/dri-devel/2016-July/113083.html
that making clear this fence applies to DMA operations was a good thing.
Since then the patch has grown a bit as usage increases, so hopefully it
remains a good thing!

(v2...: rebase, rerun spatch)
v3: Compile on msm, spotted a manual fixup that I broke.
v4: Try again for msm, sorry Daniel

coccinelle script:
@@

@@
- struct fence
+ struct dma_fence
@@

@@
- struct fence_ops
+ struct dma_fence_ops
@@

@@
- struct fence_cb
+ struct dma_fence_cb
@@

@@
- struct fence_array
+ struct dma_fence_array
@@

@@
- enum fence_flag_bits
+ enum dma_fence_flag_bits
@@

@@
(
- fence_init
+ dma_fence_init
|
- fence_release
+ dma_fence_release
|
- fence_free
+ dma_fence_free
|
- fence_get
+ dma_fence_get
|
- fence_get_rcu
+ dma_fence_get_rcu
|
- fence_put
+ dma_fence_put
|
- fence_signal
+ dma_fence_signal
|
- fence_signal_locked
+ dma_fence_signal_locked
|
- fence_default_wait
+ dma_fence_default_wait
|
- fence_add_callback
+ dma_fence_add_callback
|
- fence_remove_callback
+ dma_fence_remove_callback
|
- fence_enable_sw_signaling
+ dma_fence_enable_sw_signaling
|
- fence_is_signaled_locked
+ dma_fence_is_signaled_locked
|
- fence_is_signaled
+ dma_fence_is_signaled
|
- fence_is_later
+ dma_fence_is_later
|
- fence_later
+ dma_fence_later
|
- fence_wait_timeout
+ dma_fence_wait_timeout
|
- fence_wait_any_timeout
+ dma_fence_wait_any_timeout
|
- fence_wait
+ dma_fence_wait
|
- fence_context_alloc
+ dma_fence_context_alloc
|
- fence_array_create
+ dma_fence_array_create
|
- to_fence_array
+ to_dma_fence_array
|
- fence_is_array
+ dma_fence_is_array
|
- trace_fence_emit
+ trace_dma_fence_emit
|
- FENCE_TRACE
+ DMA_FENCE_TRACE
|
- FENCE_WARN
+ DMA_FENCE_WARN
|
- FENCE_ERR
+ DMA_FENCE_ERR
)
(
...
)

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Gustavo Padovan <gustavo.padovan@collabora.co.uk>
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20161025120045.28839-1-chris@chris-wilson.co.uk

drm/nouveau: Remove call to reservation_object_test_signaled_rcu before wait

Since fence_wait_timeout_reservation_object_wait_timeout_rcu() with a
timeout of 0 becomes reservation_object_test_signaled_rcu(), we do not
need to handle such conversion in the caller. The only challenge are
those callers that wish to differentiate the error code between the
nonblocking busy check and potentially blocking wait.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Sumit Semwal <sumit.semwal@linaro.org>
Link: http://patchwork.freedesktop.org/patch/msgid/20160829070834.22296-4-chris@chris-wilson.co.uk

drm/nouveau: rename nouveau_drm.h to nouveau_drv.h

Fixes out-of-tree build issue where uapi/drm/nouveau_drm.h gets picked
up instead.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm: remove unused dev variables

After drm_gem_object_lookup() was changed along with all its callers,
we have several drivers that have unused variables:

drm/armada/armada_crtc.c: In function 'armada_drm_crtc_cursor_set':
drm/armada/armada_crtc.c:900:21: error: unused variable 'dev' [-Werror=unused-variable]
drm/nouveau/nouveau_gem.c: In function 'validate_init':
drm/nouveau/nouveau_gem.c:371:21: error: unused variable 'dev' [-Werror=unused-variable]
drm/nouveau/nv50_display.c: In function 'nv50_crtc_cursor_set':
drm/nouveau/nv50_display.c:1308:21: error: unused variable 'dev' [-Werror=unused-variable]
drm/radeon/radeon_cs.c: In function 'radeon_cs_parser_relocs':
drm/radeon/radeon_cs.c:77:21: error: unused variable 'ddev' [-Werror=unused-variable]

This fixes all the instances I found with ARM randconfig builds so far.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Fixes: a8ad0bd84f98 ("drm: Remove unused drm_device from drm_gem_object_lookup()")
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1463587653-3035181-6-git-send-email-arnd@arndb.de

drm: Remove unused drm_device from drm_gem_object_lookup()

drm_gem_object_lookup() has never required the drm_device for its file
local translation of the user handle to the GEM object. Let's remove the
unused parameter and save some space.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: dri-devel@lists.freedesktop.org
Cc: Dave Airlie <airlied@redhat.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
[danvet: Fixup kerneldoc too.]
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

drm/ttm: remove lazy parameter from ttm_bo_wait

Not used any more.

Reviewed-by: Sinclair Yeh <syeh@vmware.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: remove use_ticket parameter from ttm_bo_reserve

Not used any more.

Reviewed-by: Sinclair Yeh <syeh@vmware.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/nouveau/abi16: remove unused argument from nouveau_abi16_get()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fix memory leak

If pm_runtime_get_sync() we were going to "out" but we missed freeing
vma.

Signed-off-by: Sudip Mukherjee <sudip@vectorindia.org>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: remove unnecessary usage of object handles

No longer required in a lot of cases, as objects are identified over NVIF
via an alternate mechanism since the rework.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gem: return only valid domain when there's only one

On nv50+, we restrict the valid domains to just the one where the buffer
was originally created. However after the buffer is evicted to system
memory, we might move it back to a different domain that was not
originally valid. When sharing the buffer and retrieving its GEM_INFO
data, we still want the domain that will be valid for this buffer in a
pushbuf, not the one where it currently happens to be.

This resolves fdo#92504 and several others. These are due to suspend
evicting all buffers, making it more likely that they temporarily end up
in the wrong place.

Cc: stable@vger.kernel.org
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92504
Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gem: return only valid domain when there's only one

On nv50+, we restrict the valid domains to just the one where the buffer
was originally created. However after the buffer is evicted to system
memory, we might move it back to a different domain that was not
originally valid. When sharing the buffer and retrieving its GEM_INFO
data, we still want the domain that will be valid for this buffer in a
pushbuf, not the one where it currently happens to be.

This resolves fdo#92504 and several others. These are due to suspend
evicting all buffers, making it more likely that they temporarily end up
in the wrong place.

Cc: stable@vger.kernel.org
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92504
Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/fb: convert to new-style nvkm_subdev

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/nvif: simplify and tidy library interfaces

A variety of tweaks to the NVIF library interfaces, mostly ripping out
things that turned out to be not so useful.

- Removed refcounting from nvif_object, callers are expected to not be
stupid instead.
- nvif_client is directly reachable from anything derived from nvif_object,
removing the need for heuristics to locate it
- _new() versions of interfaces, that allocate memory for the object
they construct, have been removed. The vast majority of callers used
the embedded _init() interfaces.
- No longer storing constructor arguments (and the data returned from
nvkm) inside nvif_object, it's more or less unused and just wastes
memory.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: use dev_* for logging

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/fb: cosmetic changes

This is purely preparation for upcoming commits, there should be no
code changes here.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gem: use kvfree() in u_free()

Use kvfree() instead of open-coding it.

Signed-off-by: Pekka Enberg <penberg@kernel.org>
Cc: David Airlie <airlied@linux.ie>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

drm/nouveau/gem: allow user-space to specify an object should be coherent

User-space use mappable BOs notably for fences, and expects that a
value update by the GPU will be immediatly visible through the
user-space mapping.

ARM has a property that may prevent this from happening though: memory
can be mapped multiple times only if the different mappings share the
same caching properties. However all the lowmem memory is already
identity-mapped into the kernel with cache enabled, so when user-space
requests an uncached mapping, we actually get an "undefined caching
policy" one and this has strange side-effects described on Freedesktop
bug 86690.

To prevent this from happening, allow user-space to explicitly specify
which objects should be coherent, and create such objects with the
TTM_PL_FLAG_UNCACHED flag. This will make TTM allocate memory using the
DMA API, which will fix the identify mapping and allow us to safely map
the objects to user-space uncached.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Lucas Stach <dev@lynxeye.de>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: finalise nvkm namespace switch (no binary change)

The namespace of NVKM is being changed to nvkm_ instead of nouveau_,
which will be used for the DRM part of the driver. This is being
done in order to make it very clear as to what part of the driver a
given symbol belongs to, and as a minor step towards splitting the
DRM driver out to be able to stand on its own (for virt).

Because there's already a large amount of churn here anyway, this is
as good a time as any to also switch to NVIDIA's device and chipset
naming to ease collaboration with them.

A comparison of objdump disassemblies proves no code changes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/nvif: namespace of nvkm accessors (no binary change)

NVKM is having it's namespace switched to nvkm_, which will conflict
with these functions (which are workarounds for the fact that as of
yet, we still aren't able to split DRM and NVKM completely).

A comparison of objdump disassemblies proves no code changes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gem: Remove unused function

Remove the function domain_to_ttm() that is not used anywhere.

This was partially found by using a static code analysis program
called cppcheck.

Signed-off-by: Rickard Strandqvist <rickard_strandqvist@spectrumdigital.se>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Revert "drm/gem: Warn on illegal use of the dumb buffer interface v2"

This reverts commit 355a70183848f21198e9f6296bd646df3478a26d.

This had some bad side effects under normal operation, and should
have been dropped earlier.

Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: wake up the card if necessary during gem callbacks

The failure paths if we fail to wake the card are less than desirable,
but there's not really a graceful way to handle this case currently.

I'll keep this situation in mind when I get to fixing other vm-related
issues.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: synchronize BOs when required

On architectures for which access to GPU memory is non-coherent,
caches need to be flushed and invalidated explicitly when BO control
changes between CPU and GPU.

This patch adds buffer synchronization functions which invokes the
correct API (PCI or DMA) to ensure synchronization is effective.

Based on the TTM DMA cache helper patches by Lucas Stach.

Signed-off-by: Lucas Stach <dev@lynxeye.de>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/gem: Warn on illegal use of the dumb buffer interface v2

It happens on occasion that developers of generic user-space applications
abuse the dumb buffer API to get hold of drm buffers that they can both
mmap() and use for GPU acceleration, using the assumptions that dumb buffers
and buffers available for GPU are
a) The same type and can be aribtrarily type-casted.
b) fully coherent.

This patch makes the most widely used drivers warn nicely when that happens,
the next step will be to fail.

v2: Move drmP.h changes to drm_gem.h. Fix Radeon dumb mmap breakage.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: export reservation_object from dmabuf to ttm

Adds an extra argument to nouveau_bo_new, which is only used in nouveau_prime.c.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>

drm/nouveau: specify if interruptible wait is desired in nouveau_fence_sync

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>

drm/nouveau: use shared fences for readable objects

nouveau keeps track in userspace whether a buffer is being
written to or being read, but it doesn't use that information.

Change this to allow multiple readers on the same bo.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Acked-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Keep only a single list for validation.

Maintain the original order to handle VRAM/GART/mixed correctly for <nv50,
it's likely not as important on newer cards.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Acked-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: use rcu in nouveau_gem_ioctl_cpu_prep

With the conversion to the reservation api this should be safe.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Acked-by: Ben Skeggs <bskeggs@redhat.com>

drm/ttm: flip the switch, and convert to dma_fence

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>

drm/nouveau: rework to new fence interface

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Acked-by: Ben Skeggs <bskeggs@redhat.com>

drm/ttm: kill fence_lock

No users are left, kill it off! :D
Conversion to the reservation api is next on the list, after
that the functionality can be restored with rcu.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>

drm/nouveau: add reservation to nouveau_gem_ioctl_cpu_prep

Apart from some code inside ttm itself and nouveau_bo_vma_del,
this is the only place where ttm_bo_wait is used without a reservation.
Fix this so we can remove the fence_lock later on.

After the switch to rcu the reservation lock will be
removed again.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Acked-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: port to nvif client/device/objects

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: initial pass at moving to struct nvif_device

This is an attempt at isolating some of the changes necessary to port
to NVIF in a separate commit.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fix some usages of the wrong print function

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: store a pointer to vm in nouveau_cli

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/ttm: Fix a few sparse warnings

The final parameter to ttm_bo_reserve() is a pointer, therefore callers
should use NULL instead of 0.

Fixes a bunch of sparse warnings of this type:

warning: Using plain integer as NULL pointer

Signed-off-by: Thierry Reding <treding@nvidia.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm: init TTM dev_mapping in ttm_bo_device_init()

With dev->anon_inode we have a global address_space ready for operation
right from the beginning. Therefore, there is no need to do a delayed
setup with TTM. Instead, set dev_mapping during initialization in
ttm_bo_device_init() and remove any "if (dev_mapping)" conditions.

Cc: Dave Airlie <airlied@redhat.com>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Cc: Alex Deucher <alexdeucher@gmail.com>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: David Herrmann <dh.herrmann@gmail.com>

drm: use anon-inode instead of relying on cdevs

DRM drivers share a common address_space across all character-devices of a
single DRM device. This allows simple buffer eviction and mapping-control.
However, DRM core currently waits for the first ->open() on any char-dev
to mark the underlying inode as backing inode of the device. This delayed
initialization causes ugly conditions all over the place:
if (dev->dev_mapping)
do_sth();

To avoid delayed initialization and to stop reusing the inode of the
char-dev, we allocate an anonymous inode for each DRM device and reset
filp->f_mapping to it on ->open().

Signed-off-by: David Herrmann <dh.herrmann@gmail.com>

drm/nouveau/gem: remove (now) unneeded pre-validate fence sync

Now that nouveau_bo.c can handle sync when it actually needs to, we can
remove this and avoid a double semaphore acquire when syncing in the
command submission path.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm: Kill DRM_COPY_(TO|FROM)_USER

Less yelling ftw!

Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: when bailing out of a pushbuf ioctl, do not remove previous fence

Cc: stable@vger.kernel.org
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: allow nouveau_fence_ref() to be a noop

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm: kill ->gem_init_object() and friends

All drivers embed gem-objects into their own buffer objects. There is no
reason to keep drm_gem_object_alloc(), gem->driver_private and
->gem_init_object() anymore.

New drivers are highly encouraged to do the same. There is no benefit in
allocating gem-objects separately.

Cc: Dave Airlie <airlied@gmail.com>
Cc: Alex Deucher <alexdeucher@gmail.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Jerome Glisse <jglisse@redhat.com>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Inki Dae <inki.dae@samsung.com>
Cc: Ben Skeggs <skeggsb@gmail.com>
Cc: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
Signed-off-by: David Herrmann <dh.herrmann@gmail.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: embed gem object in nouveau_bo

There is no reason to keep the gem object separately allocated. nouveau is
the last user of gem_obj->driver_private, so if we embed it, we can get
rid of 8bytes per gem-object.

The implementation follows the radeon driver. bo->gem is only valid, iff
the bo was created via the gem helpers _and_ iff the user holds a valid
gem reference. That is, as the gem object holds a reference to the
nouveau_bo. If you use nouveau_ref() to gain a bo reference, you are not
guaranteed to also hold a gem reference. The gem object might get
destroyed after the last user drops the gem-ref via
drm_gem_object_unreference(). Use drm_gem_object_reference() to gain a
gem-reference.

For debugging, we can use bo->gem.filp != NULL to test whether a gem-bo is
valid. However, this shouldn't be used for real functionality to avoid
gem-internal dependencies.

Note that the implementation follows the previous style. However, we no
longer can check for bo->gem != NULL to test for a valid gem object. This
wasn't done before, so we should be safe now.

Signed-off-by: David Herrmann <dh.herrmann@gmail.com>
Acked-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: fix command submission to use vmalloc for big allocations

I was getting a order 4 allocation failure from kmalloc when testing some
game after a few days uptime with some suspend/resumes.

For big allocations vmalloc should be used instead.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/ttm: convert to unified vma offset manager

Use the new vma-manager infrastructure. This doesn't change any
implementation details as the vma-offset-manager is nearly copied 1-to-1
from TTM.

The vm_lock is moved into the offset manager so we can drop it from TTM.
During lookup, we use the vma locking helpers to take a reference to the
found object.
In all other scenarios, locking stays the same as before. We always
guarantee that drm_vma_offset_remove() is called only during destruction.
Hence, helpers like drm_vma_node_offset_addr() are always safe as long as
the node has a valid offset.

This also drops the addr_space_offset member as it is a copy of vm_start
in vma_node objects. Use the accessor functions instead.

v4:
- remove vm_lock
- use drm_vma_offset_lock_lookup() to protect lookup (instead of vm_lock)

Cc: Dave Airlie <airlied@redhat.com>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Cc: Martin Peres <martin.peres@labri.fr>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: David Herrmann <dh.herrmann@gmail.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Dave Airlie <airlied@gmail.com>

drm/nouveau: do not unpin in nouveau_gem_object_del

This should no longer be required, and is harmful for framebuffer pinning.
Also add a warning if unpin causes the pin count to drop below 0.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: delay busy bo vma removal until fence signals

As opposed to an explicit wait. Allows userspace to not stall waiting
on buffer deletion.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/ttm: convert to the reservation api

Now that the code is compatible in semantics, flip the switch.
Use ww_mutex instead of the homegrown implementation.

ww_mutex uses -EDEADLK to signal that the caller has to back off,
and -EALREADY to indicate this buffer is already held by the caller.

ttm used -EAGAIN and -EDEADLK for those, respectively. So some changes
were needed to handle this correctly.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/ttm: make ttm reservation calls behave like reservation calls

This commit converts the source of the val_seq counter to
the ww_mutex api. The reservation objects are converted later,
because there is still a lockdep splat in nouveau that has to
resolved first.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: complain loudly if buffer is pinned during destruction

Shouldn't happen, and we invert the struct_mutex with reservation here,
potentially leading to deadlocks. Once reservations become lockdep annotated,
lockdep will go splat on this.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Acked-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nv84-/fence: prepare for emit/sync support of sysram sequences

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: report channel owner in ioctl error paths

Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: use prime helpers

Simplify the Nouveau prime implementation by using the default behavior provided
by drm_gem_prime_import and drm_gem_prime_export.

v2: Rename functions to nouveau_gem_prime_get_sg_table and
nouveau_gem_prime_import_sg_table.

Signed-off-by: Aaron Plattner <aplattner@nvidia.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: David Airlie <airlied@linux.ie>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: use ttm_bo_reserve_slowpath in validate_init, v2

Similar rationale to the identical commit in drm/ttm.
Instead of only waiting for unreservation, we make sure we actually
own the reservation, then retry to get the rest.

Changes since v1:
- Increase the seqno before calling ttm_bo_reserve_slowpath

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>

drm/nouveau: increase reservation sequence every retry

This is temporary until the fence framework can be used. With the
lru/reservation atomicity removal it is possible to see your old
sequence number and the buffer being reserved, leading to erroneously
reporting -EDEADLK.

Workaround it by bumping the sequence number every retry.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>

drm/ttm: remove no_wait_reserve, v3

All items on the lru list are always reservable, so this is a stupid
thing to keep. Not only that, it is used in a way which would
guarantee deadlocks if it were ever to be set to block on reserve.

This is a lot of churn, but mostly because of the removal of the
argument which can be nested arbitrarily deeply in many places.

No change of code in this patch except removal of the no_wait_reserve
argument, the previous patch removed the use of no_wait_reserve.

v2:
- Warn if -EBUSY is returned on reservation, all objects on the list
should be reservable. Adjusted patch slightly due to conflicts.
v3:
- Focus on no_wait_reserve removal only.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: port remainder of drm code, and rip out compat layer

v2: Ben Skeggs <bskeggs@redhat.com>
- fill in nouveau_pm.dev to prevent oops
- fix ppc issues (build + OF shadow)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: port all engines to new engine module format

This is a HUGE commit, but it's not nearly as bad as it looks - any problems
can be isolated to a particular chipset and engine combination. It was
simply too difficult to port each one at a time, the compat layers are
*already* ridiculous.

Most of the changes here are simply to the glue, the process for each of the
engine modules was to start with a standard skeleton and copy+paste the old
code into the appropriate places, fixing up variable names etc as needed.

v2: Marcin Slusarz <marcin.slusarz@gmail.com>
- fix find/replace bug in license header

v3: Ben Skeggs <bskeggs@redhat.com>
- bump indirect pushbuf size to 8KiB, 4KiB barely enough for userspace and
left no space for kernel's requirements during GEM pushbuf submission.
- fix duplicate assignments noticed by clang

v4: Marcin Slusarz <marcin.slusarz@gmail.com>
- add sparse annotations to nv04_fifo_pause/nv04_fifo_start
- use ioread32_native/iowrite32_native for fifo control registers

v5: Ben Skeggs <bskeggs@redhat.com>
- rebase on v3.6-rc4, modified to keep copy engine fix intact
- nv10/fence: unmap fence bo before destroying
- fixed fermi regression when using nvidia gr fuc
- fixed typo in supported dma_mask checking

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/fb: merge fb/vram and port to subdev interfaces

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: implement module init functions in nouveau_drm.c

These currently just call the existing ones in nouveau_drv.c, but will be
extended in upcoming commits. This needed to be separated from the current
code as there will be some header clashes until things are ported.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gem: use bo.offset rather than mm_node.start

Won't necessarily be a drm_mm_node in the future, and I can't think of any
good reason to not use the offset from the bo struct. There may have been
some reason once apon a time, but, separate commit just in case.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

UAPI: (Scripted) Convert #include "..." to #include <path/...> in drivers/gpu/

Convert #include "..." to #include <path/...> in drivers/gpu/.

Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Dave Jones <davej@redhat.com>

UAPI: (Scripted) Remove redundant DRM UAPI header #inclusions from drivers/gpu/.

Remove redundant DRM UAPI header #inclusions from drivers/gpu/.

Remove redundant #inclusions of core DRM UAPI headers (drm.h, drm_mode.h and
drm_sarea.h). They are now #included via drmP.h and drm_crtc.h via a preceding
patch.

Without this patch and the patch to make include the UAPI headers from the core
headers, after the UAPI split, the DRM C sources cannot find these UAPI headers
because the DRM code relies on specific -I flags to make #include "..." work
on headers in include/drm/ - but that does not work after the UAPI split without
adding more -I flags.

Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Dave Jones <davej@redhat.com>

drm/nouveau/gem: fix object reference leak in a failure path

Reported-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>

drm: track dev_mapping in more robust and flexible way

Setting dev_mapping (pointer to the address_space structure
used for memory mappings) to the address_space of the first
opener's inode and then failing if other openers come in
through a different inode has a few restrictions that are
eliminated by this patch.

If we already have valid dev_mapping and we spot an opener
with different i_node, we force its i_mapping pointer to the
already established address_space structure (first opener's
inode). This will make all mappings from drm device hang off
the same address_space object.

Some benefits (things that now work and didn't work
before) of this patch are:

* user space can mknod and use any number of device
nodes and they will all work fine as long as the major
device number is that of the drm module.
* user space can even remove the first opener's device
nodes and mknod the new one and the applications and
windowing system will still work.
* GPU drivers can safely assume that dev->dev_mapping is
correct address_space and just blindly copy it
into their (private) bdev.dev_mapping

For reference, some discussion that lead to this patch can
be found here:

http://lists.freedesktop.org/archives/dri-devel/2012-April/022283.html

Signed-off-by: Ilija Hadzic <ihadzic@research.bell-labs.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau/fence: convert to exec engine, and improve channel sync

Now have a somewhat simpler semaphore sync implementation for nv17:nv84,
and a switched to using semaphores as fences on nv84+ and making use of
the hardware's >= acquire operation.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/fence: minor api changes for an upcoming rework

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

nouveau: add PRIME support

This adds prime->fd and fd->prime support to nouveau,
it passes the SG object to TTM, and then populates the
GART entries using it.

v2: add stubbed kmap + use new function to fill out pages array
for faulting + add reimport test.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau/ttm: always do buffer moves on kernel channel

There was once good reasons for wanting the drm to be able to use M2MF etc
on user channels, but they're not relevant anymore. For the general
buffer move case, we've already lost by transferring between vram/sysmem
already so the context switching overhead is minimal in comparison.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gem: fix fence_sync race / oops

Due to a race it was possible for a fence to be destroyed while another
thread was trying to synchronise with it. If this happened in the fallback
non-semaphore path, it lead to the following oops due to fence->channel
being NULL.

BUG: unable to handle kernel NULL pointer dereference at (null)
IP: [<fa9632ce>] nouveau_fence_update+0xe/0xe0 [nouveau]
*pde = a649c067
SMP
Modules linked in: fuse nouveau(O) ttm(O) drm_kms_helper(O) drm(O) mxm_wmi video wmi netconsole configfs lockd bnep bluetooth rfkill ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ip6table_filter ip6_tables snd_hda_codec_realtek snd_hda_intel snd_hda_cobinfmt_misc uinput ata_generic pata_acpi pata_aet2c_algo_bit i2c_core [last unloaded: wmi]

Pid: 2255, comm: gnome-shell Tainted: G O 3.2.0-0.rc5.git0.1.fc17.i686 #1 System manufacturer System Product Name/M2A-VM
EIP: 0060:[<fa9632ce>] EFLAGS: 00010296 CPU: 1
EIP is at nouveau_fence_update+0xe/0xe0 [nouveau]
EAX: 00000000 EBX: ddfc6dd0 ECX: dd111580 EDX: 00000000
ESI: 00003e80 EDI: dd111580 EBP: dd121d00 ESP: dd121ce8
DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
Process gnome-shell (pid: 2255, ti=dd120000 task=dd111580 task.ti=dd120000)
Stack:
7dc86c76 00000000 00003e80 ddfc6dd0 00003e80 dd111580 dd121d0c fa96371f
00000000 dd121d3c fa963773 dd111580 01000246 000ec53d 00000000 ddfc6dd0
00001f40 00000000 ddfc6dd0 00000010 dc7df840 dd121d6c fa9639a0 00000000
Call Trace:
[<fa96371f>] __nouveau_fence_signalled+0x1f/0x30 [nouveau]
[<fa963773>] __nouveau_fence_wait+0x43/0xd0 [nouveau]
[<fa9639a0>] nouveau_fence_sync+0x1a0/0x1c0 [nouveau]
[<fa964046>] validate_list+0x176/0x300 [nouveau]
[<f7d9c9c0>] ? ttm_bo_mem_put+0x30/0x30 [ttm]
[<fa964b8a>] nouveau_gem_ioctl_pushbuf+0x48a/0xfd0 [nouveau]
[<c0406481>] ? die+0x31/0x80
[<f7c93d98>] drm_ioctl+0x388/0x490 [drm]
[<c0406481>] ? die+0x31/0x80
[<fa964700>] ? nouveau_gem_ioctl_new+0x150/0x150 [nouveau]
[<c0635c7b>] ? file_has_perm+0xcb/0xe0
[<f7c93a10>] ? drm_copy_field+0x80/0x80 [drm]
[<c0564f56>] do_vfs_ioctl+0x86/0x5b0
[<c0406481>] ? die+0x31/0x80
[<c0635f22>] ? selinux_file_ioctl+0x62/0x130
[<c0554f30>] ? fget_light+0x30/0x340
[<c05654ef>] sys_ioctl+0x6f/0x80
[<c099e3a4>] syscall_call+0x7/0xb
[<c0406481>] ? die+0x31/0x80
[<c0406481>] ? die+0x31/0x80

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Cc: stable@vger.kernel.org

Revert "drm/ttm: add a way to bo_wait for either the last read or last write"

This reverts commit dfadbbdb57b3f2bb33e14f129a43047c6f0caefa.

Further upstream discussion between Marek and Thomas decided this wasn't
fully baked and needed further work, so revert it before it hits mainline.

Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/ttm: add a way to bo_wait for either the last read or last write

Sometimes we want to know whether a buffer is busy and wait for it (bo_wait).
However, sometimes it would be more useful to be able to query whether
a buffer is busy and being either read or written, and wait until it's stopped
being either read or written. The point of this is to be able to avoid
unnecessary waiting, e.g. if a GPU has written something to a buffer and is now
reading that buffer, and a CPU wants to map that buffer for read, it needs to
only wait for the last write. If there were no write, there wouldn't be any
waiting needed.

This, or course, requires user space drivers to send read/write flags
with each relocation (like we have read/write domains in radeon, so we can
actually use those for something useful now).

Now how this patch works:

The read/write flags should passed to ttm_validate_buffer. TTM maintains
separate sync objects of the last read and write for each buffer, in addition
to the sync object of the last use of a buffer. ttm_bo_wait then operates
with one the sync objects.

Signed-off-by: Marek Olšák <maraeo@gmail.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: fix nouveau_vma object leak

Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: remove implicit mapping of every bo into chan_vm

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: remove 'chan' argument from nouveau_bo_new

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fixup gem_info ioctl to return client-specific bo virtual

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nv50-nvc0: completely disable relocs

GPU virtual addresses are constant now so this should never be getting hit
anyway and userspace shouldn't break from them being ignored.

This is being done in preference to teaching the code how to deal with BOs
that exist at different virtual addresses within separate VMs.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: convert some bo.offset use to vma.offset

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gem: implement stub hooks for GEM object open/close

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: remove 'chan' argument from nouveau_gem_new

Userspace hasn't passed us a channel_hint for a long long time now, and
there isn't actually a need to do so anymore anyway.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: store a per-client channel list

Removes the need to disable IRQs to lookup channel struct on every pushbuf
ioctl, among others.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm: fix "persistant" typo

Signed-off-by: Jan Engelhardt <jengelh@medozas.de>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: properly handle pushbuffer check failures

When "buffer in list" check does not pass, don't free validation lists - they were
not initialized yet.

Fixes this oops:

[drm] nouveau 0000:02:00.0: push 105 buffer not in list
BUG: unable to handle kernel NULL pointer dereference at 000000000000057c
IP: [<ffffffff81236aa4>] do_raw_spin_lock+0x14/0x13c
PGD 1ac6cb067 PUD 1aaa52067 PMD 0
CPU 0
Modules linked in: nouveau ttm drm_kms_helper snd_hda_codec_realtek snd_hda_intel snd_hda_codec

Pid: 6265, comm: OilRush_x86 Not tainted 2.6.38-rc6-nv+ #632 System manufacturer System Product Name/P6T SE
RIP: 0010:[<ffffffff81236aa4>] [<ffffffff81236aa4>] do_raw_spin_lock+0x14/0x13c
(...)
Process OilRush_x86 (pid: 6265, threadinfo ffff8801a6aee000, task ffff8801a26c0000)
0000000000000000 ffff8801ac74c618 0000000000000000 0000000000000578
0000000000000000 ffff8801ac74c618 0000000000000000 ffff8801bd9d0000
[<ffffffff81417f78>] _raw_spin_lock+0x1e/0x22
[<ffffffffa00a2746>] nouveau_bo_fence+0x2e/0x60 [nouveau]
[<ffffffffa00a540b>] validate_fini_list+0x35/0xeb [nouveau]
[<ffffffffa00a54d3>] validate_fini+0x12/0x31 [nouveau]
[<ffffffffa00a6386>] nouveau_gem_ioctl_pushbuf+0xe94/0xf6b [nouveau]
[<ffffffff8141ac56>] ? sub_preempt_count+0x9e/0xb2
[<ffffffff81417e94>] ? _raw_spin_unlock_irqrestore+0x30/0x4d
[<ffffffff8105dea2>] ? __wake_up+0x3f/0x48
[<ffffffff812aebb4>] drm_ioctl+0x289/0x361
[<ffffffff8141ac56>] ? sub_preempt_count+0x9e/0xb2
[<ffffffffa00a54f2>] ? nouveau_gem_ioctl_pushbuf+0x0/0xf6b [nouveau]
[<ffffffff8141ac56>] ? sub_preempt_count+0x9e/0xb2
[<ffffffffa010caa2>] nouveau_compat_ioctl+0x16/0x1c [nouveau]
[<ffffffff81142c0d>] compat_sys_ioctl+0x1c8/0x12d7
[<ffffffff814179ca>] ? trace_hardirqs_off_thunk+0x3a/0x6c
[<ffffffff81058099>] sysenter_dispatch+0x7/0x30
[<ffffffff8141798e>] ? trace_hardirqs_on_thunk+0x3a/0x3c
RIP [<ffffffff81236aa4>] do_raw_spin_lock+0x14/0x13c
RSP <ffff8801a6aefb88>
---[ end trace 0014d5d93e6147e1 ]---

Additionally, don't call validate_fini twice in case of validation failure.

Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Maarten Maathuis <madman2003@gmail.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nv50-nvc0: restrict memtype to those specified at creation time

Upcoming patches are going to enable full support for buffers that keep
a constant GPU virtual address whenever they're validated for use by
the GPU.

In order for this to work properly while keeping support for large pages,
we need to know if it's ever going to be possible for a buffer to end
up in GART, and if so, disable large pages for the buffer's VMA.

This is a new restriction that's not present in earlier kernel's, but
should not break userspace as the current code never attempts to validate
buffers into a memtype other than it was created with.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: pass domain rather than ttm flags to gem_new()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: remove no_vm/mappable flags from nouveau_bo

'mappable' isn't really used at all, nor is it necessary anymore as the
bo code is capable of moving buffers to mappable vram as required.

'no_vm' isn't necessary anymore either, any places that don't want to be
mapped into a GPU address space should allocate the VRAM directly instead.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: kick vram functions out into an "engine"

NVC0 will be able to share some of nv50's paths this way. This also makes
it the card-specific vram code responsible for deciding if a given set
of tile_flags is valid, rather than duplicating the allowed types in
nv50_vram.c and nouveau_gem.c

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: wrap calls to ttm_bo_validate()

This will be used later to fixup bo.offset with a buffer's fixed GPU
virtual address.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Implement the pageflip ioctl.

nv0x-nv4x should be mostly fine, nv50 doesn't work yet.

Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fix annoying nouveau_fence type issue

nouveau_fence_* functions are not type safe, which could lead to bugs.
Additionally every use of nouveau_fence_unref had to cast struct
nouveau_fence to void **.
Fix it by renaming old functions and creating static inline functions with
new prototypes. We still need old functions, because we pass function
pointers to ttm.
As we are wrapping functions, drop unused "void *arg" parameter where possible.

Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: use interruptible waits during pushbuf validation

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: remove cpu_writers lock

No other driver uses this, and userspace should be responsible for handling
locking between them if they share BOs.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: add more fine-grained locking to channel list + structures

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/ttm/radeon/nouveau: Kill the bo lock in favour of a bo device fence_lock

The bo lock used only to protect the bo sync object members, and since it
is a per bo lock, fencing a buffer list will see a lot of locks and unlocks.
Replace it with a per-device lock that protects the sync object members on
*all* bos. Reading and setting these members will always be very quick, so
the risc of heavy lock contention is microscopic. Note that waiting for
sync objects will always take place outside of this lock.

The bo device fence lock will eventually be replaced with a seqlock /
rcu mechanism so we can determine that a bo is idle under a
rcu / read seqlock.

However this change will allow us to batch fencing and unreserving of
buffers with a minimal amount of locking.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Jerome Glisse <j.glisse@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: Expose some BO usage flags to userspace.

This will be needed for Z compression and to take smarter placement
decisions.

Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Acked-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Minor refactoring/cleanup of the fence code.

Mainly to make room for inter-channel sync.

Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/gem: handlecount isn't really a kref so don't make it one.

There were lots of places being inconsistent since handle count
looked like a kref but it really wasn't.

Fix this my just making handle count an atomic on the object,
and have it increase the normal object kref.

Now i915/radeon/nouveau drivers can drop the normal reference on
userspace object creation, and have the handle hold it.

This patch fixes a memory leak or corruption on unload, because
the driver had no way of knowing if a handle had been actually
added for this object, and the fbcon object needed to know this
to clean itself up properly.

Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: Don't take struct_mutex around the pushbuf IOCTL.

We don't need it and it can lead to lock order inversions with respect
to drm_global_mutex, potentially causing dead locks.

Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: drop drm_global_mutex before sleeping in submission path

If we keep hold of the mutex here, the process which currently holds the
buffer object will never be able to release it, causing a deadlock.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nv20: Don't use pushbuf calls on the original nv20.

The "return" command is buggy on the original nv20, it jumps back to
the caller address as expected, but it doesn't clear the subroutine
active bit making the subsequent pushbuf calls fail with a "stack"
overflow.

Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fix race condition when under memory pressure

When VRAM is running out it's possible that the client's push buffers get
evicted to main memory. When they're validated back in, the GPU may
be used for the copy back to VRAM, but the existing synchronisation code
only deals with inter-channel sync, not sync between PFIFO and PGRAPH on
the same channel. This leads to PFIFO fetching from command buffers that
haven't quite been copied by PGRAPH yet.

This patch marks push buffers as so, and forces any GPU-assisted buffer
moves to be done on a different channel, which triggers the correct
synchronisation to happen before we submit them.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm: Use ENOENT consistently for the error return for an unmatched handle.

This is consistent with trying to access a filename that not exist
within a directory which is a good analogy here. The main reason for the
change is that it is easy to confuse the error code of EBADF as an
performing an ioctl on an invalid file descriptor (rather than an
unknown object).

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: remove dev_priv->init_state and friends

Nouveau will no longer load at all if card initialisation fails, so all
these checks are unnecessary.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Fix a couple of sparse warnings.

Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm: free core gem object from driver callbacks

When drivers embed the core gem object into their own structures,
they'll have to do this. Temporarily this results in an ugly

kfree(gem_obj);

in every gem driver.

Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: Eric Anholt <eric@anholt.net>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: Make use of TTM busy_placements.

Previously we were filling it the same as "placements", but in some
cases there're valid alternatives that we were ignoring completely.
Keeping a back-up memory type helps on several low-mem situations.

Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/ttm: split no_wait argument in 2 GPU or reserve wait

There is case where we want to be able to wait only for the
GPU while not waiting for other buffer to be unreserved. This
patch split the no_wait argument all the way down in the whole
ttm path so that upper level can decide on what to wait on or
not.

[airlied: squashed these 4 for bisectability reasons.]
drm/radeon/kms: update to TTM no_wait splitted argument
drm/nouveau: update to TTM no_wait splitted argument
drm/vmwgfx: update to TTM no_wait splitted argument
[vmwgfx patch: Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>]

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Acked-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: fix missing spin_unlock in failure path

Found by sparse.

Signed-off-by: Luca Barbieri <luca@luca-barbieri.com>
Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: new gem pushbuf interface, bump to 0.0.16

This commit breaks the userspace interface, and requires a new libdrm for
nouveau to operate again.

The multiple GEM_PUSHBUF ioctls that were present in 0.0.15 for
compatibility purposes are now gone, and replaced with the new ioctl which
allows for multiple push buffers to be submitted (necessary for hw index
buffers in the nv50 3d driver) and relocations to be applied on any buffer.

A number of other ioctls (CARD_INIT, GEM_PIN, GEM_UNPIN) that were needed
for userspace modesetting have also been removed.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Francisco Jerez <currojerez@riseup.net>

drm/nouveau: remove PUSHBUF_CAL macro

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nv50: switch to indirect push buffer controls

PFIFO on G80 and up has a new mode where the main ring buffer is simply a
ring of pointers to indirect buffers containing the actual command/data
packets. In order to be able to implement index buffers in the 3D driver
we need to be able to submit data-only push buffers right after the cmd
packet header, which is only possible using the new command submission
method.

This commit doesn't make it possible to implement index buffers yet, some
userspace interface changes will be required, but it does allow for
testing/debugging of the hardware-side support in the meantime.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

Use drm_gem_object_[handle_]unreference_unlocked where possible

Mostly obvious simplifications.

The i915 pread/pwrite ioctls, intel_overlay_put_image and
nouveau_gem_new were incorrectly using the locked versions
without locking: this is also fixed in this patch.

Signed-off-by: Luca Barbieri <luca@luca-barbieri.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: call ttm_bo_wait with the bo lock held to prevent hang

nouveau_gem_ioctl_cpu_prep calls ttm_bo_wait without the bo lock held.
ttm_bo_wait unlocks that lock, and so must be called with it held.

Currently this bug causes libdrm nouveau_bo_busy() to hang the machine.

Signed-off-by: Luca Barbieri <luca at luca-barbieri.com>
Acked-by: Maarten Maathuis <madman2003@gmail.com>
Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fix failure path if userspace specifies no valid memtypes

We need to add the buffer to the list even if we fail, otherwise the
validate_fini() call won't unreserve + unreference the GEM object,
making TTM very unhappy.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: check pushbuffer bounds in ioctl

Currently there is no check that the pushbuffer request bounds are inside
the TTM BO.

This allows to instruct the kernel to do relocations on user-selected
addresses, since the relocation bounds checking relies on the request
bounds.

This can oops the kernel accidentally and is easily exploitable.

This patch adds bound checking and alignment checking for ->offset and
->nr_dwords.

It also makes some variables unsigned, which should have no effect,
but prevents possible bounds checking problems.

Signed-off-by: Luca Barbieri <luca@luca-barbieri.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Fix null deref in nouveau_fence_emit due to deleted fence

Currently Nouveau will unvalidate all buffers if it is forced to wait on
one, and then start revalidating from the beginning. While doing so, it
destroys the operation fence, causing nouveau_fence_emit to crash.

This patch fixes this bug by taking the fence object out of validate_op
and creating it just before emit. The fence pointer is initialized to 0
and unref'ed unconditionally.

In addition to fixing the bug, this prevents its reintroduction and
simplifies the code.

Signed-off-by: Luca Barbieri <luca@luca-barbieri.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Fix null deref in nouveau_fence_emit due to deleted fence

Currently Nouveau will unvalidate all buffers if it is forced to wait on
one, and then start revalidating from the beginning. While doing so, it
destroys the operation fence, causing nouveau_fence_emit to crash.

This patch fixes this bug by taking the fence object out of validate_op
and creating it just before emit. The fence pointer is initialized to 0
and unref'ed unconditionally.

In addition to fixing the bug, this prevents its reintroduction and
simplifies the code.

Signed-off-by: Luca Barbieri <luca@luca-barbieri.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: remove use of -ERESTART

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Add DRM driver for NVIDIA GPUs

This adds a drm/kms staging non-API stable driver for GPUs from NVIDIA.

This driver is a KMS-based driver and requires a compatible nouveau
userspace libdrm and nouveau X.org driver.

This driver requires firmware files not available in this kernel tree,
interested parties can find them via the nouveau project git archive.

This driver is reverse engineered, and is in no way supported by nVidia.

Support for nearly the complete range of nvidia hw from nv04->g80 (nv50)
is available, and the kms driver should support driving nearly all
output types (displayport is under development still) along with supporting
suspend/resume.

This work is all from the upstream nouveau project found at
nouveau.freedesktop.org.

The original authors list from nouveau git tree is:
Anssi Hannula <anssi.hannula@iki.fi>
Ben Skeggs <bskeggs@redhat.com>
Francisco Jerez <currojerez@riseup.net>
Maarten Maathuis <madman2003@gmail.com>
Marcin Kościelnicki <koriakin@0x04.net>
Matthew Garrett <mjg@redhat.com>
Matt Parnell <mparnell@gmail.com>
Patrice Mandin <patmandin@gmail.com>
Pekka Paalanen <pq@iki.fi>
Xavier Chantry <shiningxc@gmail.com>
along with project founder Stephane Marchesin <marchesin@icps.u-strasbg.fr>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>