drm/nouveau: implement new VM_BIND uAPI

This commit provides the implementation for the new uapi motivated by the
Vulkan API. It allows user mode drivers (UMDs) to:

1) Initialize a GPU virtual address (VA) space via the new
DRM_IOCTL_NOUVEAU_VM_INIT ioctl for UMDs to specify the portion of VA
space managed by the kernel and userspace, respectively.

2) Allocate and free a VA space region as well as bind and unbind memory
to the GPUs VA space via the new DRM_IOCTL_NOUVEAU_VM_BIND ioctl.
UMDs can request the named operations to be processed either
synchronously or asynchronously. It supports DRM syncobjs
(incl. timelines) as synchronization mechanism. The management of the
GPU VA mappings is implemented with the DRM GPU VA manager.

3) Execute push buffers with the new DRM_IOCTL_NOUVEAU_EXEC ioctl. The
execution happens asynchronously. It supports DRM syncobj (incl.
timelines) as synchronization mechanism. DRM GEM object locking is
handled with drm_exec.

Both, DRM_IOCTL_NOUVEAU_VM_BIND and DRM_IOCTL_NOUVEAU_EXEC, use the DRM
GPU scheduler for the asynchronous paths.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230804182406.5222-12-dakr@redhat.com

drm/ttm: merge ttm_bo_api.h and ttm_bo_driver.h v2

Merge and cleanup the two headers into a single description of the
object API. Also move all the documentation to the implementation and
drop unnecessary includes from the header.

No functional change.

v2: minimal checkpatch.pl cleanup

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221125102137.1801-4-christian.koenig@amd.com

drm/nouveau: fix a use-after-free in nouveau_gem_prime_import_sg_table()

nouveau_bo_init() is backed by ttm_bo_init() and ferries its return code
back to the caller. On failures, ttm will call nouveau_bo_del_ttm() and
free the memory.Thus, when nouveau_bo_init() returns an error, the gem
object has already been released. Then the call to nouveau_bo_ref() will
use the freed "nvbo->bo" and lead to a use-after-free bug.

We should delete the call to nouveau_bo_ref() to avoid the use-after-free.

Signed-off-by: Jianglei Nie <niejianglei2021@163.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Signed-off-by: Lyude Paul <lyude@redhat.com>
Fixes: 019cbd4a4feb ("drm/nouveau: Initialize GEM object before TTM object")
Cc: Thierry Reding <treding@nvidia.com>
Cc: <stable@vger.kernel.org> # v5.4+
Link: https://patchwork.freedesktop.org/patch/msgid/20220705132546.2247677-1-niejianglei2021@163.com

dma-buf: wait for map to complete for static attachments

We have previously done that in the individual drivers but it is
more defensive to move that into the common code.

Dynamic attachments should wait for map operations to complete by themselves.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20220407085946.744568-12-christian.koenig@amd.com

drm/nouveau: wait for moving fence after pinning v2

We actually need to wait for the moving fence after pinning
the BO to make sure that the pin is completed.

v2: grab the lock while waiting

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
References: https://lore.kernel.org/dri-devel/20210621151758.2347474-1-daniel.vetter@ffwll.ch/
CC: stable@kernel.org
Link: https://patchwork.freedesktop.org/patch/msgid/20210622114506.106349-1-christian.koenig@amd.com

drm/ttm: cleanup BO size handling v3

Based on an idea from Dave, but cleaned up a bit.

We had multiple fields for essentially the same thing.

Now bo->base.size is the original size of the BO in
arbitrary units, usually bytes.

bo->mem.num_pages is the size in number of pages in the
resource domain of bo->mem.mem_type.

v2: use the GEM object size instead of the BO size
v3: fix printks in some places

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com> (v1)
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/406831/

drm/gem: Use struct dma_buf_map in GEM vmap ops and convert GEM backends

This patch replaces the vmap/vunmap's use of raw pointers in GEM object
functions with instances of struct dma_buf_map. GEM backends are
converted as well. For most of them, this simply changes the returned type.

TTM-based drivers now return information about the location of the memory,
either system or I/O memory. GEM VRAM helpers and qxl now use ttm_bo_vmap()
et al. Amdgpu, nouveau and radeon use drm_gem_ttm_vmap() et al instead of
implementing their own vmap callbacks.

v7:
* init QXL cursor to mapped BO buffer (kernel test robot)
v5:
* update vkms after switch to shmem
v4:
* use ttm_bo_vmap(), drm_gem_ttm_vmap(), et al. (Daniel, Christian)
* fix a trailing { in drm_gem_vmap()
* remove several empty functions instead of converting them (Daniel)
* comment uses of raw pointers with a TODO (Daniel)
* TODO list: convert more helpers to use struct dma_buf_map

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Acked-by: Christian König <christian.koenig@amd.com>
Tested-by: Sam Ravnborg <sam@ravnborg.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20201103093015.1063-7-tzimmermann@suse.de

drm/nouveau: Introduce GEM object functions

GEM object functions deprecate several similar callback interfaces in
struct drm_driver. This patch replaces the per-driver callbacks with
per-instance callbacks in nouveau.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200923102159.24084-11-tzimmermann@suse.de

drm/nouveau: stop using TTM placement flags

Those are going to be removed, stop using them here.

Instead use the GEM flags from the UAPI.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/389825/?series=81551&rev=1

drm: allow limiting the scatter list size.

Add drm_device argument to drm_prime_pages_to_sg(), so we can
call dma_max_mapping_size() to figure the segment size limit
and call into __sg_alloc_table_from_pages() with the correct
limit.

This fixes virtio-gpu with sev. Possibly it'll fix other bugs
too given that drm seems to totaly ignore segment size limits
so far ...

v2: place max_segment in drm driver not gem object.
v3: move max_segment next to the other gem fields.
v4: just use dma_max_mapping_size().

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20200907112425.15610-2-kraxel@redhat.com

drm/nouveau/prime: Extend DMA reservation object lock

Prior to commit 019cbd4a4feb ("drm/nouveau: Initialize GEM object before
TTM object"), the reservation object was locked across all of the buffer
object creation.

After splitting nouveau_bo_new() into separate nouveau_bo_alloc() and
nouveau_bo_init() functions, the reservation object is passed to the
latter, so the lock needs to be held across that function as well.

Fixes: 019cbd4a4feb ("drm/nouveau: Initialize GEM object before TTM object")
Signed-off-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Fix fallout from reservation object rework

Commit 019cbd4a4feb ("drm/nouveau: Initialize GEM object before TTM
object") introduced a subtle change in how the buffer allocation size is
handled. Prior to that change, the size would get aligned to at least a
page, whereas after that change a non-page-aligned size would get passed
through unmodified. This ultimately causes a BUG_ON() to trigger in
drm_gem_private_object_init() and crashes the system.

Fix this by restoring the code that align the allocation size.

Fixes: 019cbd4a4feb ("drm/nouveau: Initialize GEM object before TTM object")
Reported-by: Ilia Mirkin <imirkin@alum.mit.edu>
Signed-off-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: drop use of drmp.h

Drop use of the deprecated drmP.h file from drm/nouveau.

Build tested using allyesconfig and allmodconfig.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: nouveau@lists.freedesktop.org
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Initialize GEM object before TTM object

TTM assumes that drivers initialize the embedded GEM object before
calling the ttm_bo_init() function. This is not currently the case
in the Nouveau driver. Fix this by splitting up nouveau_bo_new()
into nouveau_bo_alloc() and nouveau_bo_init() so that the GEM can
be initialized before TTM BO initialization when necessary.

Fixes: b96f3e7c8069 ("drm/ttm: use gem vma_node")
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190814093524.GA31345@ulmo

dma-buf: rename reservation_object to dma_resv

Be more consistent with the naming of the other DMA-buf objects.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/323401/

drm/nouveau: use embedded gem object

Drop drm_gem_object from nouveau_bo, use the
ttm_buffer_object.base instead.

Build tested only.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Christian König <christian.koenig@amd.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20190805140119.7337-7-kraxel@redhat.com

dma-buf: add more reservation object locking wrappers

Complete the abstraction of the ww_mutex inside the reservation object.

This allows us to add more handling and debugging to the reservation
object in the future.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/320761/

drm/nouveau: Fill out gem_object->resv

That way we can ditch our gem_prime_res_obj implementation. Since ttm
absolutely needs the right reservation object all the boilerplate is
already there and we just have to wire it up correctly.

Note that gem/prime doesn't care when we do this, as long as we do it
before the bo is registered and someone can call the handle2fd ioctl
on it.

Aside: ttm_buffer_object.ttm_resv could probably be ditched in favour
of always passing a non-NULL resv to ttm_bo_init(). At least for gem
drivers that would avoid having two of these, on in ttm_buffer_object
and the other in drm_gem_object, one just there for confusion.

Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: nouveau@lists.freedesktop.org
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190725132655.11951-3-daniel.vetter@ffwll.ch

drm/nouveau: pass nvif_client to nouveau_bo_new() instead of drm_device

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: rename nouveau_drm.h to nouveau_drv.h

Fixes out-of-tree build issue where uapi/drm/nouveau_drm.h gets picked
up instead.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: teach nouveau_bo_pin() how to force a contig vram allocation

We have the ability to move buffers around in the kernel if necessary,
and should probably use it rather than failing if userspace passes us
a non-contig buffer for a plane.

The NOUVEAU_GEM_TILE_NONCONTIG flag from userspace will become a mere
initial placement hint once all the relevant paths have been updated.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: export reservation_object from dmabuf to ttm

Adds an extra argument to nouveau_bo_new, which is only used in nouveau_prime.c.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>

drm: Pass dma-buf as argument to gem_prime_import_sg_table

Allows importing dma_reservation_objects from a dma-buf.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>

dma-buf: use reservation objects

This allows reservation objects to be used in dma-buf. it's required
for implementing polling support on the fences that belong to a dma-buf.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Acked-by: Mauro Carvalho Chehab <m.chehab@samsung.com> #drivers/media/v4l2-core/
Acked-by: Thomas Hellstrom <thellstrom@vmware.com> #drivers/gpu/drm/ttm
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Daniel Vetter <daniel@ffwll.ch>
Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net> #drivers/gpu/drm/armada/
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

drm/nouveau: embed gem object in nouveau_bo

There is no reason to keep the gem object separately allocated. nouveau is
the last user of gem_obj->driver_private, so if we embed it, we can get
rid of 8bytes per gem-object.

The implementation follows the radeon driver. bo->gem is only valid, iff
the bo was created via the gem helpers _and_ iff the user holds a valid
gem reference. That is, as the gem object holds a reference to the
nouveau_bo. If you use nouveau_ref() to gain a bo reference, you are not
guaranteed to also hold a gem reference. The gem object might get
destroyed after the last user drops the gem-ref via
drm_gem_object_unreference(). Use drm_gem_object_reference() to gain a
gem-reference.

For debugging, we can use bo->gem.filp != NULL to test whether a gem-bo is
valid. However, this shouldn't be used for real functionality to avoid
gem-internal dependencies.

Note that the implementation follows the previous style. However, we no
longer can check for bo->gem != NULL to test for a valid gem object. This
wasn't done before, so we should be safe now.

Signed-off-by: David Herrmann <dh.herrmann@gmail.com>
Acked-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: implement prime helper unpin function

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Acked-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: use prime helpers

Simplify the Nouveau prime implementation by using the default behavior provided
by drm_gem_prime_import and drm_gem_prime_export.

v2: Rename functions to nouveau_gem_prime_get_sg_table and
nouveau_gem_prime_import_sg_table.

Signed-off-by: Aaron Plattner <aplattner@nvidia.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: David Airlie <airlied@linux.ie>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/prime: drop reference on imported dma-buf come from gem

Increasing ref counts of both dma-buf and gem for imported dma-buf come from gem
makes memory leak. release function of dma-buf cannot be called because f_count
of dma-buf increased by importing gem and gem ref count cannot be decrease
because of exported dma-buf.

So I add dma_buf_put() for imported gem come from its own gem into each drivers
having prime_import and prime_export capabilities. With this, only gem ref
count is increased if importing gem exported from gem of same driver.

Signed-off-by: Seung-Woo Kim <sw0312.kim@samsung.com>
Signed-off-by: Kyungmin.park <kyungmin.park@samsung.com>
Cc: Inki Dae <inki.dae@samsung.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Rob Clark <rob.clark@linaro.org>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: remove misleading comment from nouveau_prime_new

Copy/pasted from nouveau_gem_new.

Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: port remainder of drm code, and rip out compat layer

v2: Ben Skeggs <bskeggs@redhat.com>
- fill in nouveau_pm.dev to prevent oops
- fix ppc issues (build + OF shadow)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: port all engines to new engine module format

This is a HUGE commit, but it's not nearly as bad as it looks - any problems
can be isolated to a particular chipset and engine combination. It was
simply too difficult to port each one at a time, the compat layers are
*already* ridiculous.

Most of the changes here are simply to the glue, the process for each of the
engine modules was to start with a standard skeleton and copy+paste the old
code into the appropriate places, fixing up variable names etc as needed.

v2: Marcin Slusarz <marcin.slusarz@gmail.com>
- fix find/replace bug in license header

v3: Ben Skeggs <bskeggs@redhat.com>
- bump indirect pushbuf size to 8KiB, 4KiB barely enough for userspace and
left no space for kernel's requirements during GEM pushbuf submission.
- fix duplicate assignments noticed by clang

v4: Marcin Slusarz <marcin.slusarz@gmail.com>
- add sparse annotations to nv04_fifo_pause/nv04_fifo_start
- use ioread32_native/iowrite32_native for fifo control registers

v5: Ben Skeggs <bskeggs@redhat.com>
- rebase on v3.6-rc4, modified to keep copy engine fix intact
- nv10/fence: unmap fence bo before destroying
- fixed fermi regression when using nvidia gr fuc
- fixed typo in supported dma_mask checking

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: implement module init functions in nouveau_drm.c

These currently just call the existing ones in nouveau_drv.c, but will be
extended in upcoming commits. This needed to be separated from the current
code as there will be some header clashes until things are ported.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

UAPI: (Scripted) Convert #include "..." to #include <path/...> in drivers/gpu/

Convert #include "..." to #include <path/...> in drivers/gpu/.

Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Dave Jones <davej@redhat.com>

UAPI: (Scripted) Remove redundant DRM UAPI header #inclusions from drivers/gpu/.

Remove redundant DRM UAPI header #inclusions from drivers/gpu/.

Remove redundant #inclusions of core DRM UAPI headers (drm.h, drm_mode.h and
drm_sarea.h). They are now #included via drmP.h and drm_crtc.h via a preceding
patch.

Without this patch and the patch to make include the UAPI headers from the core
headers, after the UAPI split, the DRM C sources cannot find these UAPI headers
because the DRM code relies on specific -I flags to make #include "..." work
on headers in include/drm/ - but that does not work after the UAPI split without
adding more -I flags.

Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Dave Jones <davej@redhat.com>

drm/nouveau: add license header to prime.

Just forgot this when I posted it, and yes I'm the only person
to have changed the file since.

Signed-off-by: Dave Airlie <airlied@redhat.com>

nouveau: add vmap support to nouveau prime support

Tested sharing to udl.

Signed-off-by: Dave Airlie <airlied@redhat.com>

nouveau: add stub dma-buf mmap functionality.

This just adds a stub until we have some users in place to test
this with.

Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau/radeon: add static const to the dma-buf ops.

Reported-by: wfg@linux.intel.com
Signed-off-by: Dave Airlie <airlied@redhat.com>

nouveau: add PRIME support

This adds prime->fd and fd->prime support to nouveau,
it passes the SG object to TTM, and then populates the
GART entries using it.

v2: add stubbed kmap + use new function to fill out pages array
for faulting + add reimport test.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>