drm/nouveau: uapi: don't pass NO_PREFETCH flag implicitly

Currently, NO_PREFETCH is passed implicitly through
drm_nouveau_gem_pushbuf_push::length and drm_nouveau_exec_push::va_len.

Since this is a direct representation of how the HW is programmed it
isn't really future proof for a uAPI. Hence, fix this up for the new
uAPI and split up the va_len field of struct drm_nouveau_exec_push,
such that we keep 32bit for va_len and 32bit for flags.

For drm_nouveau_gem_pushbuf_push::length at least provide
NOUVEAU_GEM_PUSHBUF_NO_PREFETCH to indicate the bit shift.

While at it, fix up nv50_dma_push() as well, such that the caller
doesn't need to encode the NO_PREFETCH flag into the length parameter.

Signed-off-by: Danilo Krummrich <dakr@redhat.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230823181746.3446-1-dakr@redhat.com

drm/nouveau: uapi: don't pass NO_PREFETCH flag implicitly

Currently, NO_PREFETCH is passed implicitly through
drm_nouveau_gem_pushbuf_push::length and drm_nouveau_exec_push::va_len.

Since this is a direct representation of how the HW is programmed it
isn't really future proof for a uAPI. Hence, fix this up for the new
uAPI and split up the va_len field of struct drm_nouveau_exec_push,
such that we keep 32bit for va_len and 32bit for flags.

For drm_nouveau_gem_pushbuf_push::length at least provide
NOUVEAU_GEM_PUSHBUF_NO_PREFETCH to indicate the bit shift.

While at it, fix up nv50_dma_push() as well, such that the caller
doesn't need to encode the NO_PREFETCH flag into the length parameter.

Signed-off-by: Danilo Krummrich <dakr@redhat.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230823181746.3446-1-dakr@redhat.com

drm/nouveau/fifo: add new channel classes

Exposes a bunch of the new features that became possible as a result
of the earlier commits. DRM will build on this in the future to add
support for features such as SCG ("async compute") and multi-device
rendering, as part of the work necessary to be able to write a half-
decent vulkan driver - finally.

For the moment, this just crudely ports DRM to the API changes.

- channel class interfaces now the same for all HW classes
- channel group class exposed (SCG)
- channel runqueue selector exposed (SCG)
- channel sub-device id control exposed (multi-device rendering)
- channel names in logging will reflect creating process, not fd owner
- explicit USERD allocation required by VOLTA_CHANNEL_GPFIFO_A and newer
- drm is smarter about determining the appropriate channel class to use

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fbcon: convert imageblit() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau: drop use of DRM_UDELAY

The DRM_UDELAY is a simple wrapper for udealy() and to be consistent
call udelay() direct like in may other places.
This avoids the need to pull in drm_os_linux.h when we later
drop drmP.h uses in nouveau.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: nouveau@lists.freedesktop.org
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/fifo/gv100: return work submission token in channel ctor args

The token will also contain runlist ID on Turing, so instead expose it as
an opaque value from NVKM so the client doesn't need to care.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/fifo/gv100: initial support

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gem: lookup VMAs for buffers referenced by pushbuf ioctl

We previously only did this for push buffers, but an upcoming patch will
need to attach fences to all VMAs to resolve another issue.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: separate constant-va tracking from nvkm vma structure

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: rename nouveau_drm.h to nouveau_drv.h

Fixes out-of-tree build issue where uapi/drm/nouveau_drm.h gets picked
up instead.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/nvif: simplify and tidy library interfaces

A variety of tweaks to the NVIF library interfaces, mostly ripping out
things that turned out to be not so useful.

- Removed refcounting from nvif_object, callers are expected to not be
stupid instead.
- nvif_client is directly reachable from anything derived from nvif_object,
removing the need for heuristics to locate it
- _new() versions of interfaces, that allocate memory for the object
they construct, have been removed. The vast majority of callers used
the embedded _init() interfaces.
- No longer storing constructor arguments (and the data returned from
nvkm) inside nvif_object, it's more or less unused and just wastes
memory.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: finalise nvkm namespace switch (no binary change)

The namespace of NVKM is being changed to nvkm_ instead of nouveau_,
which will be used for the DRM part of the driver. This is being
done in order to make it very clear as to what part of the driver a
given symbol belongs to, and as a minor step towards splitting the
DRM driver out to be able to stand on its own (for virt).

Because there's already a large amount of churn here anyway, this is
as good a time as any to also switch to NVIDIA's device and chipset
naming to ease collaboration with them.

A comparison of objdump disassemblies proves no code changes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: remove as much direct use of core headers as possible

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: port to nvif client/device/objects

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: initial pass at moving to struct nvif_device

This is an attempt at isolating some of the changes necessary to port
to NVIF in a separate commit.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm: Kill DRM_*MEMORYBARRIER

The real linux interfaces are soooo much easier on the eyes ...

Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nouveau: port all engines to new engine module format

This is a HUGE commit, but it's not nearly as bad as it looks - any problems
can be isolated to a particular chipset and engine combination. It was
simply too difficult to port each one at a time, the compat layers are
*already* ridiculous.

Most of the changes here are simply to the glue, the process for each of the
engine modules was to start with a standard skeleton and copy+paste the old
code into the appropriate places, fixing up variable names etc as needed.

v2: Marcin Slusarz <marcin.slusarz@gmail.com>
- fix find/replace bug in license header

v3: Ben Skeggs <bskeggs@redhat.com>
- bump indirect pushbuf size to 8KiB, 4KiB barely enough for userspace and
left no space for kernel's requirements during GEM pushbuf submission.
- fix duplicate assignments noticed by clang

v4: Marcin Slusarz <marcin.slusarz@gmail.com>
- add sparse annotations to nv04_fifo_pause/nv04_fifo_start
- use ioread32_native/iowrite32_native for fifo control registers

v5: Ben Skeggs <bskeggs@redhat.com>
- rebase on v3.6-rc4, modified to keep copy engine fix intact
- nv10/fence: unmap fence bo before destroying
- fixed fermi regression when using nvidia gr fuc
- fixed typo in supported dma_mask checking

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: restructure source tree, split core from drm implementation

Future work will be headed in the way of separating the policy supplied by
the nouveau drm module from the mechanisms provided by the driver core.

There will be a couple of major classes (subdev, engine) of driver modules
that have clearly defined tasks, and the further directory structure change
is to reflect this.

No code changes here whatsoever, aside from fixing up a couple of include
file pathnames.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

UAPI: (Scripted) Convert #include "..." to #include <path/...> in drivers/gpu/

Convert #include "..." to #include <path/...> in drivers/gpu/.

Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Dave Jones <davej@redhat.com>

UAPI: (Scripted) Remove redundant DRM UAPI header #inclusions from drivers/gpu/.

Remove redundant DRM UAPI header #inclusions from drivers/gpu/.

Remove redundant #inclusions of core DRM UAPI headers (drm.h, drm_mode.h and
drm_sarea.h). They are now #included via drmP.h and drm_crtc.h via a preceding
patch.

Without this patch and the patch to make include the UAPI headers from the core
headers, after the UAPI split, the DRM C sources cannot find these UAPI headers
because the DRM code relies on specific -I flags to make #include "..." work
on headers in include/drm/ - but that does not work after the UAPI split without
adding more -I flags.

Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Dave Jones <davej@redhat.com>

drm/nouveau: remove m2mf creation on userspace channels

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Fix pushbufs over the 4GB mark.

Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Tested-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nv50-nvc0: lookup pushbuf virtual address on dma_push

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: convert some bo.offset use to vma.offset

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fix allocation of notifier object

Commit 73412c3854c877e5f37ad944ee8977addde4d35a ("drm/nouveau: allocate
kernel's notifier object at end of block") intended to align end of
notifier block to page boundary, but start of block was miscalculated
to be off by -16 bytes. Fix it.

Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Cc: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: allocate kernel's notifier object at end of block

The nv30/nv40 3d driver is about to start using DMA_FENCE from the 3D
object which, it turns out, doesn't like its DMA object to not be
aligned to a 4KiB boundary.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/nv50: simplify bo moves now that they're all through the vm

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nvc0: fix channel dma init paths

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: implicitly insert non-DMA objects into RAMHT

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Minor refactoring/cleanup of the fence code.

Mainly to make room for inter-channel sync.

Signed-off-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nv50: prevent (IB_PUT == IB_GET) for occurring unless idle

Should fix a DMA race condition I've never seen myself, but could be
the culprit in some random hangs that have been reported.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: remove nouveau_gpuobj_ref completely, replace with sanity

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: remove second map of notifier bo

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: remove left-over !DRIVER_MODESET paths

It's far preferable to have the driver do nothing at all for "nomodeset".

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nv50: add a memory barrier to pushbuf submission

- This is useful for vram pushbuffers that are write combined.
- pre-nv50 has one too (in WRITE_PUT).

Signed-off-by: Maarten Maathuis <madman2003@gmail.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: new gem pushbuf interface, bump to 0.0.16

This commit breaks the userspace interface, and requires a new libdrm for
nouveau to operate again.

The multiple GEM_PUSHBUF ioctls that were present in 0.0.15 for
compatibility purposes are now gone, and replaced with the new ioctl which
allows for multiple push buffers to be submitted (necessary for hw index
buffers in the nv50 3d driver) and relocations to be applied on any buffer.

A number of other ioctls (CARD_INIT, GEM_PIN, GEM_UNPIN) that were needed
for userspace modesetting have also been removed.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Francisco Jerez <currojerez@riseup.net>

drm/nv50: make pushbuf dma object cover entire vm

This allows us to submit push buffers from any memtype to the hardware.
We'll need this ability for VRAM index buffers at some point.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nv50: switch to indirect push buffer controls

PFIFO on G80 and up has a new mode where the main ring buffer is simply a
ring of pointers to indirect buffers containing the actual command/data
packets. In order to be able to implement index buffers in the 3D driver
we need to be able to submit data-only push buffers right after the cmd
packet header, which is only possible using the new command submission
method.

This commit doesn't make it possible to implement index buffers yet, some
userspace interface changes will be required, but it does allow for
testing/debugging of the hardware-side support in the meantime.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fix a race condition in nouveau_dma_wait()

Can be triggered easily on certain cards (NV46 and NV50 of mine) by
running "dmesg", the DRM's channel will lockup.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: initialise DMA tracking parameters earlier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: use dma.max rather than pushbuf size for checking GET validity

Some upcoming G80 DMA changes will depend on this, but it's split out for
bisectibility just in case it causes some unexpected issues.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Allocate a per-channel instance of NV_SW.

It will be useful for various synchronization purposes, mostly stolen
from "[PATCH] drm/nv50: synchronize user channel after buffer object
move on kernel channel" by Maarten Maathuis.

Signed-off-by: Francisco Jerez <currojerez@riseup.net>

drm/nouveau: initialise DMA tracking parameters earlier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: use dma.max rather than pushbuf size for checking GET validity

Some upcoming G80 DMA changes will depend on this, but it's split out for
bisectibility just in case it causes some unexpected issues.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Allocate a per-channel instance of NV_SW.

It will be useful for various synchronization purposes, mostly stolen
from "[PATCH] drm/nv50: synchronize user channel after buffer object
move on kernel channel" by Maarten Maathuis.

Signed-off-by: Francisco Jerez <currojerez@riseup.net>

drm/nouveau: Add DRM driver for NVIDIA GPUs

This adds a drm/kms staging non-API stable driver for GPUs from NVIDIA.

This driver is a KMS-based driver and requires a compatible nouveau
userspace libdrm and nouveau X.org driver.

This driver requires firmware files not available in this kernel tree,
interested parties can find them via the nouveau project git archive.

This driver is reverse engineered, and is in no way supported by nVidia.

Support for nearly the complete range of nvidia hw from nv04->g80 (nv50)
is available, and the kms driver should support driving nearly all
output types (displayport is under development still) along with supporting
suspend/resume.

This work is all from the upstream nouveau project found at
nouveau.freedesktop.org.

The original authors list from nouveau git tree is:
Anssi Hannula <anssi.hannula@iki.fi>
Ben Skeggs <bskeggs@redhat.com>
Francisco Jerez <currojerez@riseup.net>
Maarten Maathuis <madman2003@gmail.com>
Marcin Kościelnicki <koriakin@0x04.net>
Matthew Garrett <mjg@redhat.com>
Matt Parnell <mparnell@gmail.com>
Patrice Mandin <patmandin@gmail.com>
Pekka Paalanen <pq@iki.fi>
Xavier Chantry <shiningxc@gmail.com>
along with project founder Stephane Marchesin <marchesin@icps.u-strasbg.fr>

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>