drm/msm: Add param for the highest bank bit

This parameter is programmed by the kernel and influences the tiling
layout of images. Exposing it to userspace will allow it to tile/untile
images correctly without guessing what value the kernel programmed, and
allow us to change it in the future without breaking userspace.

Signed-off-by: Connor Abbott <cwabbott0@gmail.com>
Patchwork: https://patchwork.freedesktop.org/patch/571181/
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm/gem: Add metadata

The EXT_external_objects extension is a bit awkward as it doesn't pass
explicit modifiers, leaving the importer to guess with incomplete
information. In the case of vk (turnip) exporting and gl (freedreno)
importing, the "OPTIMAL_TILING_EXT" layout depends on VkImageCreateInfo
flags (among other things), which the importer does not know. Which
unfortunately leaves us with the need for a metadata back-channel.

The contents of the metadata are defined by userspace. The
EXT_external_objects extension is only required to work between
compatible versions of gl and vk drivers, as defined by device and
driver UUIDs.

v2: add missing metadata kfree
v3: Rework to move copy_from/to_user out from under gem obj lock
to avoid angering lockdep about deadlocks against fs-reclaim

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/566157/

drm/msm: Rename drm_msm_gem_submit_reloc::or in C++ code

Clashes with C++ `or` keyword

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Patchwork: https://patchwork.freedesktop.org/patch/528751/
Link: https://lore.kernel.org/r/20230326163813.535762-1-robdclark@gmail.com
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>

drm/msm: Add wait-boost support

Add a way for various userspace waits to signal urgency.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/525817/
Link: https://lore.kernel.org/r/20230308155322.344664-14-robdclark@gmail.com

drm/msm: Add MSM_SUBMIT_BO_NO_IMPLICIT

In cases where implicit sync is used, it is still useful (for things
like sub-allocation, etc) to allow userspace to opt-out of implicit
sync on per-BO basis.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Patchwork: https://patchwork.freedesktop.org/patch/514216/
Link: https://lore.kernel.org/r/20221206192123.661448-1-robdclark@gmail.com

drm/msm: Add MSM_INFO_GET_FLAGS

In some cases crosvm needs a way to query the cache flags to communicate
them to the guest kernel for guest userspace mapping.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Patchwork: https://patchwork.freedesktop.org/patch/504453/
Link: https://lore.kernel.org/r/20220923173307.2429872-1-robdclark@gmail.com
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>

drm/msm: Add a way for userspace to allocate GPU iova

The motivation at this point is mainly native userspace mesa driver in a
VM guest. The one remaining synchronous "hotpath" is buffer allocation,
because guest needs to wait to know the bo's iova before it can start
emitting cmdstream/state that references the new bo. By allocating the
iova in the guest userspace, we no longer need to wait for a response
from the host, but can just rely on the allocation request being
processed before the cmdstream submission. Allocation failures (OoM,
etc) would just be treated as context-lost (ie. GL_GUILTY_CONTEXT_RESET)
or subsequent allocations (or readpix, etc) can raise GL_OUT_OF_MEMORY.

v2: Fix inuse check
v3: Change mismatched iova case to -EBUSY

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Reviewed-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Link: https://lore.kernel.org/r/20220411215849.297838-11-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Add a way to override processes comm/cmdline

In the cause of using the GPU via virtgpu, the host side process is
really a sort of proxy, and not terribly interesting from the PoV of
crash/fault logging. Add a way to override these per process so that
we can see the guest process's name.

v2: Handle kmalloc failure, add comment to explain kstrdup returns
NULL if passed NULL [Dan Carpenter]

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20220317165144.222101-4-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Add support for pointer params

The 64b value field is already suffient to hold a pointer instead of
immediate, but we also need a length field.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20220317165144.222101-2-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Add MSM_SUBMIT_FENCE_SN_IN

Add a way for userspace to specify the sequence number fence used to
track completion of the submit. As the seqno fence is simply an
incrementing counter which is local to the submitqueue, it is easy for
userspace to know the next value.

This is useful for native userspace drivers in a vm guest, as the guest
to host roundtrip can have high latency. Assigning the fence seqno in
the guest userspace allows the guest to continue without waiting for
response from the host.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20220224222321.60653-1-robdclark@gmail.com

drm/msm: Add SYSPROF param (v2)

Add a SYSPROF param for system profiling tools like Mesa's pps-producer
(perfetto) to control behavior related to system-wide performance
counter collection. In particular, for profiling, one wants to ensure
that GPU context switches do not effect perfcounter state, and might
want to suppress suspend (which would cause counters to lose state).

v2: Swap the order in msm_file_private_set_sysprof() [sboyd] and
initialize the sysprof_active refcount to one (because the under/
overflow checking in refcount_t doesn't expect a 0->1 transition)
meaning that values greater than 1 means sysprof is active.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20220304005317.776110-4-robdclark@gmail.com

drm/msm: Add SET_PARAM ioctl

It was always expected to have a use for this some day, so we left a
placeholder. Now we do. (And I expect another use in the not too
distant future when we start allowing userspace to allocate GPU iova.)

Signed-off-by: Rob Clark <robdclark@chromium.org>
Link: https://lore.kernel.org/r/20220304005317.776110-3-robdclark@gmail.com

drm/msm: Utilize gpu scheduler priorities

The drm/scheduler provides additional prioritization on top of that
provided by however many number of ringbuffers (each with their own
priority level) is supported on a given generation. Expose the
additional levels of priority to userspace and map the userspace
priority back to ring (first level of priority) and schedular priority
(additional priority levels within the ring).

Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Christian König <christian.koenig@amd.com>
Link: https://lore.kernel.org/r/20210728010632.2633470-13-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: deprecate MSM_BO_UNCACHED (map as writecombine instead)

There shouldn't be any reason to ever use uncached over writecombine,
so just use writecombine for MSM_BO_UNCACHED.

Note: userspace never used MSM_BO_UNCACHED anyway

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Acked-by: Jordan Crouse <jordan@cosmicpenguin.net>
Link: https://lore.kernel.org/r/20210423190833.25319-6-jonathan@marek.ca
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: add MSM_BO_CACHED_COHERENT

Add a new cache mode for creating coherent host-cached BOs.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Reviewed-by: Jordan Crouse <jcrouse@codeaurora.org>
Link: https://lore.kernel.org/r/20210423190833.25319-5-jonathan@marek.ca
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: Add param for userspace to query suspend count

Performance counts, and ALWAYS_ON counters used for capturing GPU
timestamps, lose their state across suspend/resume cycles. Userspace
tooling for performance monitoring needs to be aware of this. For
example, after a suspend userspace needs to recalibrate it's offset
between CPU and GPU time.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Jordan Crouse <jordan@cosmicpenguin.net>
Link: https://lore.kernel.org/r/20210325012358.1759770-3-robdclark@gmail.com
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm: msm_drm.h: delete duplicated words in comments

Drop the doubled word "to" in comments.

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: dri-devel@lists.freedesktop.org
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20200715052349.23319-6-rdunlap@infradead.org

drm/msm: Add syncobj support.

This

1) Enables core DRM syncobj support.
2) Adds options to the submission ioctl to wait/signal syncobjs.

Just like the wait fence fd, this does inline waits. Using the
scheduler would be nice but I believe it is out of scope for
this work.

Support for timeline syncobjs is implemented and the interface
is ready for it, but I'm not enabling it yet until there is
some code for turnip to use it.

The reset is mostly in there because in the presence of waiting
and signalling the same semaphores, resetting them after
signalling can become very annoying.

v2:
- Fixed style issues
- Removed a cleanup issue in a failure case
- Moved to a copy_from_user per syncobj

v3:
- Fixed a missing declaration introduced in v2
- Reworked to use ERR_PTR/PTR_ERR
- Simplified failure gotos.

Used by: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2769

Signed-off-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm/gpu: Add submit queue queries

Add the capability to query information from a submit queue.
The first available parameter is for querying the number of GPU faults
(hangs) that can be attributed to the queue.

This is useful for implementing context robustness. A user context can
regularly query the number of faults to see if it is responsible for any
and if so it can invalidate itself.

This is also helpful for testing by confirming to the user driver if a
particular command stream caused a fault (or not as the case may be).

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@chromium.org>

drm/msm: add param to retrieve # of GPU faults (global)

For KHR_robustness, userspace wants to know two things, the count of GPU
faults globally, and the count of faults attributed to a given context.
This patch providees the former, and the next patch provides the latter.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Jordan Crouse <jcrouse@codeaurora.org>

drm/msm/gpu: add per-process pagetables param

For now it always returns '0' (false), but once the iommu work is in
place to enable per-process pagetables we can update the value returned.

Userspace needs to know this to make an informed decision about exposing
KHR_robustness.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Jordan Crouse <jcrouse@codeaurora.org>

drm/msm: add uapi to get/set debug name

Add UAPI to get/set GEM objects' debug name.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: rework GEM_INFO ioctl

Prep work to add a way to get/set the GEM objects debug name.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm/gpu: add submit flag to hint which buffers should be dumped

To lower CPU overhead, future userspace will be switching to pinning
iova and avoiding the use of relocs, and only include cmds table entries
for IB1 level cmdstream (but not IB2 or state-groups).

This leaves the kernel unsure what to dump for rd/hangrd cmdstream
dumping. So add a MSM_SUBMIT_BO_DUMP flag so userspace can indicate
buffers that contain cmdstream (or are otherwise important to dump).

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: add sudo flag to submit ioctl

This flags cause cmdstream to be executed from the ringbuffer (RB)
instead of IB1. Normally not something you'd ever want to do, but
it is super useful for firmware debugging.

Hidden behind CAP_SYS_RAWIO and a default=n kconfig option which
depends on EXPERT (and has a suitably scary warning), to prevent
it from being used on accident.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Add a parameter query for the number of ringbuffers

In order to manage ringbuffer priority to its fullest userspace
should know how many ringbuffers it has to work with. Add a
parameter to return the number of active rings.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Support multiple ringbuffers

Add the infrastructure to support the idea of multiple ringbuffers.
Assign each ringbuffer an id and use that as an index for the various
ring specific operations.

The biggest delta is to support legacy fences. Each fence gets its own
sequence number but the legacy functions expect to use a unique integer.
To handle this we return a unique identifier for each submission but
map it to a specific ring/sequence under the covers. Newer users use
a dma_fence pointer anyway so they don't care about the actual sequence
ID or ring.

The actual mechanics for multiple ringbuffers are very target specific
so this code just allows for the possibility but still only defines
one ringbuffer for each target family.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Add per-instance submit queues

Currently the behavior of a command stream is provided by the user
application during submission and the application is expected to internally
maintain the settings for each 'context' or 'rendering queue' and specify
the correct ones.

This works okay for simple cases but as applications become more
complex we will want to set context specific flags and do various
permission checks to allow certain contexts to enable additional
privileges.

Add kernel-side submit queues to be analogous to 'contexts' or
'rendering queues' on the application side. Each file descriptor
instance will maintain its own list of queues. Queues cannot be
shared between file descriptors.

For backwards compatibility context id '0' is defined as a default
context specifying no priority and no special flags. This is
intended to be the usual configuration for 99% of applications so
that a garden variety application can function correctly without
creating a queue. Only those applications requiring the specific
benefit of different queues need create one.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Remove __user from __u64 data types

__user should be used to identify user pointers and not __u64
variables containing pointers.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Add hint to DRM_IOCTL_MSM_GEM_INFO to return an object IOVA

Modify the 'pad' member of struct drm_msm_gem_info to 'flags'. If the
user sets 'flags' to non-zero it means that they want a IOVA for the
GEM object instead of a mmap() offset. Return the iova in the 'offset'
member.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
[robclark: s/hint/flags in commit msg]
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Remove DRM_MSM_NUM_IOCTLS

The ioctl array is sparsely populated but the compiler will make sure
that it is sufficiently sized for all the values that we have so we
can safely use ARRAY_SIZE() instead of having a constantly changing
#define in the uapi header.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: Add MSM_PARAM_GMEM_BASE

User space needs to know where the GMEM whole starts so that they
can set up the addressing correctly.

Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: update uapi header license

The same file in libdrm is, as is the tradition with the rest of libdrm,
etc, using an MIT license. To avoid complications in the future with
sync'ing the uapi header to libdrm, lets fix the license mismatch now
before there are any non-trivial commits from someone other than myself.

Cc: Emil Velikov <emil.l.velikov@gmail.com>
Cc: Gabriel Laskar <gabriel@lse.epita.fr>
Cc: Mikko Rapeli <mikko.rapeli@iki.fi>
Signed-off-by: Rob Clark <robdclark@gmail.com>
Acked-by: Emil Velikov <emil.l.velikov@gmail.com>

drm/msm: submit support for out-fences

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: submit support for in-fences

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: extend the submit ioctl to pass in flags

We'll want to be able to pass in flags, such as asking for explicit
fencing, and possibly other things down the road. Fortunately we
don't need a full 32b for the pipe-id. So use the upper 16 bits
for flags (which could be extended or reduced later if needed, so
start adding flags from the high bits).

Since anything with the upper bits set would not be a valid pipe-id,
an old userspace would not set any of the upper bits, and an old
kernel would reject it as an invalid pipe-id.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: add madvise ioctl

Doesn't do anything too interesting until we wire up shrinker. Pretty
much lifted from i915.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: add extern C guard for the UAPI header

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Acked-by: Rob Clark <robdclark@gmail.com> (over irc)

drm/msm: add timestamp param

We need this for GL_TIMESTAMP queries.

Note: currently only supported on a4xx.. a3xx doesn't have this
always-on counter. I think we could emulate it with the one CP
counter that is available, but for now it is of limited usefulness
on a3xx (since we can't seem to do time-elapsed queries in any sane
way with the existing firmware on a3xx, and if you are trying to do
profiling on a tiler you want time-elapsed). We can add that later
if it becomes useful.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: add max-freq gpu param to uapi

We need this in userspace for interpreting some of the perf ctrs.

Note possibly not quite sufficient if we had some frequency mgmt
approach other than race-to-idle. Not really sure what the best
thing to do if we did. Although displaying results as a percentage
of max frequence seems sensible(ish) if we did.

Signed-off-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/msm: trivial whitespace fix

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm: fix inclusion of drm.h in msm_drm.h

Using `#include "drm.h"` instead of `#include <drm/drm.h>` allow drm
headers to be moved in another directory without changes, like for the
libdrm imports.

Signed-off-by: Gabriel Laskar <gabriel@lse.epita.fr>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
CC: Emil Velikov <emil.l.velikov@gmail.com>
CC: Mikko Rapeli <mikko.rapeli@iki.fi>

drm/msm: use __s32, __s64, __u32 and __u64 from linux/types.h for uabi

Fixes userspace compilation errors like:

error: unknown type name ‘uint32_t’

Signed-off-by: Mikko Rapeli <mikko.rapeli@iki.fi>
Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: validate flags, etc

After reading a nice article on LWN[1], I went back and double checked
my handling of invalid-input checking. Turns out there were a couple
places I had missed.

Since the driver is fairly young, and the devices it supports are really
only just barely usable for basic stuff (serial console) with an
upstream kernel, I think we should fix this now and revert specific
parts of this patch later in the unlikely event that a regression is
reported.

[1] https://lwn.net/Articles/588444/

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: add chip-id param

Some of the w/a or different behavior of userspace blob driver seem to
be keyed to gpu patch revision, rather than gpu-id. So expose the full
chip-id to userspace so it can DTRT.

Signed-off-by: Rob Clark <robdclark@gmail.com>

drm/msm: add a3xx gpu support

Add initial support for a3xx 3d core.

So far, with hardware that I've seen to date, we can have:
+ zero, one, or two z180 2d cores
+ a3xx or a2xx 3d core, which share a common CP (the firmware
for the CP seems to implement some different PM4 packet types
but the basics of cmdstream submission are the same)

Which means that the eventual complete "class" hierarchy, once
support for all past and present hw is in place, becomes:
+ msm_gpu
+ adreno_gpu
+ a3xx_gpu
+ a2xx_gpu
+ z180_gpu

This commit splits out the parts that will eventually be common
between a2xx/a3xx into adreno_gpu, and the parts that are even
common to z180 into msm_gpu.

Note that there is no cmdstream validation required. All memory access
from the GPU is via IOMMU/MMU. So as long as you don't map silly things
to the GPU, there isn't much damage that the GPU can do.

Signed-off-by: Rob Clark <robdclark@gmail.com>