drm/nouveau/gr/gf100: Remove second semicolon

There is a statement with two semicolons. Remove the second one, it
is redundant.

Signed-off-by: Colin Ian King <colin.i.king@gmail.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20240315090930.2429958-1-colin.i.king@gmail.com

drm/nouveau/gr/gf100: don't misuse kernel-doc comments

Change kernel-doc "/**" comments to common "/*" comments to prevent
kernel-doc warnings:

gf100.c:1044: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst
* Wait until GR goes idle. GR is considered idle if it is disabled by the
gf100.c:1044: warning: missing initial short description on line:
* Wait until GR goes idle. GR is considered idle if it is disabled by the

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Karol Herbst <kherbst@redhat.com>
Cc: Lyude Paul <lyude@redhat.com>
Cc: Danilo Krummrich <dakr@redhat.com>
Cc: nouveau@lists.freedesktop.org
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Maxime Ripard <mripard@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20231231233633.6596-3-rdunlap@infradead.org

drm/nouveau/gr/gf100-: lose contents of global ctxbufs across suspend

Some of these buffers are quite large, and there's no need to preserve
them across suspend.

Mark the contents as lost to speedup suspend/resume.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Acked-by: Danilo Krummrich <me@dakr.org>
Signed-off-by: Lyude Paul <lyude@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230919220442.202488-4-lyude@redhat.com

drm/nouveau/fifo/ga100-: add per-runlist nonstall intr handling

GSP-RM will enforce this, so implement on HW too so we can share code.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230525003106.3853741-8-skeggsb@gmail.com

drm/nouveau/fifo: remove left-over references to nvkm_fifo_chan

This was renamed to nvkm_chan in the host rework.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230525003106.3853741-5-skeggsb@gmail.com

drm/nouveau/gr/ga102: initial support

v2:
- whitespace

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Gourav Samaiya <gsamaiya@nvidia.com>

drm/nouveau/gr/tu102-: fix support for sw_bundle64_init

We weren't sending the high bits, though they're zero currently anyway.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/tu102-: use sw_veid_bundle_init from firmware

NVIDIA provided this on Turing, but we kept using the hardcoded version
from Volta (where they didn't).

Switch to the firmware version prior to Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gv100-: move init_419bd8() after sw_ctx load

Match RM.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gv100-: port smid mapping code from nvgpu

Essentially ripped verbatim from NVGPU, comments and all, and adapted to
nvkm's structs and style.

- maybe fixes an nvgpu bug though, a small tweak was needed to match RM

v2:
- remove unnecessary WARN_ON

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf117-: make ppc_nr[gpc] accurate

We're going to be pulling in a chunk of code from NVGPU to fixup our
SMID mappings on Volta and above, which depends on ppc_nr[gpc]
reflecting the actual number of PPCs present, not the maximum number.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: switch to newer style interrupt handler

Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: move some init to init_exception2()

Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: move some init to init_rop_exceptions()

Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: move reset during golden ctx init to fecs_reset()

Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: gpfifo_ctl zero before init

Match RM.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: call FECS HALT_PIPE method before RC reset

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: call FECS WFI_GOLDEN_SAVE method

This won't work on Ampere, and, it's questionable whether we should have
been using our FW's method of storing the golden context image with NV's
firmware to begin with.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: stop using NV_PGRAPH_FECS_CTXSW_MAILBOX_CLEAR

This doesn't work on Ampere for some reason, switch to directly modifying
NV_PGRAPH_FECS_CTXSW_MAILBOX instead.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: make global attrib_cb actually global

This was thought to be per-channel initially - it's not. The backing
pages for the VMM mappings are shared for all channels.

- switches to more straight-forward patch interfaces
- prepares for sub-context support
- this is saving a *sizeable* amount of vram

v2:
- whitespace

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: move misc context patching out of attrib_cb funcs

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: make global bundle_cb actually global

This was thought to be per-channel initially - it's not. The backing
pages for the VMM mappings are shared for all channels.

- switches to more straight-forward patch interfaces
- prepares for sub-context support

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: make global pagepool actually global

This was thought to be per-channel initially - it's not. The backing
pages for the VMM mappings are shared for all channels.

- switches to more straight-forward patch interfaces
- prepares for sub-context support

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: generate golden context during first object alloc

Needed for GV100 (and only GV100 for some reason) for WFI_GOLDEN_SAVE.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: move some code around to make next commits nicer

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fifo: expose function to read engine ctxsw status

Needed to support Ampere differences in gr/gf100-:

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/ltc: split color vs depth/stencil zbc counts

These differ on Ampere.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/flcn: rework falcon reset

Mostly preparation to fit in Ampere changes, but should result in reset
sequences a lot closer to RM's, and perhaps help out with the issues we
sometimes see reported in this area.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fifo: add new channel lookup interfaces

- supports per-runlist CHIDs
- channel group lock held across reference, rather than global lock

v2:
- remove unnecessary parenthesis

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: Clean up some inconsistent indenting

Eliminate the follow smatch warning:

drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.c:1925
gf100_gr_oneinit_tiles() warn: inconsistent indenting.

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Signed-off-by: Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Signed-off-by: Lyude Paul <lyude@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220505081345.89762-1-jiapeng.chong@linux.alibaba.com

drm/nouveau/gr: switch to instanced constructor

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gm200-: explicitly handle nofw

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: make some functions static

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gp107,gp108: implement workaround for HW hanging during init

Certain boards with GP107/GP108 chipsets hang (often, but randomly) for
unknown reasons during GR initialisation.

The first tell-tale symptom of this issue is:

nouveau 0000:01:00.0: bus: MMIO read of 00000000 FAULT at 409800 [ TIMEOUT ]

appearing in dmesg, likely followed by many other failures being logged.

Karol found this WAR for the issue a while back, but efforts to isolate
the root cause and proper fix have not yielded success so far. I've
modified the original patch to include a few more details, limit it to
GP107/GP108 by default, and added a config option to override this choice.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>

drm/nouveau/gr/tu10x: initial support

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/acr: implement new subdev to replace "secure boot"

ACR is responsible for managing the firmware for LS (Low Secure) falcons,
this was previously handled in the driver by SECBOOT.

This rewrite started from some test code that attempted to replicate the
procedure RM uses in order to debug early Turing ACR firmwares that were
provided by NVIDIA for development.

Compared with SECBOOT, the code is structured into more individual steps,
with the aim of making the process easier to follow/debug, whilst making
it possible to support newer firmware versions that may have a different
binary format or API interface.

The HS (High Secure) binary(s) are now booted earlier in device init, to
match the behaviour of RM, whereas SECBOOT would delay this until we try
to boot the first LS falcon.

There's also additional debugging features available, with the intention
of making it easier to solve issues during FW/HW bring-up in the future.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/flcn: specify FBIF offset from subdev

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: initialise SW state for falcon from constructor

This will allow us to register the falcon with ACR, and further customise
its behaviour by providing the nvkm_falcon_func structure directly.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: select implementation based on available FW

This will allow for further customisation of the subdev depending on what
firmware is available.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: move fecs/gpccs ucode into their substructures

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: drop fuc_ prefix on sw init

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: use nvkm_blob structure for fecs/gpccs fw

It serves the exact same purpose.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gv100-: modify gr init to match newer version of RM

Will be used as a basis for implementing changes needed for Turing.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: remove dtor

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/core: pass subdev into nvkm_firmware_get, rather than device

It'd be nice to have FW loading debug messages to appear for the relevant
subsystem, when enabled.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: expose method to determine current context

MMU will need access to this info.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: expose fecs methods for pausing ctxsw

MMU will need access to these.

v2. Apply fix from Rhys Kidd to send correct FECS method for STOP_CTXSW.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: store fecs/gpccs falcon pointers in substructures

Future changes will want to add some additional things here, keep them
grouped together.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: move fecs bind_pointer into a function

Makes the code somewhat less magic.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: remove some unnecessary reg writes

This is already done during golden context creation.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: move fecs elpg setup into functions

Makes the code somewhat less magic.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: move fecs discover_pm_image_size into a function

Makes the code somewhat less magic.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: move fecs discover_zcull_image_size into a function

Makes the code somewhat less magic.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: move fecs discover_image_size into a function

Makes the code somewhat less magic.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: move fecs set_watchdog_timeout method into a function

Makes the code somewhat less magic.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: insert some WFIs during gr init

Inserted wait-for-gr-idle in the places it seems that RM does it, seems
to prevent some random mmio timeouts on Quadro GV100.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gv100: initial support

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gp102-: setup stencil zbc

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gp100-: use correct registers for zbc colour/depth setup

These were missed the first time around due to the driver version I traced
using the older registers still.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gp100-: fix attrib cb setup

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-gm10x: update register lists

There are differences on GM200 and newer too, but we can't fix them there
as they come from firmware packages.

A request has been made to NVIDIA to release updated firmware.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: calculate and use sm mapping table

There's a number of places that require this data, so let's separate out
the calculations to ensure they remain consistent.

This is incorrect for GM200 and newer, but will produce the same results
as we did before.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: port zcull tile mapping calculations from NVGPU

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: port tile mapping calculations from NVGPU

There's also a couple of hardcoded tables for a couple of very specific
configurations that NVGPU's algorithm didn't work for.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise trap_mp

Required to support Volta.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise dist_skip_table + improve algorithm

The algorithm for GM200 and newer matches RM for all the boards I have, but
I don't have enough data to try and figure something out for earlier boards,
so these will still write zeroes to the table as we did before.

The code in NVGPU isn't helpful here, it appears to handle specific cases.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise alpha_beta_tables + improve algorithms

I haven't yet been able to find a fully programatic way of calculating the
same mapping as NVIDIA for GF100-GF119, so the algorithm partially depends
on data tables for specific configurations.

I couldn't find traces for every possibility, so the algorithm will switch
to a mapping similar to what GK104-GM10x use if it encounters one. We did
the wrong thing before anyway, so shouldn't matter too much.

The algorithm used in the GK104 implementation was ported from NVGPU.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise sm_id/tpc_nr

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_400054

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: apply be exception fixes from traces

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_shader_exceptions

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_504430

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_tex_hww_esr

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_ppc_exceptions

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_419c9c + apply fixes from traces

Deliberately removed from non-GP100, as RM doesn't touch it.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_419eb4 + apply fixes from traces

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_419cc0 + apply fixes from traces

Pulled some init out of main per-GPC/TPC loops to match RM.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_sked_hww_esr

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_40601c

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_ds_hww_esr_2

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_fecs_exceptions + apply fixes from traces

The value for GF100 has changed here, but it matches RM now.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100: write 0x400124 during init

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_swdx_pes_mask

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: implement another chunk of bios-provided init

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_rop_active_fbps

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_num_active_ltcs

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_zcull

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_vsc_stream_master

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_bios

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: support clkgate_pack everywhere

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise r405a14

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: support firmware-provided sw_nonctx everywhere

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise init_gpc_mmu + apply fixes from traces

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: don't prevent module load if firmware missing

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: use new interfaces for vmm operations

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/fifo: initialise vmm with new interfaces

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/core/memory: add reference counting

We need to be able to prevent memory from being freed while it's still
mapped in a GPU's address-space.

Will be used by upcoming MMU changes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/core/memory: change map interface to support upcoming mmu changes

Map flags (access, kind, etc) are currently defined in either the VMA,
or the memory object, which turns out to not be ideal for things like
suballocated buffers, etc.

These will become per-map flags instead, so we need to support passing
these arguments in nvkm_memory_map().

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-gk208: copy big page size setting from fb

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-gk208: make use of init_gpc_mmu() hook to share setup

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/secboot: allow to boot multiple falcons

Change the secboot and msgqueue interfaces to take a mask of falcons to
reset instead of a single falcon. The GP10B firmware interface requires
FECS and GPCCS to be booted in a single firmware command.

For firmwares that only support single falcon boot, it is trivial to
loop over the mask and boot each falcons individually.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon: delay construction of falcons to oneinit()

Reading registers at device construction time can be harmful, as there
is no guarantee the underlying engine will be up, or in its runtime
configuration. Defer register reading to the oneinit() hook and update
users accordingly.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: implement chsw_load() method

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: fix ccache error logging

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100: instantiate and reserve GR falcons

Create instances for the FECS and GPCCS falcons and use the init() and
fini() hooks to reserve them for as long as GR controls them.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100: split gf100_gr_init_ctxctl()

gf100_gr_init_ctxctl() is basically two different functions (one for
use of internal firmware, the other for use of external firmware), but
its current layout makes it look more complex than it is. Split it to
better reflect that fact.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: handle GPC/TPC/MPC trap

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: FECS intr handling is not relevant on proprietary ucode

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: properly ack all FECS error interrupts

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: fallback to legacy paths during firmware lookup

Look for firmware files using the legacy ("nouveau/nvxx_fucxxxx") path
if they cannot be found in the new, "official" path. User setups were
broken by the switch, which is bad.

There are only 4 firmware files we may want to look up that way, so
hardcode them into the lookup function. All new firmware files should
use the standard "nvidia/<chip>/gr/" path.

Fixes: 8539b37acef7 ("drm/nouveau/gr: use NVIDIA-provided external firmwares")
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Cc: stable@vger.kernel.org

drm/nouveau: mark symbols static where possible

We get a few warnings when building kernel with W=1:
drivers/gpu/drm/nouveau/nvkm/subdev/bios/fan.c:29:1: warning: no previous prototype for 'nvbios_fan_table' [-Wmissing-prototypes]
drivers/gpu/drm/nouveau/nvkm/subdev/bios/fan.c:56:1: warning: no previous prototype for 'nvbios_fan_entry' [-Wmissing-prototypes]
drivers/gpu/drm/nouveau/nvkm/subdev/clk/gt215.c:184:1: warning: no previous prototype for 'gt215_clk_info' [-Wmissing-prototypes]
drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramgt215.c:99:1: warning: no previous prototype for 'gt215_link_train_calc' [-Wmissing-prototypes]
drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramgt215.c:153:1: warning: no previous prototype for 'gt215_link_train' [-Wmissing-prototypes]
drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramgt215.c:271:1: warning: no previous prototype for 'gt215_link_train_init' [-Wmissing-prototypes]
....

In fact, both functions are only used in the file in which they are
declared and don't need a declaration, but can be made static.
So this patch marks these functions with 'static'.

Signed-off-by: Baoyou Xie <baoyou.xie@linaro.org>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100: handle secure boot errors

Handle and propagate secure boot errors. Failure to do so results in
Nouveau incorrectly believing init has succeeded and a completely
black display during boot. If we propagate the error, GR init will fail
and the user will at least have a working display.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/mc: take nvkm_device as argument to public functions

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: update sm error decoding from gk20a nvgpu headers

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Cc: stable@vger.kernel.org

drm/nouveau/fb/gf100-: allocate mmu debug buffers

Later chipsets require setting this up both in FB and GR, so let's just
move the allocation to FB.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: abstract fetching rop count

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: rename magic_not_rop_nr to screen_tile_row_offset

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/core: remove pmc_enable argument from subdev ctor

These are now specified directly in the MC subdev.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100: select a stream master to fixup tfb offset queries

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Cc: stable@vger.kernel.org

drm/nouveau/gr/gf100: add support for securely-managed falcons

Start securely-managed falcons using secboot functions since the process
for them is different from just writing CPUCTL.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100: load firmware in outer function

The firmwares required by GR may vary from chip to chip, especially with
the introduction of secure boot and NVIDIA-provided firmwares. Move the
firmware loading outside of gf100_gr_ctor so other chips may still call
it while managing their firmwares themselves.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gk20a: move firmware bundle release to gf100

Some members of gf100_gr were freed by the gk20a driver. That's not
where it should be done - free them in gf100 so other chips that use
NVIDIA-provided firmware free these structures properly.

This also removes the need for a GK20A-specific destructor.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100: use the nvkm_firmware functions

Use the nvkm_firmware_* functions when loading external firmware to
avoid duplicate code.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100: provide a bit more info for various errors

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: subclass nvkm_object to store channel pointer

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/nvif: modify nvif_unvers/nvif_unpack macros to be more obvious

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/nvif: split out fermi interface definitions

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: fix oops when calling zbc methods

Somehow missed these two when removing dodgy void casts during the
rework.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf117-: assume no PPC if NV_PGRAPH_GPC_GPM_PD_PES_TPC_ID_MASK is zero

fdo#92761

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: document mp error 0x10

NVIDIA provided the documentation for mp error 0x10, INVALID_ADDR_SPACE,
which apparently happens when trying to use an atomic operation on
local or shared memory (instead of global memory).

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/device: remove pci/platform_device from common struct

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: convert to new-style nvkm_engine

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/mc: convert to new-style nvkm_subdev

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/ltc: convert to new-style nvkm_subdev

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: convert user classes to new-style nvkm_object

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/fifo: convert user classes to new-style nvkm_object

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: remove dependence on namedb/engctx lookup

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: directly use instmem where currently possible

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/device: prepare for new-style subdevs

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/object: rename some functions to avoid upcoming conflicts

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: switch to gpuobj accessor macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: switch to subdev printk macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: switch to new-style timer macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: switch to device pri macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: cosmetic changes

This is purely preparation for upcoming commits, there should be no
code changes here.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/fifo: cosmetic changes

This is purely preparation for upcoming commits, there should be no
code changes here.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/device: include core/device.h automatically for subdevs/engines

Pretty much every subdev/engine is going to need access to nvkm_device
shortly to touch registers and/or output messages.

The odd placement of the includes is necessary to work around some
inter-dependencies that currently exist. This will be fixed later.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: add GM20B support

Add support for GM20B's graphics engine, based on GK20A. Note that this
code alone will not allow the engine to initialize on released devices
which require PMU-assisted secure boot.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gk20a: use same initialization sequence as nvgpu

GK20A's initialization was based on GK104, but differences exist in the
way the initial context is built and the initialization process itself.

This patch follows the same initialization sequence as nvgpu performs
to avoid bad surprises. Since the register bundles initialization also
differ considerably from GK104, the register packs are now loaded from
firmware files, again similarly to what is done with nvgpu.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: use NVIDIA-provided external firmwares

NVIDIA will officially start providing GR firmwares through
linux-firmware for GPUs that require it. Change the GR firmware lookup
function to use these files.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100: wait for GR idle after GO_IDLE bundle

After submitting a GO_IDLE bundle, one must wait for GR to effectively
be idle before submitting the next bundle. Failure to do so may result
in undefined behavior in some rare cases.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reported-by: Kary Jin <karyj@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100: wait on bottom half of FE's pipeline

When emitting the ICMD bundle, wait on the bottom half (bit 3 of the
GR_STATUS register) instead of upper half (bit 2) to make sure methods
are effectively emitted.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gk104-: prevent reading non-existent regs in intr handler

Under certain circumstances the trapped address will contain subc 7,
which GK104 GR doesn't have anymore.

Notice this case to avoid causing additional priv ring faults.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: add symbolic names for classes

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100: Clear notify interrupt

Notify interrupt is only used for cyclestats. We can just clear it and
avoid an "unknown stat" error that gets printed to dmesg otherwise.

Signed-off-by: Lauri Peltonen <lpeltonen@nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/graph/nvc0: Fix engine pointer retrieval

Other methods in this file suggest this is the correct way to retrieve
the engine pointer.

Signed-off-by: Lauri Peltonen <lpeltonen@nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: namespace + nvidia gpu names (no binary change)

The namespace of NVKM is being changed to nvkm_ instead of nouveau_,
which will be used for the DRM part of the driver. This is being
done in order to make it very clear as to what part of the driver a
given symbol belongs to, and as a minor step towards splitting the
DRM driver out to be able to stand on its own (for virt).

Because there's already a large amount of churn here anyway, this is
as good a time as any to also switch to NVIDIA's device and chipset
naming to ease collaboration with them.

A comparison of objdump disassemblies proves no code changes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>