drm/nouveau/gr: enable memory loads on helper invocation on all channels

We have a lurking bug where Fragment Shader Helper Invocations can't load
from memory. But this is actually required in OpenGL and is causing random
hangs or failures in random shaders.

It is unknown how widespread this issue is, but shaders hitting this can
end up with infinite loops.

We enable those only on all Kepler and newer GPUs where we use our own
Firmware.

Nvidia's firmware provides a way to set a kernelspace controlled list of
mmio registers in the gr space from push buffers via MME macros.

v2: drop code for gm200 and newer.

Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: David Airlie <airlied@gmail.com>
Cc: nouveau@lists.freedesktop.org
Cc: stable@vger.kernel.org # 4.19+
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230622152017.2512101-1-kherbst@redhat.com

drm/nouveau/gr/gf117-: make ppc_nr[gpc] accurate

We're going to be pulling in a chunk of code from NVGPU to fixup our
SMID mappings on Volta and above, which depends on ppc_nr[gpc]
reflecting the actual number of PPCs present, not the maximum number.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: make global attrib_cb actually global

This was thought to be per-channel initially - it's not. The backing
pages for the VMM mappings are shared for all channels.

- switches to more straight-forward patch interfaces
- prepares for sub-context support
- this is saving a *sizeable* amount of vram

v2:
- whitespace

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: make global bundle_cb actually global

This was thought to be per-channel initially - it's not. The backing
pages for the VMM mappings are shared for all channels.

- switches to more straight-forward patch interfaces
- prepares for sub-context support

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-: make global pagepool actually global

This was thought to be per-channel initially - it's not. The backing
pages for the VMM mappings are shared for all channels.

- switches to more straight-forward patch interfaces
- prepares for sub-context support

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gr/gf100-gm10x: update register lists

There are differences on GM200 and newer too, but we can't fix them there
as they come from firmware packages.

A request has been made to NVIDIA to release updated firmware.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: delete duplicated grctx init code

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise r419e00

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise gpc_tpc_nr

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise r406500

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise dist_skip_table + improve algorithm

The algorithm for GM200 and newer matches RM for all the boards I have, but
I don't have enough data to try and figure something out for earlier boards,
so these will still write zeroes to the table as we did before.

The code in NVGPU isn't helpful here, it appears to handle specific cases.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise alpha_beta_tables + improve algorithms

I haven't yet been able to find a fully programatic way of calculating the
same mapping as NVIDIA for GF100-GF119, so the algorithm partially depends
on data tables for specific configurations.

I couldn't find traces for every possibility, so the algorithm will switch
to a mapping similar to what GK104-GM10x use if it encounters one. We did
the wrong thing before anyway, so shouldn't matter too much.

The algorithm used in the GK104 implementation was ported from NVGPU.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise rop_mapping

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise tpc_per_gpc

GM20B now also shares the same code, as NVGPU shows it doesn't need
special treatment.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: virtualise sm_id/tpc_nr

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: use new interfaces for vmm operations

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: mark symbols static where possible

We get a few warnings when building kernel with W=1:
drivers/gpu/drm/nouveau/nvkm/subdev/bios/fan.c:29:1: warning: no previous prototype for 'nvbios_fan_table' [-Wmissing-prototypes]
drivers/gpu/drm/nouveau/nvkm/subdev/bios/fan.c:56:1: warning: no previous prototype for 'nvbios_fan_entry' [-Wmissing-prototypes]
drivers/gpu/drm/nouveau/nvkm/subdev/clk/gt215.c:184:1: warning: no previous prototype for 'gt215_clk_info' [-Wmissing-prototypes]
drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramgt215.c:99:1: warning: no previous prototype for 'gt215_link_train_calc' [-Wmissing-prototypes]
drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramgt215.c:153:1: warning: no previous prototype for 'gt215_link_train' [-Wmissing-prototypes]
drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramgt215.c:271:1: warning: no previous prototype for 'gt215_link_train_init' [-Wmissing-prototypes]
....

In fact, both functions are only used in the file in which they are
declared and don't need a declaration, but can be made static.
So this patch marks these functions with 'static'.

Signed-off-by: Baoyou Xie <baoyou.xie@linaro.org>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gm107-: fix touching non-existent ppcs in attrib cb setup

Also removes an XXX; according to nvgpu headers the field is called
NV_PGRAPH_GPCS_SWDX_TC_BETA_CB_SIZE_DIV3, so, apparently not some
magic we need to figure out :)

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gk104-: move rop_active_fbps init to nonctx

Matches newer RM.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: remove hardcoded idle_timeout values

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: convert user classes to new-style nvkm_object

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: switch to device pri macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: cosmetic changes

This is purely preparation for upcoming commits, there should be no
code changes here.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: add GM20B support

Add support for GM20B's graphics engine, based on GK20A. Note that this
code alone will not allow the engine to initialize on released devices
which require PMU-assisted secure boot.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gm204: initial init+ctx code

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gm107: very slightly demagic part of attrib cb setup

No idea if "3" is a constant or derived from something else, but the
value is unchanged in the limited traces of gm107/gm204 I have here.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gk104-: correct crop/zrop num_active_fbps setting

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100: fix some accidental or'ing of buffer addresses

fdo#83992

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: namespace + nvidia gpu names (no binary change)

The namespace of NVKM is being changed to nvkm_ instead of nouveau_,
which will be used for the DRM part of the driver. This is being
done in order to make it very clear as to what part of the driver a
given symbol belongs to, and as a minor step towards splitting the
DRM driver out to be able to stand on its own (for virt).

Because there's already a large amount of churn here anyway, this is
as good a time as any to also switch to NVIDIA's device and chipset
naming to ease collaboration with them.

A comparison of objdump disassemblies proves no code changes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr: rename from graph (no binary change)

Shorter device name, match Tegra and our existing enums.

The namespace of NVKM is being changed to nvkm_ instead of nouveau_,
which will be used for the DRM part of the driver. This is being
done in order to make it very clear as to what part of the driver a
given symbol belongs to, and as a minor step towards splitting the
DRM driver out to be able to stand on its own (for virt).

Because there's already a large amount of churn here anyway, this is
as good a time as any to also switch to NVIDIA's device and chipset
naming to ease collaboration with them.

A comparison of objdump disassemblies proves no code changes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>