drm/radeon: deprecate and remove KFD interface

To quote Felix: "For testing KV with current user mode stack, please use
amdgpu. I don't expect this to work with radeon and I'm not planning to
spend any effort on making radeon work with a current user mode stack."

Only compile tested, but should be straight forward.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

drm/amd: Spelling s/SDMA_WRTIE_SUB_OPCODE_TILED/SDMA_WRITE_SUB_OPCODE_TILED/

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: dri-devel@lists.freedesktop.orgamd-gfx@lists.freedesktop.org
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: switch UVD code to use UVD_NO_OP for padding

Replace packet2's with packet0 writes to UVD_NO_OP. The
value written to UVD_NO_OP does not matter.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: handle more than 10 UVD sessions

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Arindam Nath <arindam.nath@amd.com>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Add ATC VMID<-->PASID functions to kfd->kgd

This patch adds three new interfaces to kfd2kgd interface file of radeon.

The interfaces are:

- Check if a specific VMID has a valid PASID mapping
- Retrieve the PASID which is mapped to a specific VMID
- Issue a VMID invalidation request to the ATC

Signed-off-by: Alexey Skidanov <Alexey.Skidanov@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

drm/radeon: Add init interrupt kfd->kgd interface

This patch adds a new interface function to the kfd->kgd interface.
The function is kgd_init_interrupts() and its function is to
initialize a pipe's interrupts.

The function currently enables the timestamp interrupt and the
bad opcode interrupt.

Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

drm/radeon: programm the VCE fw BAR as well

Otherwise the VCE firmware needs to be in the first 256MB of VRAM.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add get_allowed_info_register for CIK

Registers that can be fetched from the info ioctl.

Tested-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: enable SRBM timeout interrupt on CIK v2

v2: disable it on suspend

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Assign VMID to PASID for IH in non-HWS mode

This patch fixes a bug in kgd_set_pasid_vmid_mapping(), where the function
only updated the ATC registers (IOMMU) with the new VMID <--> PASID mapping,
but didn't update the IH (Interrupt) registers.

The bug only occurs when using non-HWS mode. In HWS mode, the CP automatically
does the VMID <--> PASID mapping.

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: fix typo in new fan control registers for SI/CI

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add smc fan control for CI (v2)

Enable smc fan control for CI boards. Should
reduce the fan noise on systems with a higher
default fan profile.

v2: disable by default, add additional fan setup, rpm control

bug:
https://bugs.freedesktop.org/show_bug.cgi?id=73338

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Add radeon <--> amdkfd interface

This patch adds the interface between the radeon driver and the amdkfd driver.
The interface implementation is contained in radeon_kfd.c and radeon_kfd.h.

The interface itself is represented by a pointer to struct
kfd_dev. The pointer is located inside radeon_device structure.

All the register accesses that amdkfd need are done using this interface. This
allows us to avoid direct register accesses in amdkfd proper, while also
avoiding locking between amdkfd and radeon.

The single exception is the doorbells that are used in both of the drivers.
However, because they are located in separate pci bar pages, the danger of
sharing registers between the drivers is minimal.

Having said that, we are planning to move the doorbells as well to radeon.

v3:

Add interface for sa manager init and fini. The init function will allocate a
buffer on system memory and pin it to the GART address space via the radeon sa
manager.

All mappings of buffers to GART address space are done via the radeon sa
manager. The interface of allocate memory will use the radeon sa manager to sub
allocate from the single buffer that was allocated during the init function.

Change lower_32/upper_32 calls to use linux macros

Add documentation for the interface

v4:

Change ptr field type in kgd_mem from uint32_t* to void* to match to type that
is returned by radeon_sa_bo_cpu_addr

v5:

Change format of mqd structure to work with latest KV firmware
Add support for AQL queues creation to enable working with open-source HSA
runtime.
Move generic kfd-->kgd interface and other generic kgd definitions to a generic
header file that will be used by AMD's radeon and amdgpu drivers

Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>

drm/radeon/cik: fix typo in EOP packet

Volatile bit was in the wrong location. This bit is
not used at the moment.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: add proper support for RADEON_VM_BLOCK_SIZE v2

This patch makes it possible to decide how many address
bits are spend on the page directory vs the page tables.

v2: remove unintended change

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: use pflip irq on R600+ v2

Testing the update pending bit directly after issuing an
update is nonsense cause depending on the pixel clock the
CRTC needs a bit of time to execute the flip even when we
are in the VBLANK period.

This is just a non invasive patch to solve the problem at
hand, a more complete and cleaner solution should follow
in the next merge window.

Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=76564

v2: fix source IDs for CRTC2-6

Signed-off-by: Christian König <christian.koenig@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: add support for vce 2.0 clock gating

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: fill in set_vce_clocks for CIK asics

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: initial VCE support v4

Only VCE 2.0 support so far.

v2: squashing multiple patches into this one
v3: add IRQ support for CIK, major cleanups,
basic code documentation
v4: remove HAINAN from chipset list

Signed-off-by: Christian König <christian.koenig@amd.com>

drm/radeon: reduce number of free VMIDs and pipes in KV

To support HSA on KV, we need to limit the number of vmids and pipes
that are available for radeon's use with KV.

This patch reserves VMIDs 8-15 for amdkfd (so radeon can only use VMIDs
0-7) and also makes radeon thinks that KV has only a single MEC with a single
pipe in it

v3: Use define for static vmid allocation in radeon

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>

drm/radeon: implement pci config reset for CIK (v3)

pci config reset is a low level reset that resets
the entire chip from the bus interface. It can
be more reliable if soft reset fails.

v2: fix rebase
v3: hide behind module parameter

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: update rb setup for hawaii

The formula needs to be adjusted since there are 4 RBs
per SH rather than 2 as on previous asics.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: update cik_tiling_mode_table_init() for hawaii

Hawaii uses a different tiling configuration. Add support
for it.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: minor updates to cik.c for hawaii

Skip programming a register that was removed and
adjust the mask of the VM client status.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: update cik_gpu_init() for hawaii

This adds the hawaii asic specific configuration
details.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: implement blit copy callback for CIK

Uses the CP ring rather than the DMA ring. Useful
for debugging and benchmarking.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Add support for programming the FMT blocks

The FMT blocks control how data is sent from the backend
of the display pipe to to monitor. Proper set up of the
FMT blocks are required for 30bpp formats. Additionally,
dithering can be enabled on for better display with 18 and
24bpp displays. The exception is LVDS/eDP which atom
takes care of in the SelectCRTC_Source table. For now
just enable truncation until we test dithering more.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: update line buffer allocation for dce8

We need to allocate line buffer to each display when
setting up the watermarks. Failure to do so can lead
to a blank screen. This fixes blank screen problems
on dce8 asics.

Based on an initial fix from:
Jay Cornwall <jay.cornwall@amd.com>

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: restructure cg/pg on cik (v2)

- use new cg/pg flags for finer grained clock and
powergating control
- restructure the cg/pg code so it can be called from
other components such as dpm

v2: fix build breakage from rebase

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add debugfs support for KB/KV

This allows you to look at the current DPM state via
debugfs.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add debugfs support for CI

This allows you to look at the current DPM state via debugfs.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add dpm support for CI dGPUs (v2)

This adds dpm support for btc asics. This includes:
- dynamic engine clock scaling
- dynamic memory clock scaling
- dynamic voltage scaling
- dynamic pcie gen switching

Set radeon.dpm=1 to enable.

v2: remove unused radeon_atombios.c changes,
make missing smc ucode non-fatal

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/kms: add dpm support for KB/KV

This adds dpm support for KB/KV asics. This includes:
- dynamic engine clock scaling
- dynamic voltage scaling
- power containment
- shader power scaling

Set radeon.dpm=1 to enable.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add get_temperature() callbacks for CIK (v2)

This added support for the on-chip thermal sensors on
CIK asics.

v2: fix register offset.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/cik: add rlc helpers for DPM

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: implement clock and power gating for CIK (v3)

Only the APUs support power gating.

v2: disable cgcg for now
v3: workaround hw issue in mgcg

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/cik: restructure rlc setup

Restructure rlc setup to handle clock and power
gating.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add support for ASPM on CIK asics

Enables PCIE ASPM (Active State Power Management) on
CIK asics.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/cik: add support for pcie gen1/2/3 switching

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add fault decode function for CIK

Helpful for debugging GPUVM errors as we can see what
hw block and page generated the fault in the log.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/cik: Add support for compute queues (v4)

On CIK, the compute rings work slightly differently than
on previous asics, however the basic concepts are the same.

The main differences:
- New MEC engines for compute queues
- Multiple queues per MEC:
- CI/KB: 1 MEC, 4 pipes per MEC, 8 queues per pipe = 32 queues
- KV: 2 MEC, 4 pipes per MEC, 8 queues per pipe = 64 queues
- Queues can be allocated and scheduled by another queue
- New doorbell aperture allows you to assign space in the aperture
for the wptr which allows for userspace access to queues

v2: add wptr shadow, fix eop setup
v3: fix comment
v4: switch to new callback method

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>

drm/radeon: add UVD support for CIK (v3)

v2: agd5f: fix clock dividers setup for bonaire
v3: agd5f: rebase

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/cik: add pcie_port indirect register accessors

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add get_xclk() callback for CIK

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: update CIK soft reset

Update to the newer programming model.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add get_gpu_clock_counter() callback for cik

Used for GPU clock counter snapshots.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dce8: add support for display watermark setup

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/cik: add support for sDMA dma engines (v8)

CIK has new asynchronous DMA engines called sDMA
(system DMA). Each engine supports 1 ring buffer
for kernel and gfx and 2 userspace queues for compute.

TODO: fill in the compute setup.

v2: update to the latest reset code
v3: remove ib_parse
v4: fix copy_dma()
v5: drop WIP compute sDMA queues
v6: rebase
v7: endian fixes for IB
v8: cleanup for release

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/cik: log and handle VM page fault interrupts

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add support for interrupts on CIK (v5)

Todo:
- handle interrupts for compute queues

v2: add documentation
v3: update to latest reset code
v4: update to latest illegal CP handling
v5: fix missing break in interrupt handler switch statement

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Add support for RLC init on CIK (v4)

RLC handles the interrupt controller and other tasks
on the GPU.

v2: add documentation
v3: update programming sequence
v4: additional setup

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add IB and fence dispatch functions for CIK gfx (v7)

For gfx ring only. Compute is still todo.

v2: add documentation
v3: update to latest reset changes, integrate emit update patch.
v4: fix count on wait_reg_mem for HDP flush
v5: use old hdp flush method for fence
v6: set valid bit for IB
v7: cleanup for release

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Add CP init for CIK (v7)

Sets up the GFX ring and loads ucode for GFX and Compute.

Todo:
- handle compute queue setup.

v2: add documentation
v3: integrate with latest reset changes
v4: additional init fixes
v5: scratch reg write back no longer supported on CIK
v6: properly set CP_RB0_BASE_HI
v7: rebase

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add support mc ucode loading on CIK (v2)

Load the GDDR5 ucode and train the links.

v2: update ucode

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/cik: stop page faults from hanging the system (v2)

Redirect invalid memory accesses to the default page
instead of locking up the memory controller.

v2: rebase on top of 2 level PTs

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add support for MC/VM setup on CIK (v6)

The vm callbacks are the same as the SI ones right now
(same regs and bits). We could share the SI variants, and
I may yet do that, but I figured I would add CIK specific
ones for now in case we need to change anything.

V2: add documentation, minor fixes.
V3: integrate vram offset fixes for APUs
V4: enable 2 level VM PTs
V5: index SH_MEM_* regs properly
V6: add ib_parse()

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Add support for CIK GPU reset (v2)

v2: split soft reset into compute and gfx. Still need
to make reset more fine grained, but this should be a
start.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add gpu init support for CIK (v9)

v2: tiling fixes
v3: more tiling fixes
v4: more tiling fixes
v5: additional register init
v6: rebase
v7: fix gb_addr_config for KV/KB
v8: drop wip KV bits for now, add missing config reg
v9: fix cu count on Bonaire

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>