drm/amdgpu: move UVD and VCE sched entity init after sched init

We need kernel scheduling entities to deal with handle clean up
if apps are not cleaned up properly. With commit 56e449603f0ac5
("drm/sched: Convert the GPU scheduler to variable number of run-queues")
the scheduler entities have to be created after scheduler init, so
change the ordering to fix this.

v2: Leave logic in UVD and VCE code

Fixes: 56e449603f0a ("drm/sched: Convert the GPU scheduler to variable number of run-queues")
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Luben Tuikov <ltuikov89@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: ltuikov89@gmail.com

drm/amd: Split up UVD suspend into prepare and suspend steps

amdgpu_uvd_suspend() allocates memory and copies objects into that
allocated memory. This fails under memory pressure. Instead move
majority of this code into a prepare step when swap can still be
allocated.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: expand cg_flags from u32 to u64

With this, we can support more CG flags.

Signed-off-by: Evan Quan <evan.quan@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Fix crash on device remove/driver unload

Crash:
BUG: unable to handle page fault for address: 00000000000010e1
RIP: 0010:vega10_power_gate_vce+0x26/0x50 [amdgpu]
Call Trace:
pp_set_powergating_by_smu+0x16a/0x2b0 [amdgpu]
amdgpu_dpm_set_powergating_by_smu+0x92/0xf0 [amdgpu]
amdgpu_dpm_enable_vce+0x2e/0xc0 [amdgpu]
vce_v4_0_hw_fini+0x95/0xa0 [amdgpu]
amdgpu_device_fini_hw+0x232/0x30d [amdgpu]
amdgpu_driver_unload_kms+0x5c/0x80 [amdgpu]
amdgpu_pci_remove+0x27/0x40 [amdgpu]
pci_device_remove+0x3e/0xb0
device_release_driver_internal+0x103/0x1d0
device_release_driver+0x12/0x20
pci_stop_bus_device+0x79/0xa0
pci_stop_and_remove_bus_device_locked+0x1b/0x30
remove_store+0x7b/0x90
dev_attr_store+0x17/0x30
sysfs_kf_write+0x4b/0x60
kernfs_fop_write_iter+0x151/0x1e0

Why:
VCE/UVD had dependency on SMC block for their suspend but
SMC block is the first to do HW fini due to some constraints

How:
Since the original patch was dealing with suspend issues
move the SMC block dependency back into suspend hooks as
was done in V1 of the original patches.
Keep flushing idle work both in suspend and HW fini seuqnces
since it's essential in both cases.

Fixes: 859e4659273f1d ("drm/amdgpu: add missing cleanups for more ASICs on UVD/VCE suspend")
Fixes: bf756fb833cbe8 ("drm/amdgpu: add missing cleanups for Polaris12 UVD/VCE on suspend")
Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add missing cleanups for more ASICs on UVD/VCE suspend

This is a supplement for commit below:
"drm/amdgpu: add missing cleanups for Polaris12 UVD/VCE on suspend".

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Guchun Chen <guchun.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add the sched_score to amdgpu_ring_init

Allow separate ring to share the same scheduler score.

No functional change.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-and-Tested-by: Leo Liu <leo.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu/uvd_v5_0: Fix a bunch of kernel-doc function documentation issues

Fixes the following W=1 kernel build warning(s):

drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c:153: warning: Function parameter or member 'handle' not described in 'uvd_v5_0_hw_init'
drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c:153: warning: Excess function parameter 'adev' description in 'uvd_v5_0_hw_init'
drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c:210: warning: Function parameter or member 'handle' not described in 'uvd_v5_0_hw_fini'
drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c:210: warning: Excess function parameter 'adev' description in 'uvd_v5_0_hw_fini'
drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c:463: warning: Function parameter or member 'addr' not described in 'uvd_v5_0_ring_emit_fence'
drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c:463: warning: Function parameter or member 'seq' not described in 'uvd_v5_0_ring_emit_fence'
drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c:463: warning: Function parameter or member 'flags' not described in 'uvd_v5_0_ring_emit_fence'
drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c:463: warning: Excess function parameter 'fence' description in 'uvd_v5_0_ring_emit_fence'
drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c:529: warning: Function parameter or member 'job' not described in 'uvd_v5_0_ring_emit_ib'
drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c:529: warning: Function parameter or member 'flags' not described in 'uvd_v5_0_ring_emit_ib'

Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Nirmoy Das <nirmoy.das@amd.com>
Cc: amd-gfx@lists.freedesktop.org
Cc: dri-devel@lists.freedesktop.org
Signed-off-by: Lee Jones <lee.jones@linaro.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd: fix typoes in comments

Change the comment typo: "programm" -> "program".

Signed-off-by: Bernard Zhao <bernard@vivo.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: rework sched_list generation

Generate HW IP's sched_list in amdgpu_ring_init() instead of
amdgpu_ctx.c. This makes amdgpu_ctx_init_compute_sched(),
ring.has_high_prio and amdgpu_ctx_init_sched() unnecessary.
This patch also stores sched_list for all HW IPs in one big
array in struct amdgpu_device which makes amdgpu_ctx_init_entity()
much more leaner.

v2:
fix a coding style issue
do not use drm hw_ip const to populate amdgpu_ring_type enum

v3:
remove ctx reference and move sched array and num_sched to a struct
use num_scheds to detect uninitialized scheduler list

v4:
use array_index_nospec for user space controlled variables
fix possible checkpatch.pl warnings

Signed-off-by: Nirmoy Das <nirmoy.das@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: stop disable the scheduler during HW fini

When we stop the HW for example for GPU reset we should not stop the
front-end scheduler. Otherwise we run into intermediate failures during
command submission.

The scheduler should only be stopped in very few cases:
1. We can't get the hardware working in ring or IB test after a GPU reset.
2. The KIQ scheduler is not used in the front-end and should be disabled during GPU reset.
3. In amdgpu_ring_fini() when the driver unloads.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Nirmoy Das <nirmoy.das@amd.com>
Test-by: Dennis Li <dennis.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove unnecessary conversion to bool

Better clean that up before some automation starts to complain about it

Signed-off-by: Nirmoy Das <nirmoy.das@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd: drop use of drmP.h in remaining files

With this commit drm/amd/ has no longer any uses of
the deprecated drmP.h header file.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: "David (ChunMing) Zhou" <David1.Zhou@amd.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20190609220757.10862-11-sam@ravnborg.org

drm/amd: drop dependencies on drm_os_linux.h

Fix so no files in drm/amd/ depends on the
deprecated drm_os_linux.h header file.

It was done manually:
- remove drm_os_linux.h from drmP.h
- fix all build errros

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: "David (ChunMing) Zhou" <David1.Zhou@amd.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20190609220757.10862-3-sam@ravnborg.org

drm/amdgpu/UVD: set no_user_fence flag to true

There is no user fence support for UVD

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add flags to emit_ib interface v2

Replace the last bool type parameter with a general flags parameter,
to make the last parameter be able to contain more information.

v2: drop setting need_ctx_switch = false

Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Jack Xiao <Jack.Xiao@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Reorder uvd ring init before uvd resume

As amd_uvd_resume() accesses the uvd ring, it must be initialised first
or else we trigger errors like:

[ 5.595963] [drm] Found UVD firmware Version: 1.87 Family ID: 17
[ 5.595969] [drm] PSP loading UVD firmware
[ 5.596266] ------------[ cut here ]------------
[ 5.596268] ODEBUG: assert_init not available (active state 0) object type: timer_list hint: (null)
[ 5.596285] WARNING: CPU: 0 PID: 507 at lib/debugobjects.c:329 debug_print_object+0x6a/0x80
[ 5.596286] Modules linked in: amdgpu(+) hid_logitech_hidpp(+) chash gpu_sched amd_iommu_v2 ttm drm_kms_helper crc32c_intel drm hid_sony ff_memless igb hid_logitech_dj nvme dca i2c_algo_bit nvme_core wmi pinctrl_amd uas usb_storage
[ 5.596299] CPU: 0 PID: 507 Comm: systemd-udevd Tainted: G W 4.20.0-0.rc1.git4.1.fc30.x86_64 #1
[ 5.596301] Hardware name: System manufacturer System Product Name/ROG STRIX X470-I GAMING, BIOS 0901 07/23/2018
[ 5.596303] RIP: 0010:debug_print_object+0x6a/0x80
[ 5.596305] Code: 8b 43 10 83 c2 01 8b 4b 14 4c 89 e6 89 15 e6 82 b0 02 4c 8b 45 00 48 c7 c7 60 fd 34 a6 48 8b 14 c5 a0 da 08 a6 e8 6a 6a b8 ff <0f> 0b 5b 83 05 d0 45 3e 01 01 5d 41 5c c3 83 05 c5 45 3e 01 01 c3
[ 5.596306] RSP: 0018:ffffa02ac863f8c0 EFLAGS: 00010282
[ 5.596307] RAX: 0000000000000000 RBX: ffffa02ac863f8e0 RCX: 0000000000000006
[ 5.596308] RDX: 0000000000000007 RSI: ffff9160e9a7bfe8 RDI: ffff9160f91d6c60
[ 5.596310] RBP: ffffffffa6742740 R08: 0000000000000002 R09: 0000000000000000
[ 5.596311] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa634ff69
[ 5.596312] R13: 00000000000b79d0 R14: ffffffffa80f76d8 R15: 0000000000266000
[ 5.596313] FS: 00007f762abf7940(0000) GS:ffff9160f9000000(0000) knlGS:0000000000000000
[ 5.596314] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 5.596315] CR2: 000055fdc593f000 CR3: 00000007e999c000 CR4: 00000000003406f0
[ 5.596317] Call Trace:
[ 5.596321] debug_object_assert_init+0x14a/0x180
[ 5.596327] del_timer+0x2e/0x90
[ 5.596383] amdgpu_fence_process+0x47/0x100 [amdgpu]
[ 5.596430] amdgpu_uvd_resume+0xf6/0x120 [amdgpu]
[ 5.596475] uvd_v7_0_sw_init+0xe0/0x280 [amdgpu]
[ 5.596523] amdgpu_device_init.cold.30+0xf97/0x14b6 [amdgpu]
[ 5.596563] ? amdgpu_driver_load_kms+0x53/0x330 [amdgpu]
[ 5.596604] amdgpu_driver_load_kms+0x86/0x330 [amdgpu]
[ 5.596614] drm_dev_register+0x115/0x150 [drm]
[ 5.596654] amdgpu_pci_probe+0xbd/0x120 [amdgpu]
[ 5.596658] local_pci_probe+0x41/0x90
[ 5.596661] pci_device_probe+0x188/0x1a0
[ 5.596666] really_probe+0xf8/0x3b0
[ 5.596669] driver_probe_device+0xb3/0xf0
[ 5.596672] __driver_attach+0xe1/0x110
[ 5.596674] ? driver_probe_device+0xf0/0xf0
[ 5.596676] bus_for_each_dev+0x79/0xc0
[ 5.596679] bus_add_driver+0x155/0x230
[ 5.596681] ? 0xffffffffc07d9000
[ 5.596683] driver_register+0x6b/0xb0
[ 5.596685] ? 0xffffffffc07d9000
[ 5.596688] do_one_initcall+0x5d/0x2be
[ 5.596691] ? rcu_read_lock_sched_held+0x79/0x80
[ 5.596693] ? kmem_cache_alloc_trace+0x264/0x290
[ 5.596695] ? do_init_module+0x22/0x210
[ 5.596698] do_init_module+0x5a/0x210
[ 5.596701] load_module+0x2137/0x2430
[ 5.596703] ? lockdep_hardirqs_on+0xed/0x180
[ 5.596714] ? __do_sys_init_module+0x150/0x1a0
[ 5.596715] __do_sys_init_module+0x150/0x1a0
[ 5.596722] do_syscall_64+0x60/0x1f0
[ 5.596725] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 5.596726] RIP: 0033:0x7f762b877dee
[ 5.596728] Code: 48 8b 0d 9d 20 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca b8 af 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 6a 20 0c 00 f7 d8 64 89 01 48
[ 5.596729] RSP: 002b:00007ffc777b8558 EFLAGS: 00000246 ORIG_RAX: 00000000000000af
[ 5.596730] RAX: ffffffffffffffda RBX: 000055fdc48da320 RCX: 00007f762b877dee
[ 5.596731] RDX: 00007f762b9f284d RSI: 00000000006c5fc6 RDI: 000055fdc527a060
[ 5.596732] RBP: 00007f762b9f284d R08: 0000000000000003 R09: 0000000000000002
[ 5.596733] R10: 000055fdc48ad010 R11: 0000000000000246 R12: 000055fdc527a060
[ 5.596734] R13: 000055fdc48dca20 R14: 0000000000020000 R15: 0000000000000000
[ 5.596740] irq event stamp: 134618
[ 5.596743] hardirqs last enabled at (134617): [<ffffffffa513d52e>] console_unlock+0x45e/0x610
[ 5.596744] hardirqs last disabled at (134618): [<ffffffffa50037e8>] trace_hardirqs_off_thunk+0x1a/0x1c
[ 5.596746] softirqs last enabled at (133146): [<ffffffffa5e00365>] __do_softirq+0x365/0x47c
[ 5.596748] softirqs last disabled at (133139): [<ffffffffa50c64f9>] irq_exit+0x119/0x120
[ 5.596749] ---[ end trace eaee508abfebccdc ]---

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=108709
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Alex Deucher <alexdeucher@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Modify the argument of emit_ib interface

use the point of struct amdgpu_job as the function
argument instand of vmid, so the other members of
struct amdgpu_job can be visit in emit_ib function.

v2: add a wrapper for getting the VMID
add the job before the ib on the parameter list.
v3: refine the wrapper name

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: further ring test cleanups

Move all error messages from IP specific code into the common helper.
This way we now uses the ring name in the messages instead of the index
and note which device is affected as well.

Also cleanup error handling in the IP specific code and consequently use
ETIMEDOUT when the ring test timed out.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Retire amdgpu_ring.ready flag v4

Start using drm_gpu_scheduler.ready isntead.

v3:
Add helper function to run ring test and set
sched.ready flag status accordingly, clean explicit
sched.ready sets from the IP specific files.

v4: Add kerneldoc and rebase.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: move more defines into amdgpu_irq.h

Everything that isn't related to the IH ring.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/uvd: UVD entity initialization relys on ring initialization

Entity init should after ring init, as the entity's sched_rq's initialization
is in ring init.

SWDEV-161495

Signed-off-by: Emily Deng <Emily.Deng@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd: Use newly added interrupt source defs for VI v3.

v2: Rebase
v3: Use defines for CP_SQ and CP_ECC_ERROR interrupts.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix insert nop for UVD5 ring

NO_OP register should be writen to 0

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/vg20:Restruct uvd to support multiple uvds

Vega20 has dual-UVD. Need Restruct amdgpu_device::uvd to support
multiple uvds. There are no any logical changes here.

Signed-off-by: James Zhu <James.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove now superflous *_hdp operation

All HDP invalidation and most flush can now be replaced by the generic
ASIC function.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: rename vm_id to vmid

sed -i "s/vm_id/vmid/g" drivers/gpu/drm/amd/amdgpu/*.c
sed -i "s/vm_id/vmid/g" drivers/gpu/drm/amd/amdgpu/*.h

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: fix UVD mc offsets

When UVD bo is created, its size is based on the information from firmware
header (ucode_size_bytes). The same value should be be used when programming
UVD mc controller offsets, otherwise it can happen that
(mmUVD_VCPU_CACHE_OFFSET2 + mmUVD_VCPU_CACHE_SIZE2) will point
AMDGPU_GPU_PAGE_SIZE bytes after the UVD bo end.

Second issue is that when programming the mmUVD_VCPU_CACHE_SIZE0 register,
AMDGPU_UVD_FIRMWARE_OFFSET should be taken into account. If it isn't,
(mmUVD_VCPU_CACHE_OFFSET2 + mmUVD_VCPU_CACHE_SIZE2) will always point
AMDGPU_UVD_FIRMWARE_OFFSET bytes after the UVD bo end.

v2: move firmware size calculation into macro definition
v3: align firmware size to the gpu page size

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Piotr Redlewski <predlewski@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: change redundant init logs to debug level

When this VF stays in exclusive mode for long, other VFs will be
impacted.

The redundant messages causes exclusive mode timeout when they're
redirected. That is a normal use case for cloud service to redirect
guest log to virtual serial port.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: pding <Pixel.Ding@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: various cleanups for uvd/vce.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: switch ih handling to two levels (v3)

Newer asics have a two levels of irq ids now:
client id - the IP
src id - the interrupt src within the IP

v2: integrated Christian's comments.
v3: fix rebase fail in SI and CIK

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Ken Wang <Qingqing.Wang@amd.com>
Reviewed-by: Ken Wang <Qingqing.Wang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: change wptr to 64 bits (v2)

Newer asics need 64 bit wptrs. If the wptr is now
smaller than the rptr that doesn't indicate a wrap-around
anymore.

v2: integrate Christian's comments.

Signed-off-by: Ken Wang <Qingqing.Wang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: read hw register to check pg status.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: refine uvd5.0/6.0 code.

1. delete redundant cg pg mask check.
pg mask use to ctrl power on/down uvd.
not start/stop uvd.
cg mask will be check when enable mgcg.
2. no need to start uvd when initializ.
when ring test/ib test/encode, uvd was enabled.
when uvd idle, uvd was stopped.
3. chang cg pg sequence in powerplay.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add get clockgating_state method for uvd v5&v6

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove static integer for uvd pp state

At two gpu core condition, static integer will cause that second gpu
core uvd state setting will be directly skipped due to the first one
setting

Signed-off-by: Yintian Tao <yttao@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix bug mclk can't change on Polaris

the root cause is we gate the clock to uvd vcpu.
mclk's change should need the response from uvd if
it is power on.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: refine uvd 5.0 clock gate feature.

1. fix uvd cg status not correct.
2. fix uvd pg can't work on tonga.
3. enable uvd mgcg.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Tom St Denis <tom.stdenis@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: not set bypass mode for uvd5.0/uvd6.0

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: enable uvd bypass mode for CI/VI.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: rework IP block registration (v2)

This makes it easier to replace specific IP blocks on
asics for handling virtual_dce, DAL, etc. and for building
IP lists for hw or tables. This also stored the status
information in the same structure.

v2: split out spelling fix into a separate patch
add a function to add IPs to the list

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: move align_mask and nop into ring funcs as well (v2)

They are constant as well.

v2: update uvd and vce phys ring structures as well

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: move the ring type into the funcs structure (v2)

It's constant, so it doesn't make to much sense to keep it
with the variable data.

v2: update vce and uvd phys mode ring structures as well

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: move IB and frame size directly into the engine description

I should have suggested that on the initial patchset. This saves us a
few CPU cycles during CS and a bunch of loc.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/uvd5: add ring callbacks for ib and dma frame size

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: switch UVD code to use UVD_NO_OP for padding

Replace packet2's with packet0 writes to UVD_NO_OP. The
value written to UVD_NO_OP does not matter.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: move UVD IB test into common code v2

Since we now raise the clocks from begin_use() we don't need
a separate function for each hw generation any more.

v2: remove unintentional lowering of the UVD clocks, fix typos for CIK hw.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use begin/end_use for UVD power/clock gating

This fixes turning power and clock on when it is actually needed.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: implement HDP functions for UVD v2

Flush and invalidate the HDP caches.

v2: fix typo in comment

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: move VM fields into job

They are the same for all IBs.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: move context switch handling into common code v2

It was a source of bugs to repeat that in each IP version.

v2: rename parameter

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: Add name field to amd_ip_funcs (v2)

Add name that we can print out in kernel messages
to aid in debugging.

v2: drop DAL changes for upstream

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: Drop print_status callbacks.

First patch in series to move to user mode
debug tools we're removing the print_status callbacks.

These functions were unused at the moment anyway.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use max_dw in ring_init

Instead of specifying the total ring size calculate that from the maximum
number of dw a submission can have and the number of concurrent submissions.

This fixes UVD with 8 concurrent submissions or more.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: handle more than 10 UVD sessions (v2)

Change History
--------------

v2:
- Make firmware version check correctly. Firmware
versions >= 1.80 should all support 40 UVD
instances.
- Replace AMDGPU_MAX_UVD_HANDLES with max_handles
variable.

v1:
- The firmware can handle upto 40 UVD sessions.

Signed-off-by: Arindam Nath <arindam.nath@amd.com>
Signed-off-by: Ayyappa Chandolu <ayyappa.chandolu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: Add SW clock gating support to UVD 5 and 6

This patch adds support for software clock gating to UVD 5
and UVD 6 blocks with a preliminary commented out hardware
gating routine.

Currently hardware gating does not work so it's not activated.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: save and restore UVD context with suspend and resume

and revert fix following it accordingly

Revert "drm/amdgpu: stop trying to suspend UVD sessions v2"
Revert "drm/amdgpu: fix the UVD suspend sequence order"

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/vi: move uvd tiling config setup into uvd code

Split uvd and gfx programming.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: send UVD IB tests directly to the ring again

We need the IB test for GPU resets as well and
the scheduler should be stoped then.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: make pad_ib a ring function v3

The padding depends on the firmware version and we need that for BO moves as
well, not only for VM updates.

v2: new approach of making pad_ib a ring function
v3: fix typo in macro name

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove the ring lock v2

It's not needed any more because all access goes through the scheduler now.

v2: Update commit message.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove some more semaphore leftovers

No longer needed since semaphores were removed.

Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <David1.Zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: clean up hw semaphore support in driver

No longer used.

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Ken Wang <Qingqing.Wang@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drma/dmgpu: move cg and pg flags into shared headers

So they can be used by powerplay.

Reviewed-by: Eric Huang <JinHuiEric.Huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: be consistent with uvd cg flags

Don't do anything if the uvd cg flags are not set.

Reviewed-by: Eric Huang <JinHuiEric.Huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: handle uvd pg flags properly

Don't attempt to start/stop the uvd block if pg is disabled.

Reviewed-by: Eric Huang <JinHuiEric.Huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove old lockup detection infrastructure

It didn't worked to well anyway.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>

drm/amdgpu: fix the UVD suspend sequence order

Fixes suspend issues with UVD.

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Cc: stable@vger.kernel.org

drm/amdgpu: add insert_nop ring func and default implementation

The insert_nop function is added to amdgpu_ring_funcs structure as
well as the default implementation

Signed-off-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: change uvd ib test to use kernel fence directly

Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <jammy.zhou@amd.com>

drm/amdgpu: disable user fence interrupt (v2)

amdgpu submits both kernel and user fences, but just need one interrupt,
disable user fence interrupt and don't effect user fence.

v2: fix merge error

Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: rename amdgpu_ip_funcs to amd_ip_funcs (v2)

The structure is renamed and moved to amd_shared.h to make
the component independent. This makes it easier to add
new components in the future.

v2: fix include path

Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Signed-off-by: yanyang1 <young.yang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Add initial VI support

This adds initial support for VI asics. This
includes Iceland, Tonga, and Carrizo. Our inital
focus as been Carrizo, so there are still gaps in
support for Tonga and Iceland, notably power
management.

Acked-by: Christian König <christian.koenig@amd.com>
Acked-by: Jammy Zhou <Jammy.Zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>