drm/radeon: Clean up errors in radeon.h

Fix the following errors reported by checkpatch:

ERROR: open brace '{' following struct go on the same line

Signed-off-by: XueBing Chen <chenxb_99091@126.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Remove the references of radeon_gem_ pread & pwrite ioctls

Removing the functions of pread & pwrite & IOCTL defines, as their
existence allows an authorized client to spam the system logs.

Fixes: db996e64b293 ("drm/radeon: Fix ENOSYS with better fitting error codes in radeon_gem.c")
Suggested-by: Christian König <christian.koenig@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Move radeon_align_pitch() next to dumb-buffer helpers

Move radeon_align_pitch() next to its caller in the dumb-buffer
code. Removes the only dependency on the radeon's fbdev source file
that is not related to fbdev emulation. No functional changes.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Use the drm suballocation manager implementation.

Use the generic suballocation helper for radeon.

v3:
- Select the suballoc helper in Kconfig (Thomas).

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Co-developed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Acked-by: Alex Deucher <alexdeucher@gmail.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230224095152.30134-4-thomas.hellstrom@linux.intel.com

drm/ttm: merge ttm_bo_api.h and ttm_bo_driver.h v2

Merge and cleanup the two headers into a single description of the
object API. Also move all the documentation to the implementation and
drop unnecessary includes from the header.

No functional change.

v2: minimal checkpatch.pl cleanup

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221125102137.1801-4-christian.koenig@amd.com

drm/radeon: Add HD-audio component notifier support (v6)

This patch adds the support for the notification of HD-audio hotplug
via the already existing drm_audio_component framework to radeon
driver. This allows us more reliable hotplug notification and ELD
transfer without accessing HD-audio bus; it's more efficient, and more
importantly, it works without waking up the runtime PM.

The implementation is rather simplistic: radeon driver provides the
get_eld ops for HD-audio, and it notifies the audio hotplug via
pin_eld_notify callback upon each radeon_audio_enable() call.
The pin->id is referred as the port number passed to the notifier
callback, and the corresponding connector is looked through the
encoder list in the get_eld callback in turn.

The bind and unbind callbacks handle the device-link so that it
assures the PM call order.

Also, as a gratis bonus, this patch "fixes" the regression by the
recent change in HD-audio to be more strict for the HDMI/DP
connection, too. Since the HD-audio HDMI/DP codec requires both the
connection bit and the valid ELD to be provided, it started failing on
some RADEON gfx boards where the ELD update performed instably. As
this change switches the communication to a direct way between the
audio and the graphics drivers, now the system receives the proper
ELD, and the HDMI/DP hotplug starts working again.

[ v2: fix the logic in radeon_audio_component_get_eld to walk the
connector list since that is where the EDID lives and we can
derive the encoder from the connector because the encoder has
not been assigned at this point (i.e., during monitor probe).

v3: the component binding is moved outside radeon_audio_init() and
_fini(), as those are called from suspend/resume, too.
Drop modeset lock calls that caused Oops.
Moved Kconfig change so that it can be applied on older kernels.

v4: revive drm_modeset_lock*() again, add the missing
device_link_remove() call at unbinding

v5: squash in mutex fix

v6: squash in fix for audio get_eld callback ]

Link: https://gitlab.freedesktop.org/drm/amd/-/issues/1569
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: remove unused declarations for radeon

radeon_combios_get_ext_tmds_info() has been removed since
commit fcec570b27a4 ("drm/radeon/kms: add support for
external tmds on legacy boards").

radeon_mst has been removed since
commit 01ad1d9c2888 ("drm/radeon: Drop legacy MST support").

r600_hdmi_acr() has been removed since
commit 64424d6e45ae ("radeon/audio: consolidate update_acr()
functions (v2)").

So remove the declarations for them from header file.

Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Gaosheng Cui <cuigaosheng1@huawei.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220913024847.552254-4-cuigaosheng1@huawei.com

drm/radeon: remove resource accounting v2

Use the one provided by TTM instead.

v2: drop new_mem parameter as well

Signed-off-by: Christian König <christian.koenig@amd.com>
Tested-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220214093439.2989-5-christian.koenig@amd.com

drm/radeon: delete useless function return values & remove meaningless if(r) check code

Function radeon_fence_driver_init always returns success,
the function type maybe coule be changed to void.
This patch first delete the check of the return
value of the function call radeon_fence_driver_init, then,
optimise the function declaration and function to void type.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Bernard Zhao <bernard@vivo.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: Disable sclk switching on Oland when two 4K 60Hz monitors are connected

Screen flickers rapidly when two 4K 60Hz monitors are in use. This issue
doesn't happen when one monitor is 4K 60Hz (pixelclock 594MHz) and
another one is 4K 30Hz (pixelclock 297MHz).

The issue is gone after setting "power_dpm_force_performance_level" to
"high". Following the indication, we found that the issue occurs when
sclk is too low.

So resolve the issue by disabling sclk switching when there are two
monitors requires high pixelclock (> 297MHz).

v2:
- Only apply the fix to Oland.
Signed-off-by: Kai-Heng Feng <kai.heng.feng@canonical.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Move AGP data structures into radeon

With the AGP code already duplicated, move over the AGP structures
from the legacy code base in to radeon. The AGP data structures that
are required by radeon are now declared within the driver. The AGP
instance is stored in struct radeon_device.agp.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210507185709.22797-3-tzimmermann@suse.de

drm/radeon: Move AGP helpers into radeon driver

Radeon calls DRMs core AGP helpers. These helpers are only required
by legacy drivers. Reimplement the code in radeon to uncouple radeon
from the legacy code.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210507185709.22797-2-tzimmermann@suse.de

drm/radeon/dpm: Disable sclk switching on Oland when two 4K 60Hz monitors are connected

Screen flickers rapidly when two 4K 60Hz monitors are in use. This issue
doesn't happen when one monitor is 4K 60Hz (pixelclock 594MHz) and
another one is 4K 30Hz (pixelclock 297MHz).

The issue is gone after setting "power_dpm_force_performance_level" to
"high". Following the indication, we found that the issue occurs when
sclk is too low.

So resolve the issue by disabling sclk switching when there are two
monitors requires high pixelclock (> 297MHz).

v2:
- Only apply the fix to Oland.
Signed-off-by: Kai-Heng Feng <kai.heng.feng@canonical.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: also init GEM funcs in radeon_gem_prime_import_sg_table

Otherwise we will run into a NULL ptr deref.

Signed-off-by: Christian König <christian.koenig@amd.com>
Bug: https://bugzilla.kernel.org/show_bug.cgi?id=212137
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: do not use drm middle layer for debugfs (v2)

Use debugfs API directly instead of drm middle layer.

v2: squash in build fix (Alex)

Signed-off-by: Nirmoy Das <nirmoy.das@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add rdev in ring struct

Retrieving radeon device struct from ring struct will be
used in next patch where debugfs's show function can only pass
one private data pointer.

Signed-off-by: Nirmoy Das <nirmoy.das@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Remove unused function pointer typedef radeon_packet3_check_t

Remove the 'radeon_packet3_check_t' typedef as it is not used.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Chen Lin <chen.lin5@zte.com.cn>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: cleanup debugfs

Remove unnecessary debugfs dentries and also radeon_ttm_debugfs_fini()
as drm_debugfs_cleanup() will recursively remove debugfs files.

Signed-off-by: Nirmoy Das <nirmoy.das@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: device naming cleanup

Rename ttm_bo_device to ttm_device.
Rename ttm_bo_driver to ttm_device_funcs.
Rename ttm_bo_global to ttm_global.

Move global and device related functions to ttm_device.[ch].

No functional change.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/415222/

drm/radeon: also init GEM funcs in radeon_gem_prime_import_sg_table

Otherwise we will run into a NULL ptr deref.

Signed-off-by: Christian König <christian.koenig@amd.com>
Bug: https://bugzilla.kernel.org/show_bug.cgi?id=212137
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org # 5.11.x

drm/radeon: Store PCI controller in struct radeon_device.hose

Moves struct drm_device.hose into struct radeon_device. The field in
struct DRM device is only for legacy drivers.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210112081035.6882-6-tzimmermann@suse.de

drm/radeon: Remove references to struct drm_device.pdev

Using struct drm_device.pdev is deprecated. Convert radeon to struct
drm_device.dev. No functional changes.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Acked-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201201103542.2182-17-tzimmermann@suse.de

drm/ttm/drivers: remove unecessary ttm_module.h include v2

ttm_module.h deals with internals of TTM and should never
be include outside of it.

v2: also move the file around

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/404885/

drm/radeon: Move prototypes to shared header

Fixes the following W=1 kernel build warning(s):

drivers/gpu/drm/radeon/radeon_kms.c:756:5: warning: no previous prototype for ‘radeon_get_vblank_counter_kms’ [-Wmissing-prototypes]
756 | u32 radeon_get_vblank_counter_kms(struct drm_crtc *crtc)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/gpu/drm/radeon/radeon_kms.c:826:5: warning: no previous prototype for ‘radeon_enable_vblank_kms’ [-Wmissing-prototypes]
826 | int radeon_enable_vblank_kms(struct drm_crtc *crtc)
| ^~~~~~~~~~~~~~~~~~~~~~~~
drivers/gpu/drm/radeon/radeon_kms.c:853:6: warning: no previous prototype for ‘radeon_disable_vblank_kms’ [-Wmissing-prototypes]
853 | void radeon_disable_vblank_kms(struct drm_crtc *crtc)
| ^~~~~~~~~~~~~~~~~~~~~~~~~

Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: amd-gfx@lists.freedesktop.org
Cc: dri-devel@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Signed-off-by: Lee Jones <lee.jones@linaro.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/gem: Use struct dma_buf_map in GEM vmap ops and convert GEM backends

This patch replaces the vmap/vunmap's use of raw pointers in GEM object
functions with instances of struct dma_buf_map. GEM backends are
converted as well. For most of them, this simply changes the returned type.

TTM-based drivers now return information about the location of the memory,
either system or I/O memory. GEM VRAM helpers and qxl now use ttm_bo_vmap()
et al. Amdgpu, nouveau and radeon use drm_gem_ttm_vmap() et al instead of
implementing their own vmap callbacks.

v7:
* init QXL cursor to mapped BO buffer (kernel test robot)
v5:
* update vkms after switch to shmem
v4:
* use ttm_bo_vmap(), drm_gem_ttm_vmap(), et al. (Daniel, Christian)
* fix a trailing { in drm_gem_vmap()
* remove several empty functions instead of converting them (Daniel)
* comment uses of raw pointers with a TODO (Daniel)
* TODO list: convert more helpers to use struct dma_buf_map

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Acked-by: Christian König <christian.koenig@amd.com>
Tested-by: Sam Ravnborg <sam@ravnborg.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20201103093015.1063-7-tzimmermann@suse.de

drm/radeon: Stop changing the drm_driver struct

With only the kms driver left, we can fold this in. This means
we need to move the ioctl table, which means one additional ioctl
must be defined in headers.

Also there's a conflict between the radeon_init macro and the module
init function, so rename the module functions to avoid that.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: amd-gfx@lists.freedesktop.org
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20201104100425.1922351-1-daniel.vetter@ffwll.ch

drm/radeon: Add new callback that exposes vddc

This patch adds a callback for reporting vddc, to the dpm field of the radeon_asic structure.

Signed-off-by: Sandeep Raghuraman <sandy.8925@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: switch over to the new pin interface

Stop using TTM_PL_FLAG_NO_EVICT.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/391610/?series=81973&rev=1

drm/ttm: protect against reentrant bind in the drivers

This moves the generic tracking into the drivers and protects
against reentrancy in the drivers. It fixes up radeon and agp
to be able to query the bound status as that is required.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200917043040.146575-2-airlied@gmail.com

drm/radeon/ttm: move to driver binding/destroy functions. (v2)

Do agp decision in the driver, instead of special binding funcs

v2: use container_of, drop some {}.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200907204630.1406528-6-airlied@gmail.com

drm/ttm: rename ttm_mem_reg to ttm_resource.

This name better reflects what the object does. I didn't rename
all the pointers it seemed too messy.

Signed-off-by: Dave Airlie <airlied@redhat.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-60-airlied@gmail.com

drm/radeon: don't use ttm bo->offset

Calculate GPU offset in radeon_bo_gpu_offset without depending on
bo->offset.

Signed-off-by: Nirmoy Das <nirmoy.das@amd.com>
Reviewed-and-tested-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/372935/

drm/radeon: use mmu_interval_notifier_insert

The new API is an exact match for the needs of radeon.

For some reason radeon tries to remove overlapping ranges from the
interval tree, but interval trees (and mmu_interval_notifier_insert())
support overlapping ranges directly. Simply delete all this code.

Since this driver is missing a invalidate_range_end callback, but
still calls get_user_pages(), it cannot be correct against all races.

Link: https://lore.kernel.org/r/20191112202231.3856-8-jgg@ziepe.ca
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>

drm/radeon: use mmu_notifier_get/put for struct radeon_mn

radeon is using a device global hash table to track what mmu_notifiers
have been registered on struct mm. This is better served with the new
get/put scheme instead.

radeon has a bug where it was not blocking notifier release() until all
the BO's had been invalidated. This could result in a use after free of
pages the BOs. This is tied into a second bug where radeon left the
notifiers running endlessly even once the interval tree became
empty. This could result in a use after free with module unload.

Both are fixed by changing the lifetime model, the BOs exist in the
interval tree with their natural lifetimes independent of the mm_struct
lifetime using the get/put scheme. The release runs synchronously and just
does invalidate_start across the entire interval tree to create the
required DMA fence.

Additions to the interval tree after release are already impossible as
only current->mm is used during the add.

Link: https://lore.kernel.org/r/20190806231548.25242-9-jgg@ziepe.ca
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>

drm/radeon: handle PCIe root ports with addressing limitations

radeon uses a need_dma32 flag to indicate to the drm core that some
allocations need to be done using GFP_DMA32, but it only checks the
device addressing capabilities to make that decision. Unfortunately
PCIe root ports that have limited addressing exist as well. Use the
dma_addressing_limited instead to also take those into account.

Reviewed-by: Christian König <christian.koenig@amd.com>
Reported-by: Atish Patra <Atish.Patra@wdc.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

dma-buf: rename reservation_object to dma_resv

Be more consistent with the naming of the other DMA-buf objects.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/323401/

drm/radeon: use embedded gem object

Drop drm_gem_object from radeon_bo, use the
ttm_buffer_object.base instead.

Build tested only.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20190805140119.7337-5-kraxel@redhat.com

drm/ttm: initialize globals during device init (v2)

Make sure that the global BO state is always correctly initialized.

This allows removing all the device code to initialize it.

v2: fix up vbox (Alex)

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: use a static ttm_mem_global instance

As the name says we only need one global instance of ttm_mem_global.

Drop all the driver initialization and just use a single exported
instance which is initialized during BO global initialization.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: use pcie functions for link width

This is the last user of drm_pcie_get_speed_cap_mask. Use the pci
version so we can drop drm_pcie_get_speed_cap_mask.

Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: only enable swiotlb path when need v2

swiotlb expands our card accessing range, but its path always is slower
than ttm pool allocation.
So add condition to use it.
v2: move a bit later

Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20180209024410.1469-3-david1.zhou@amd.com

Revert "drm/radeon: Report doorbell configuration to amdkfd"

This reverts commit ebff8453d3a57a2405c4d96d9f9c4f4acc7d4d79. Unused
since amdkfd is no longer supported with radeon.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel@daenzer.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Revert "drm/radeon: adding synchronization for GRBM GFX"

This reverts commit 1c0a46255f8d7daf5b601668836e185fd1294e94. Not needed
anymore, since amdkfd is no longer supported with radeon.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel@daenzer.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: deprecate and remove KFD interface

To quote Felix: "For testing KV with current user mode stack, please use
amdgpu. I don't expect this to work with radeon and I'm not planning to
spend any effort on making radeon work with a current user mode stack."

Only compile tested, but should be straight forward.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

lib/interval_tree: fast overlap detection

Allow interval trees to quickly check for overlaps to avoid unnecesary
tree lookups in interval_tree_iter_first().

As of this patch, all interval tree flavors will require using a
'rb_root_cached' such that we can have the leftmost node easily
available. While most users will make use of this feature, those with
special functions (in addition to the generic insert, delete, search
calls) will avoid using the cached option as they can do funky things
with insertions -- for example, vma_interval_tree_insert_after().

[jglisse@redhat.com: fix deadlock from typo vm_lock_anon_vma()]
Link: http://lkml.kernel.org/r/20170808225719.20723-1-jglisse@redhat.com
Link: http://lkml.kernel.org/r/20170719014603.19029-12-dave@stgolabs.net
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
Acked-by: Christian König <christian.koenig@amd.com>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Doug Ledford <dledford@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Jason Wang <jasowang@redhat.com>
Cc: Christian Benvenuti <benve@cisco.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

drm/radeon: Fix preferred typo

Change "prefered" to "preferred"

Signed-off-by: Kent Russell <kent.russell@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Fix stolen typo

Change "stollen" to "stolen"

Signed-off-by: Kent Russell <kent.russell@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

sched/wait: Rename wait_queue_t => wait_queue_entry_t

Rename:

wait_queue_t => wait_queue_entry_t

'wait_queue_t' was always a slight misnomer: its name implies that it's a "queue",
but in reality it's a queue *entry*. The 'real' queue is the wait queue head,
which had to carry the name.

Start sorting this out by renaming it to 'wait_queue_entry_t'.

This also allows the real structure name 'struct __wait_queue' to
lose its double underscore and become 'struct wait_queue_entry',
which is the more canonical nomenclature for such data types.

Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>

drm/radeon: Make si_support and cik_support parameters always available

This will allow amdgpu-pro / other out-of-tree amdgpu builds to make use
of these options for using the out-of-tree amdgpu driver instead of the
in-tree radeon driver in a clean way.

Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>

drm/radeon: Add module param to control SI support

If AMDGPU supports SI, add a module parameter to control SI
support in radeon. It's on by default in radeon, while it will be
off by default in AMDGPU as long as SI support is experimental.

Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Michel Dänzer <michel.daenzer@amd.com>

drm/radeon: Cleanup pageflipping IRQ handling for evergreen, si

Same as the previous patch, but for pageflipping now. This also lets us
clear up the copy paste for vblank/vline IRQs.

Changes since v1:
- Preserve the order all registers are written back

Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Lyude <lyude@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Cleanup HDMI audio interrupt handling for evergreen

Same as the previous patch, but now for handling HDMI audio interrupts.

Changes since v1:
- Preserve the order we write back all registers

Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Lyude <lyude@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Cleanup display interrupt handling for evergreen, si

The current code here is really, really bad. A huge amount of it looks
to be copy pasted, it has some weird hatred of arrays and code sharing,
switch cases everywhere for things that really don't need them, and it
makes the file seem immensely more complex then it actually is. This is
a pain for maintanence, and is vulnerable to more weird irq handling
bugs.

So, let's start cleaning this up a bit. Modify all of the IRQ handlers
for evergreen/si so that they just use for loops. As well, we add a
helper function radeon_irq_kms_set_irq_n_enabled(), whose purpose is
just to update the state of registers that enable/disable interrupts
while printing any changes to the set of enabled interrupts to the
kernel log.

Note in this commit, since vblank/vline irq acking is intertwined with
page flip irq acking, we can't cut out all of the copy paste in
evergreen/si_irq_ack() just yet.

Changes since v1:
- Preserve order we write back all registers

Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Lyude <lyude@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Add module param to control CIK support

If AMDGPU supports CIK, add a module parameter to control CIK
support in radeon. It's off by default in radeon, while it will be
on by default in AMDGPU.

Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: fix include notation and remove -Iinclude/drm flag

Include <drm/*.h> instead of relative path from include/drm, then
remove the -Iinclude/drm compiler flag.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1493009447-31524-14-git-send-email-yamada.masahiro@socionext.com

drm/radeon: Maintain prime import/export refcount for BOs

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Christopher James Halse Rogers <christopher.halse.rogers@canonical.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

gpu: drm: amd/radeon: Convert printk(KERN_<LEVEL> to pr_<level>

Use a more common logging style.

Miscellanea:

o Coalesce formats and realign arguments
o Neaten a few macros now using pr_<level>

Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

dma-buf: Rename struct fence to dma_fence

I plan to usurp the short name of struct fence for a core kernel struct,
and so I need to rename the specialised fence/timeline for DMA
operations to make room.

A consensus was reached in
https://lists.freedesktop.org/archives/dri-devel/2016-July/113083.html
that making clear this fence applies to DMA operations was a good thing.
Since then the patch has grown a bit as usage increases, so hopefully it
remains a good thing!

(v2...: rebase, rerun spatch)
v3: Compile on msm, spotted a manual fixup that I broke.
v4: Try again for msm, sorry Daniel

coccinelle script:
@@

@@
- struct fence
+ struct dma_fence
@@

@@
- struct fence_ops
+ struct dma_fence_ops
@@

@@
- struct fence_cb
+ struct dma_fence_cb
@@

@@
- struct fence_array
+ struct dma_fence_array
@@

@@
- enum fence_flag_bits
+ enum dma_fence_flag_bits
@@

@@
(
- fence_init
+ dma_fence_init
|
- fence_release
+ dma_fence_release
|
- fence_free
+ dma_fence_free
|
- fence_get
+ dma_fence_get
|
- fence_get_rcu
+ dma_fence_get_rcu
|
- fence_put
+ dma_fence_put
|
- fence_signal
+ dma_fence_signal
|
- fence_signal_locked
+ dma_fence_signal_locked
|
- fence_default_wait
+ dma_fence_default_wait
|
- fence_add_callback
+ dma_fence_add_callback
|
- fence_remove_callback
+ dma_fence_remove_callback
|
- fence_enable_sw_signaling
+ dma_fence_enable_sw_signaling
|
- fence_is_signaled_locked
+ dma_fence_is_signaled_locked
|
- fence_is_signaled
+ dma_fence_is_signaled
|
- fence_is_later
+ dma_fence_is_later
|
- fence_later
+ dma_fence_later
|
- fence_wait_timeout
+ dma_fence_wait_timeout
|
- fence_wait_any_timeout
+ dma_fence_wait_any_timeout
|
- fence_wait
+ dma_fence_wait
|
- fence_context_alloc
+ dma_fence_context_alloc
|
- fence_array_create
+ dma_fence_array_create
|
- to_fence_array
+ to_dma_fence_array
|
- fence_is_array
+ dma_fence_is_array
|
- trace_fence_emit
+ trace_dma_fence_emit
|
- FENCE_TRACE
+ DMA_FENCE_TRACE
|
- FENCE_WARN
+ DMA_FENCE_WARN
|
- FENCE_ERR
+ DMA_FENCE_ERR
)
(
...
)

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Gustavo Padovan <gustavo.padovan@collabora.co.uk>
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20161025120045.28839-1-chris@chris-wilson.co.uk

drm/radeon: Provide page_flip_target hook

Now we can program a flip during a vertical blank period, if it's the
one targeted by the flip (or a later one). This allows simplifying
radeon_flip_work_func considerably.

Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

dma-buf/fence: make fence context 64 bit v2

Fence contexts are created on the fly (for example) by the GPU scheduler used
in the amdgpu driver as a result of an userspace request. Because of this
userspace could in theory force a wrap around of the 32bit context number
if it doesn't behave well.

Avoid this by increasing the context number to 64bits. This way even when
userspace manages to allocate a billion contexts per second it takes more
than 500 years for the context number to wrap around.

v2: fix printf formats as well.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Gustavo Padovan <gustavo.padovan@collabora.co.uk>
Acked-by: Sumit Semwal <sumit.semwal@linaro.org>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1464786612-5010-2-git-send-email-deathsimple@vodafone.de

drm/radeon: handle more than 10 UVD sessions

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Arindam Nath <arindam.nath@amd.com>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add support for loading new UVD fw

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Arindam Nath <arindam.nath@amd.com>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Support DRM_MODE_PAGE_FLIP_ASYNC

When this flag is set, we program the hardware to execute the flip
during horizontal blank (i.e. for the next scanline) instead of during
vertical blank (i.e. for the next frame).

Currently this is only supported on ASICs which have a page flip
completion interrupt (>= R600), and only if the use_pflipirq parameter
has value 2 (the default).

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: delete unused struct member suspend from radeon_device

Signed-off-by: Nils Wallménius <nils.wallmenius@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: hard reset r600 and newer GPU when hibernating.

Some GPU block like UVD and VCE require hard reset to be properly
resume if there is no real powerdown of the asic like during various
hibernation step. This patch perform such hard reset.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: allow to force hard GPU reset.

In some cases, like when freezing for hibernation, we need to be
able to force hard reset even if no engine are stuck. This patch
add a bool option to current asic reset callback to allow to force
hard reset on asic that supports it.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add driver option to disable vce block.

Quite few suspend/hibernation bugs are related to this block. Add
an option to disable those as a work around.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add driver option to disable uvd block.

Quite few suspend/hibernation bugs are related to this block. Add
an option to disable those as a work around.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add a vce flag to know if need to initialize vce or not.

This will later on serve for module option to disable vce.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Avoid double gpu reset by adding a timeout on IB ring tests.

When the radeon driver resets a gpu, it attempts to test whether all the
rings can successfully handle an IB. If these rings fail to respond, the
process will wait forever. Another gpu reset can't happen at this point,
as the current reset holds a lock required to do so. Instead, make all
the IB tests run with a timeout, so the system can attempt to recover
in this case.

While this doesn't fix the underlying issue with card resets failing, it
gives the system a higher chance of recovering. These timeouts have been
confirmed to help both a Tathi and Hawaii card recover after a gpu reset.

This also adds a new function, radeon_fence_wait_timeout, that behaves like
fence_wait_timeout. It is used instead of fence_wait_timeout as it continues
to work during a reset. radeon_fence_wait is changed to be implemented
using this function.

V2:
- Changed the timeout to 1s, as the default 10s from radeon_wait_timeout was
too long. A timeout of 100ms was tested and found to be too short.
- Changed radeon_fence_wait_timeout to behave more like fence_wait_timeout.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Matthew Dawson <matthew@mjdsystems.ca>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: cleaned up VCO output settings for DP audio

This is preparation for the fixes in the following patches.

Signed-off-by: Slava Grigorev <slava.grigorev@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: Fix "slow" audio over DP on DCE8+

DP audio is derived from the dfs clock.

Signed-off-by: Slava Grigorev <slava.grigorev@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: Retry DDC probing on DVI on failure if we got an HPD interrupt

HPD signals on DVI ports can be fired off before the pins required for
DDC probing actually make contact, due to the pins for HPD making
contact first. This results in a HPD signal being asserted but DDC
probing failing, resulting in hotplugging occasionally failing.

This is somewhat rare on most cards (depending on what angle you plug
the DVI connector in), but on some cards it happens constantly. The
Radeon R5 on the machine used for testing this patch for instance, runs
into this issue just about every time I try to hotplug a DVI monitor and
as a result hotplugging almost never works.

Rescheduling the hotplug work for a second when we run into an HPD
signal with a failing DDC probe usually gives enough time for the rest
of the connector's pins to make contact, and fixes this issue.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Lyude <cpaul@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: constify radeon_asic_ring structures

The radeon_asic_ring structures are never modified, so declare them as
const.

Done with the help of Coccinelle.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: don't try to recreate sysfs entries on resume

Fixes a harmless error message caused by:
51a4726b04e880fdd9b4e0e58b13f70b0a68a7f5

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: remove obsolete kfd_bo from radeon_device

Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>

radeon: Deinline indirect register accessor functions

This patch deinlines indirect register accessor functions.

These functions perform two mmio accesses, framed by spin lock/unlock.
Spin lock/unlock by itself takes more than 50 cycles in ideal case
(if lock is exclusively cached on current CPU).

With this .config: http://busybox.net/~vda/kernel_config,
after uninlining these functions have sizes and callsite counts
as follows:

r600_uvd_ctx_rreg: 111 bytes, 4 callsites
r600_uvd_ctx_wreg: 113 bytes, 5 callsites
eg_pif_phy0_rreg: 106 bytes, 13 callsites
eg_pif_phy0_wreg: 108 bytes, 13 callsites
eg_pif_phy1_rreg: 107 bytes, 13 callsites
eg_pif_phy1_wreg: 108 bytes, 13 callsites
rv370_pcie_rreg: 111 bytes, 21 callsites
rv370_pcie_wreg: 113 bytes, 24 callsites
r600_rcu_rreg: 111 bytes, 16 callsites
r600_rcu_wreg: 113 bytes, 25 callsites
cik_didt_rreg: 106 bytes, 10 callsites
cik_didt_wreg: 107 bytes, 10 callsites
tn_smc_rreg: 106 bytes, 126 callsites
tn_smc_wreg: 107 bytes, 116 callsites
eg_cg_rreg: 107 bytes, 20 callsites
eg_cg_wreg: 108 bytes, 52 callsites

Functions r100_mm_rreg() and r100_mm_rreg() have a fast path and
a locked (slow) path. This patch deinlines only slow path.

r100_mm_rreg_slow: 78 bytes, 2083 callsites
r100_mm_wreg_slow: 81 bytes, 3570 callsites

Reduction in code size is more than 65,000 bytes:

text data bss dec hex filename
85740176 22294680 20627456 128662312 7ab3b28 vmlinux.before
85674192 22294776 20627456 128598664 7aa4288 vmlinux

Signed-off-by: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: stop using addr to check for BO move

It is theoretically possible that a swapped out BO gets the
same GTT address, but different backing pages while being swapped in.

Instead just use another VA state to note updated areas.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Use DECLARE_BITMAP

Use the generic mechanism to declare a bitmap instead of unsigned long.

Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add VCE 1.0 support v4

Initial support for VCE 1.0 using newest firmware.

v2: rebased
v3: fix for TN
v4: fix FW size calculation

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: implement si_set_vce_clocks v2

For setting clocks with VCE v1.0

v2: (chk) rebased on current tree

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: rework VCE FW size calculation

Previously we were completely over allocating, fix this
by actually implementing the size calculation.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add a GPU reset counter queryable by userspace

Userspace will be able to tell whether a GPU reset occured by comparing
an old referece value of the counter with a new value.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: stop trying to suspend UVD sessions

Saving the current UVD state on suspend and restoring it on resume
just doesn't work reliable. Just close cleanup all sessions on suspend.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: allow creating overlapping userptrs

Similar to the Intel implementation, but instead of just falling back to a
global linear list when we have an overlapping userptr request we accumulate
all overlapping userptrs in a local list.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: fix 120hz handling harder

Need to expand the check to handle short circuiting
if the selected state is the same as current state.

bug:
https://bugs.freedesktop.org/show_bug.cgi?id=87796

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: add DisplayPort MST support (v2)

This adds initial DP 1.2 MST support to radeon, on CAYMAN
and up in theory.

This is off by default.

v2: agd5f:
- add UNIPHY3 offsets
- move atom cmd table code into atombios_encoders.c
- whitespace cleanup
- replace some magic numbers with proper defines

Signed-off-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

radeon: introduce a dp_work handler

This is to be called on short HPD irqs, just introduce
the basic infrastructure for it.

Signed-off-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: program auxch directly (v2)

The atombios tables have an unfortunate restriction on only
being able to write 12 bytes, MST really wants 16-bytes here,
and since the hw can do it, we should just write directly to it.

This uses a module option to allow for it now, and maybe
we should provide the old code as a fallback for a while.

v2: (agd5f)
- move registers to a proper register header
- only enable on DCE5+
- enable by default on DCE5+
- Switch pad to aux mode before using it
- reformat instance handling to better match the
rest of the driver

Signed-off-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add new callback for info ioctl register accessor

This adds a callback for each asic family to determine what
registers are allowed to be read back via the info ioctl.

The idea here is to allow usermode to query things like GPU status
registers or GPU harvest registers for profiling and determining
the gfx config.

Tested-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add new callbacks to get the current sclk/mclk

Needed to to expose the current clocks via the INFO ioctl.

Tested-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Remove rdev->gart.pages_addr array

radeon_vm_map_gart can use rdev->gart.pages_entry instead.

Also move the masking of the page address to radeon_vm_map_gart from its
callers.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Split off gart_get_page_entry ASIC hook from set_page_entry

get_page_entry calculates the GART page table entry, which is just written
to the GART page table by set_page_entry.

This is a prerequisite for the following fix.

Reviewed-by: Christian König <christian.koenig@amd.com>
Cc: stable@vger.kernel.org
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: make MMU_NOTIFIER optional

In cases where MMU_NOTIFIER is not available, userptr will not be
available. Similar to i915, although not making an exception for
CAP_SYS_ADMIN.

The proposed userspace patches for userptr seem to handle the fall-
back properly, so a userptr-less kernel should not be a problem.

Signed-off-by: Rob Clark <robdclark@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

radeon/audio: defined initial audio interface that gets initialized via detect() call

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Slava Grigorev <slava.grigorev@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

add common fan control asic callbacks

Adds 4 callbacks for managing fan state/speed
* fan_ctrl_set_mode - manages fan control mode (nonzero == manual)
* fan_ctrl_get_mode - queries fan control mode
* set_fan_speed_percent - manages fan speed as percentage value
* get_fan_speed_percent - queries fan speed as percentage value

Signed-off-by: Oleg Chernovskiy <algonkvel@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: use pointers instead of indexes for CS chunks

Nobody is interested at which index the chunk is. What's needed is
a pointer to the chunk. Remove unused chunk_id field as well.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: remove duplicates check

Completely unnecessary since the ww_mutex used to reserve a buffer
can detect double reservations from the same thread anyway.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add spinlock for BO_VA status protection (v2)

Stop using the VM mutex for this

v2: fix typo in comment

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: rename radeon_cs_reloc to radeon_bo_list

Better match what it is actually doing.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: drop the handle from radeon_cs_reloc

It's only used for duplicate check and that
can be done on the original as well.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon drop gobj from radeon_cs_reloc

It's only used once after initializing and that
ptr can be calculated from the BO as well.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: fence BO_VAs manually

This allows us to finally remove the VM fence and
so allow concurrent use of it from different engines.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: use one VMID for each ring

Use multiple VMIDs for each VM, one for each ring. That allows
us to execute flushes separately on each ring, still not ideal
cause in a lot of cases rings can share IDs.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: track VM update fences separately

Note for each fence if it's a VM page table update or not. This allows
us to determine the last VM update in a sync object and so to figure
out if we need to flush the TLB or not.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: split semaphore and sync object handling v2

Previously we just allocated space for four hardware semaphores
in each software semaphore object. Make software semaphore objects
represent only one hardware semaphore address again by splitting
the sync code into it's own object.

v2: fix typo in comment

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: rework vm_flush parameters

Use ring structure instead of index and provide vm_id and pd_addr separately.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add smc fan control for CI (v2)

Enable smc fan control for CI boards. Should
reduce the fan noise on systems with a higher
default fan profile.

v2: disable by default, add additional fan setup, rpm control

bug:
https://bugs.freedesktop.org/show_bug.cgi?id=73338

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: grab fan info from vbios

Required for fan control support.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Try placing NO_CPU_ACCESS BOs outside of CPU accessible VRAM

This avoids them getting in the way of BOs which might be accessed by
the CPU. They can still go to the CPU accessible part of VRAM though if
there's no space outside of it.

Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: use gart memory for DMA ring tests

Avoids HDP cache flush issues when using vram which can
cause ring test failures on certain boards.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: Alexander Fyodorov <halcy@yandex.ru>
Cc: stable@vger.kernel.org

drm/radeon: cope with foreign fences inside the reservation object

Not the whole world is a radeon! :-)

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: cope with foreign fences inside display

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: split audio enable between eg and r600 (v2)

Clean up the enable sequence as well.

V2: clean up duplicate defines

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm: Extract <drm/drm_gem.h>

v2: Don't forget git add, noticed by David.

Cc: David Herrmann <dh.herrmann@gmail.com>

Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Acked-by: David Herrmann <dh.herrmann@gmail.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: add a module parameter for backlight control (v2)

Add a module parameter to disable the radeon GPU backlight
controller to override the automatic detection. Some
laptops seems to indicate that they use the integrated
controller, but appear to actually use an external
controller.

bug:
https://bugs.freedesktop.org/show_bug.cgi?id=81382

v2: fix module parameter description

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add the infrastructure for concurrent buffer access

This allows us to specify if we want to sync to
the shared fences of a reservation object or not.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm: drop DRM_DEBUG_CODE

DRM_DEBUG_CODE is currently always set, so distributions enable it. The
only reason to keep support in code is if developers wanted to disable
debug support. Sounds unlikely.

All the DRM_DEBUG() printks are still guarded by a drm_debug read. So if
its cacheline is read once, they're discarded pretty fast.. There should
hardly be any performance penalty, it's even guarded by unlikely().

Signed-off-by: David Herrmann <dh.herrmann@gmail.com>
Reviewed-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: use common fence implementation for fences, v4

Changes since v1:
- Kill the sw interrupt dance, add and use
radeon_irq_kms_sw_irq_get_delayed instead.
- Change custom wait function, lockdep complained about it.
Holding exclusive_lock in the wait function might cause deadlocks.
Instead do all the processing in .enable_signaling, and wait
on the global fence_queue to pick up gpu resets.
- Process all fences in radeon_gpu_reset after reset to close a race
with the trylock in enable_signaling.
Changes since v2:
- Small changes to work with the rewritten lockup recovery patches.
Changes since v3:
- Call radeon_fence_schedule_check when exclusive_lock cannot be
acquired to always cause a wake up.
- Reset irqs from hangup check.
- Drop reading seqno in the callback, use cached value.
- Fix indentation in radeon_fence_default_wait
- Add a radeon_test_signaled function, drop a few test_bit calls.
- Make to_radeon_fence global.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: Christian König <christian.koenig@amd.com>

drm/radeon: allow UVD to use a second 256MB segment

This improves concurrent stream decoding.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: drop doing resets in a work item

Blocking completely innocent processes with a GPU reset is
a pretty bad idea. Just set needs_reset and let the next
command submission or fence wait do the job.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: drop RADEON_FENCE_SIGNALED_SEQ v2

It's causing issues with VMID handling and comparing the
fence value two times actually doesn't make handling faster.

v2: rebased on reset changes

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: handle lockup in delayed work, v5

v5 (chk): complete rework, start when the first fence is emitted,
stop when the last fence is signalled, make it work
correctly with GPU resets, cleanup radeon_fence_wait_seq

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: take exclusive_lock in read mode during ring tests, v5

This is needed for the next commit, because the lockup detection
will need the read lock to run.

v4 (chk): split out forced fence completion, remove unrelated changes,
add and handle in_reset flag
v5 (agd5f): rebase fix

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: force fence completion only on problematic rings (v2)

Instead of resetting all fence numbers, only reset the
number of the problematic ring. Split out from a patch
from Maarten Lankhorst <maarten.lankhorst@canonical.com>

v2 (agd5f): rebase build fix

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: move fpfn and lpfn into each placement v2

This allows us to more fine grained specify where to place the buffer object.

v2: rebased on drm-next, add bochs changes as well

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: save/restore the PD addr on suspend/resume

This fixes a problem with GPU resets and TLB flushes on SI/CIK.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Only flush HDP cache for indirect buffers from userspace

It isn't necessary for command streams generated by the kernel (at least
not while we aren't storing ring or indirect buffers in VRAM).

Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add bapm module parameter

Add a module paramter to enable bapm on APUs. It's disabled
by default on certain APUs due to stability issues. This
option makes it easier to test and to enable it on systems that
are stable.

bug:
https://bugzilla.kernel.org/show_bug.cgi?id=81021

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: add userptr flag to register MMU notifier v3

Whenever userspace mapping related to our userptr change
we wait for it to become idle and unmap it from GTT.

v2: rebased, fix mutex unlock in error path
v3: improve commit message

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add userptr support v8

This patch adds an IOCTL for turning a pointer supplied by
userspace into a buffer object.

It imposes several restrictions upon the memory being mapped:

1. It must be page aligned (both start/end addresses, i.e ptr and size).

2. It must be normal system memory, not a pointer into another map of IO
space (e.g. it must not be a GTT mmapping of another object).

3. The BO is mapped into GTT, so the maximum amount of memory mapped at
all times is still the GTT limit.

4. The BO is only mapped readonly for now, so no write support.

5. List of backing pages is only acquired once, so they represent a
snapshot of the first use.

Exporting and sharing as well as mapping of buffer objects created by
this function is forbidden and results in an -EPERM.

v2: squash all previous changes into first public version
v3: fix tabs, map readonly, don't use MM callback any more
v4: set TTM_PAGE_FLAG_SG so that TTM never messes with the pages,
pin/unpin pages on bind/unbind instead of populate/unpopulate
v5: rebased on 3.17-wip, IOCTL renamed to userptr, reject any unknown
flags, better handle READONLY flag, improve permission check
v6: fix ptr cast warning, use set_page_dirty/mark_page_accessed on unpin
v7: add warning about it's availability in the API definition
v8: drop access_ok check, fix VM mapping bits

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v4)
Reviewed-by: Jérôme Glisse <jglisse@redhat.com> (v4)
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Use pflip irqs for pageflip completion if possible. (v2)

Skip the "manual" pageflip completion checks via polling and
guessing in the vblank handler radeon_crtc_handle_vblank() on
asics which are known to reliably support hw pageflip completion
irqs. Those pflip irqs are a more reliable and race-free method
of handling pageflip completion detection, whereas the "classic"
polling method has some small races in combination with dpm on,
and with the reworked pageflip implementation since Linux 3.16.

On old asics without pflip irqs, the classic method is used.

On asics with known good pflip irqs, only pflip irqs are used
by default, but a new module parameter "use_pflipirqs" allows to
override this in case we encounter asics in the wild with
unreliable or faulty pflip irqs. A module parameter of 0 allows
to use the classic method only in such a case. A parameter of 1
allows to use both classic method and pflip irqs as additional
band-aid to avoid some small races which could happen with the
classic method alone. The setting 1 gives Linux 3.16 behaviour.

Hw pflip irqs are available since R600.

Tested on DCE-4, AMD Cedar - FirePro 2270.

v2: agd5f: only enable pflip interrupts on DCE4+ as they are not
reliable on older asics.

Signed-off-by: Mario Kleiner <mario.kleiner.de@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: split PT setup in more functions

Move the decision what to use into the common VM code.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: use an intervall tree to manage the VMA v2

Scales much better than scanning the address range linearly.

v2: store pfn instead of address

Signed-off-by: Christian König <christian.koenig@amd.com>
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: invalidate moved BOs in the VM (v2)

Don't wait for the BO to be used again, just
update the PT on the next VM use.

v2: remove stray semicolon.

Signed-off-by: Christian König <christian.koenig@amd.com>
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/atom: add new voltage fetch function for hawaii

Some hawaii boards use a different method for fetching the
voltage information from the vbios.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: Always flush the HDP cache before submitting a CS to the GPU

This ensures the GPU sees all previous CPU writes to VRAM, which makes it
safe:

* For userspace to stream data from CPU to GPU via VRAM instead of GTT
* For IBs to be stored in VRAM instead of GTT
* For ring buffers to be stored in VRAM instead of GTT, if the HPD flush
is performed via MMIO

Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: s/ioctl_wait_idle/mmio_hpd_flush/

And clean up the function comment a little.

Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: fix R600_PTE_GART handling

That didn't worked correctly any more and opened up a security problem.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: remove discardable flag from radeon_gem_object_create

Unused and unimplemented. Also fix specifying the
kernel flag incorrectly at one occasion.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add a PX quirk list

Some PX laptops seems to have problems turning the dGPU on/off.
Add a quirk list to disable runpm by default on those systems.
Also convert the current PX d3 delay handling to a quirk.

bug:
https://bugzilla.kernel.org/show_bug.cgi?id=51381
https://bugzilla.kernel.org/show_bug.cgi?id=74551

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: remove visible vram size limit on bo allocation (v4)

Now that fallback to gtt is fixed for cpu access, we can
remove this limit.

bug:
https://bugs.freedesktop.org/show_bug.cgi?id=78717

v2: use new gart_pin_size to accurately track available gtt.
v3: fix comment
v4: clarify comment

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>

drm/radeon: track pinned memory (v2)

So we know how large an allocation we can allow.

v2: incorporate Michel's comments

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>

drm/radeon: Allow write-combined CPU mappings of BOs in GTT (v2)

v2: fix rebase onto drm-fixes

Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Pass GART page flags to radeon_gart_set_page() explicitly

Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Remove radeon_gart_restore()

Doesn't seem necessary, the GART table memory should be persistent.

Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Inline r100_mm_rreg, -wreg, v3

This was originally un-inlined by Andi Kleen in 2011 citing size concerns.
Indeed, a first attempt at inlining it grew radeon.ko by 7%.

However, 2% of cpu is spent in this function. Simply inlining it gave 1% more fps
in Urban Terror.

v2: We know the minimum MMIO size. Adding it to the if allows the compiler to
optimize the branch out, improving both performance and size.

The v2 patch decreases radeon.ko size by 2%. I didn't re-benchmark, but common sense
says perf is now more than 1% better.

v3: Also change _wreg, make the threshold a define.

Inlining _wreg increased the size a bit compared to v2, so now radeon.ko
is only 1% smaller.

Signed-off-by: Lauri Kasanen <cand@gmx.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/cik: Add support for new ucode format (v5)

This adds CIK support for the new ucode format.

v2: add size validation, integrate debug info
v3: add support for MEC2 on KV
v4: fix typos
v4: update to latest format

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/si: Add support for new ucode format (v3)

This adds SI support for the new ucode format.

v2: add size validation, integrate debug info
v3: update to latest version

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add support for SVI2 voltage for SI

Some newer boards use SVI2 for voltage control rather
than GPIO.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: fix VM IB handling

Calling radeon_vm_bo_find on the IB BO during CS
is illegal and can lead to an crash.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: fix handling of radeon_vm_bo_rmv v3

v3: completely rewritten. We now just remember which areas
of the PT to clear and do so on the next command submission.

Bug: https://bugs.freedesktop.org/show_bug.cgi?id=79980

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Move pinning the BO back to radeon_crtc_page_flip()

As well as enabling the vblank interrupt. These shouldn't take any
significant amount of time, but at least pinning the BO has actually been
seen to fail in practice before, in which case we need to let userspace
know about it.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Add radeon <--> amdkfd interface

This patch adds the interface between the radeon driver and the amdkfd driver.
The interface implementation is contained in radeon_kfd.c and radeon_kfd.h.

The interface itself is represented by a pointer to struct
kfd_dev. The pointer is located inside radeon_device structure.

All the register accesses that amdkfd need are done using this interface. This
allows us to avoid direct register accesses in amdkfd proper, while also
avoiding locking between amdkfd and radeon.

The single exception is the doorbells that are used in both of the drivers.
However, because they are located in separate pci bar pages, the danger of
sharing registers between the drivers is minimal.

Having said that, we are planning to move the doorbells as well to radeon.

v3:

Add interface for sa manager init and fini. The init function will allocate a
buffer on system memory and pin it to the GART address space via the radeon sa
manager.

All mappings of buffers to GART address space are done via the radeon sa
manager. The interface of allocate memory will use the radeon sa manager to sub
allocate from the single buffer that was allocated during the init function.

Change lower_32/upper_32 calls to use linux macros

Add documentation for the interface

v4:

Change ptr field type in kgd_mem from uint32_t* to void* to match to type that
is returned by radeon_sa_bo_cpu_addr

v5:

Change format of mqd structure to work with latest KV firmware
Add support for AQL queues creation to enable working with open-source HSA
runtime.
Move generic kfd-->kgd interface and other generic kgd definitions to a generic
header file that will be used by AMD's radeon and amdgpu drivers

Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>

drm/radeon: adding synchronization for GRBM GFX

Implementing a lock for selecting and accessing shader engines and arrays.
This lock will make sure that radeon and amdkfd are not colliding when
accessing shader engines and arrays with GRBM_GFX_INDEX register.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>

drm/radeon: use RADEON_MAX_CRTCS, RADEON_MAX_AFMT_BLOCKS (v2)

v2: agd5f: compile fix

Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add a module parameter to control deep color support

Some monitors seem to have problems with deep color enabled, even
though they claim to support it. I'm not sure if the monitor
need a quirk or if the driver is doing something the monitor doesn't
like. At this point lets just disable deep color by default like
we did for hdmi audio and work through the bugs so we can eventually
enable it by default.

bug:
https://bugs.freedesktop.org/show_bug.cgi?id=80531

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add query for number of active CUs

Query to find out how many compute units on a GPU.
Useful for OpenCL usermode drivers.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: make vm_block_size a module parameter

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: make VM size a module parameter (v2)

v2: agd5f: simplify patch

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: rename alt_domain to allowed_domains

And also domain to prefered_domains. That matches better
what those values represent.

Signed-off-by: Christian König <christian.koenig@amd.com>
Cc: Marek Olšák <maraeo@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: remove range check from *_gart_set_page

We never check the return value anyway and if the
index isn't valid would crash way before calling
the functions.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: rework page flip handling v4

Instead of trying to flip inside the vblank period when
the buffer is idle, offload blocking for idle to a kernel
thread and program the flip directly into the hardware.

v2: add error handling, fix EBUSY handling
v3: add proper exclusive_lock handling
v4: update crtc->primary->fb when the flip actually happens

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

Revert "drm/radeon: rework page flip handling v3"

This reverts commit 1aab5514ca9604e0263f658a067da0189c86a35b.

Apply the fixed up version instead.

drm/radeon: add define for flags used in R600+ GTT

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: rework page flip handling v3

Instead of trying to flip inside the vblank period when
the buffer is idle, offload blocking for idle to a kernel
thread and program the flip directly into the hardware.

v2: add error handling, fix EBUSY handling
v3: add proper exclusive_lock handling

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: split page flip and pending callback

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: remove (pre|post)_page_flip callbacks

They are doing the same on all generations anyway.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add large PTE support for NI, SI and CIK v5

This patch implements support for VRAM page table entry compression.
PTE construction is enhanced to identify physically contiguous page
ranges and mark them in the PTE fragment field. L1/L2 TLB support is
enabled for 64KB (SI/CIK) and 256KB (NI) PTE fragments, significantly
improving TLB utilization for VRAM allocations.

Linear store bandwidth is improved from 60GB/s to 125GB/s on Pitcairn.
Unigine Heaven 3.0 sees an average improvement from 24.7 to 27.7 FPS
on default settings at 1920x1200 resolution with vsync disabled.

See main comment in radeon_vm.c for a technical description.

v2 (chk): rebased and simplified.
v3 (chk): add missing hw setup
v4 (chk): rebased on current drm-fixes-3.15
v5 (chk): fix comments and commit text

Signed-off-by: Jay Cornwall <jay@jcornwall.me>
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: fix DCE83 check for mullins

Mullins is DCE83 just like Kabini. Set the proper number
of endpoints on mullins.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>

drm/radeon: check VCE relocation buffer range v3

v2 (chk): fix image size storage
v3 (chk): fix UV size calculation

Signed-off-by: Leo Liu <leo.liu@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>

drm/radeon: use pflip irq on R600+ v2

Testing the update pending bit directly after issuing an
update is nonsense cause depending on the pixel clock the
CRTC needs a bit of time to execute the flip even when we
are in the VBLANK period.

This is just a non invasive patch to solve the problem at
hand, a more complete and cleaner solution should follow
in the next merge window.

Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=76564

v2: fix source IDs for CRTC2-6

Signed-off-by: Christian König <christian.koenig@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: fix runpm handling on APUs (v4)

Don't try and runtime suspend the APU in PX systems. We
only want to power down the dGPU.

v2: fix harder
v3: fix stupid typo
v4: consolidate runpm enablement to a single flag

bugs:
https://bugs.freedesktop.org/show_bug.cgi?id=75127
https://bugzilla.kernel.org/show_bug.cgi?id=72701

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: fix audio pin counts for DCE6+ (v2)

There is actually quite a bit of variance based on
the asic.

v2: fix typo noticed by Jerome.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: remove struct radeon_bo_list

Just move all fields into radeon_cs_reloc, removing unused/duplicated fields.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: remove global vm lock

Not needed any more.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: use normal BOs for the page tables v4

No need to make it more complicated than necessary,
just allocate the page tables as normal BO and
flush whenever the address change.

v2: update comments and function name
v3: squash bug fixes, page directory and tables patch
v4: rebased on Mareks changes

Signed-off-by: Christian König <christian.koenig@amd.com>

drm/radeon: further cleanup vm flushing & fencing

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: fix VCE suspend/resume

Signed-off-by: Christian König <christian.koenig@amd.com>

drm/radeon: validate relocations in the order determined by userspace v3

Userspace should set the first 4 bits of drm_radeon_cs_reloc::flags to
a number from 0 to 15. The higher the number, the higher the priority,
which means a buffer with a higher number will be validated sooner.

The old behavior is preserved: Buffers used for write are prioritized over
read-only buffers if the userspace doesn't set the number.

v2: add buffers to buckets directly, then concatenate them
v3: use a stable sort

Signed-off-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>

drm/radeon: track memory statistics about VRAM and GTT usage and buffer moves v2

The statistics are:
- VRAM usage in bytes
- GTT usage in bytes
- number of bytes moved by TTM

The last one is actually a counter, so you need to sample it before and after
command submission and take the difference.

This is useful for finding performance bottlenecks. Userspace queries are
also added.

v2: use atomic64_t

Signed-off-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>

drm/radeon: add a way to get and set initial buffer domains v2

When passing buffers between processes, the receiving process needs to know
the original buffer domain, so that it doesn't accidentally move the buffer.

v2: reserve the buffer

Signed-off-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>

drm/radeon: cleanup the fence ring locking code

We no longer need to take the ring lock while checking for
a gpu lockup, so just cleanup the code.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: improve ring lockup detection code v2

Use atomics and jiffies_64, so that we don't need to have the
ring mutex locked any more and avoid wrap arounds.

v2: fix some checkpatch warnings

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: change audio enable logic

Disable audio around audio hw setup. This may avoid
hangs on certain asics.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>

drm/radeon: fix CP semaphores on CIK

The CP semaphore queue on CIK has a bug that triggers if uncompleted
waits use the same address while a signal is still pending. Work around
this by using different addresses for each sync.

Signed-off-by: Christian König <christian.koenig@amd.com>
Cc: stable@vger.kernel.org
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: drop radeon_ring_force_activity

The reason for the false positives was fixed quite some time ago and since
most engines can still execute NOPs while being locked up it leads to false
negatives.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: drop drivers copy of the rptr

In all cases where it really matters we are using the read functions anyway.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: enable dynamic vce state switching v2

enable vce states when vce is active. When vce is active,
it adjusts the currently selected state (performance, battery,
uvd, etc.)

v2: add code comments

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>

drm/radeon/dpm: fetch vce states from the vbios

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: fill in some initial vce infrastructure

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add callback for setting vce clocks

Similar to uvd clock setting.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add VCE version parsing and checking

Also make the result available to userspace.

Signed-off-by: Christian König <christian.koenig@amd.com>

drm/radeon: initial VCE support v4

Only VCE 2.0 support so far.

v2: squashing multiple patches into this one
v3: add IRQ support for CIK, major cleanups,
basic code documentation
v4: remove HAINAN from chipset list

Signed-off-by: Christian König <christian.koenig@amd.com>

drm/radeon: fix CP semaphores on CIK

The CP semaphore queue on CIK has a bug that triggers if uncompleted
waits use the same address while a signal is still pending. Work around
this by using different addresses for each sync.

Signed-off-by: Christian König <christian.koenig@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: fix VMID use tracking

Otherwise we allocate a new VMID on nearly every submit.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Report doorbell configuration to amdkfd

radeon and amdkfd share the doorbell aperture.
radeon sets it up, takes the doorbells required for its own rings
and reports the setup to amdkfd.
radeon reserved doorbells are at the start of the doorbell aperture.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>

drm/radeon: add pci config hard reset

This is used to hard reset the asic. If a soft
reset is not able to reset things, a hard reset
can be used.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add hard_reset module parameter

Enabling this parameter enables pci config reset,
aka hard reset, which is a bus level chip reset.
In some cases this works more reliably than a soft
reset. Disabled by default.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: remove generic rptr/wptr functions (v2)

Fill in asic family specific versions rather than
using the generic version. This lets us handle asic
specific differences more easily. In this case, we
disable sw swapping of the rtpr writeback value on
r6xx+ since the hw does it for us. Fixes bogus
rptr readback on BE systems.

v2: remove missed cpu_to_le32(), add comments

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add a late enable callback

Certain features need to be enabled after ring tests
(e.g., powergating, etc.). Add a function pointer
to split out late enable features.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add GART debugfs access v3

v2: add default_llseek
v3: set inode size in the open callback

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add VRAM debugfs access v3

Not very fast, but makes it possible to access even the
normally inaccessible parts of VRAM from userspace.

v2: use MM_INDEX_HI for >2GB mem access, add default_llseek
v3: set inode size in the open callback

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: expose render backend mask to the userspace

This will allow userspace to correctly program the PA_SC_RASTER_CONFIG
register, so it can be considered a fix.

Signed-off-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: add radeon_vm_bo_update trace point

Also rename the function to better reflect what it is doing.

agd5f: fix argument size warning

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/cik: Add macrotile mode array query

This is required to properly calculate the tiling parameters
in userspace.

Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: use a single doorbell for cik kms compute

A single doorbell page is plenty for cik kms compute.
Use a single page and manage doorbell allocation by
individual doorbells rather than pages. Identify
doorbells by their index rather than byte offset.

Signed-off-by: Andrew Lewycky <Andrew.Lewycky@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: allow semaphore emission to fail

To workaround bugs and/or certain limits it's sometimes
useful to fall back to waiting on fences.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: fixup locking inversion between, mmap_sem and reservations

op 08-10-13 18:58, Thomas Hellstrom schreef:
> On 10/08/2013 06:47 PM, Jerome Glisse wrote:
>> On Tue, Oct 08, 2013 at 06:29:35PM +0200, Thomas Hellstrom wrote:
>>> On 10/08/2013 04:55 PM, Jerome Glisse wrote:
>>>> On Tue, Oct 08, 2013 at 04:45:18PM +0200, Christian König wrote:
>>>>> Am 08.10.2013 16:33, schrieb Jerome Glisse:
>>>>>> On Tue, Oct 08, 2013 at 04:14:40PM +0200, Maarten Lankhorst wrote:
>>>>>>> Allocate and copy all kernel memory before doing reservations. This prevents a locking
>>>>>>> inversion between mmap_sem and reservation_class, and allows us to drop the trylocking
>>>>>>> in ttm_bo_vm_fault without upsetting lockdep.
>>>>>>>
>>>>>>> Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
>>>>>> I would say NAK. Current code only allocate temporary page in AGP case.
>>>>>> So AGP case is userspace -> temp page -> cs checker -> radeon ib.
>>>>>>
>>>>>> Non AGP is directly memcpy to radeon IB.
>>>>>>
>>>>>> Your patch allocate memory memcpy userspace to it and it will then be
>>>>>> memcpy to IB. Which means you introduce an extra memcpy in the process
>>>>>> not something we want.
>>>>> Totally agree. Additional to that there is no good reason to provide
>>>>> anything else than anonymous system memory to the CS ioctl, so the
>>>>> dependency between the mmap_sem and reservations are not really
>>>>> clear to me.
>>>>>
>>>>> Christian.
>>>> I think is that in other code path you take mmap_sem first then reserve
>>>> bo. But here we reserve bo and then we take mmap_sem because of copy
>>> >from user.
>>>> Cheers,
>>>> Jerome
>>>>
>>> Actually the log message is a little confusing. I think the mmap_sem
>>> locking inversion problem is orthogonal to what's being fixed here.
>>>
>>> This patch fixes the possible recursive bo::reserve caused by
>>> malicious user-space handing a pointer to ttm memory so that the ttm
>>> fault handler is called when bos are already reserved. That may
>>> cause a (possibly interruptible) livelock.
>>>
>>> Once that is fixed, we are free to choose the mmap_sem ->
>>> bo::reserve locking order. Currently it's bo::reserve->mmap_sem(),
>>> but the hack required in the ttm fault handler is admittedly a bit
>>> ugly. The plan is to change the locking order to
>>> mmap_sem->bo::reserve
>>>
>>> I'm not sure if it applies to this particular case, but it should be
>>> possible to make sure that copy_from_user_inatomic() will always
>>> succeed, by making sure the pages are present using
>>> get_user_pages(), and release the pages after
>>> copy_from_user_inatomic() is done. That way there's no need for a
>>> double memcpy slowpath, but if the copied data is very fragmented I
>>> guess the resulting code may look ugly. The get_user_pages()
>>> function will return an error if it hits TTM pages.
>>>
>>> /Thomas
>> get_user_pages + copy_from_user_inatomic is overkill. We should just
>> do get_user_pages which fails with ttm memory and then use copy_highpage
>> helper.
>>
>> Cheers,
>> Jerome
> Yeah, it may well be that that's the preferred solution.
>
> /Thomas
>
I still disagree, and shuffled radeon_ib_get around to be called sooner.

How does the patch below look?
8<-------
Allocate and copy all kernel memory before doing reservations. This prevents a locking
inversion between mmap_sem and reservation_class, and allows us to drop the trylocking
in ttm_bo_vm_fault without upsetting lockdep.

Changes since v1:
- Kill extra memcpy for !AGP case.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: drop CP page table updates & cleanup v2

The DMA ring seems to be stable now.

v2: remove pt_ring_index as well

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: rework and fix reset detection v2

Stop fiddling with jiffies, always wait for RADEON_FENCE_JIFFIES_TIMEOUT.
Consolidate the two wait sequence implementations into just one function.
Activate all waiters and remember if the reset was already done instead of
trying to reset from only one thread.

v2: clear reset flag earlier to avoid timeout in IB test

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add runtime PM support (v2)

This hooks radeon up to the runtime PM system to enable
dynamic power management for secondary GPUs in switchable
and powerxpress laptops.

v2: agd5f: clean up, add module parameter

Signed-off-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: convert to pmops

This is a pre-requisite for runtime pm on powerxpress systems.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: fix incompatible casting on big endian

We use u16 for voltage values throughout the driver so switch
the table values to a u16 as well. Fixes an incompatible
cast error in ci_patch_clock_voltage_limits_with_vddc_leakage()
picked up by coverity.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add infrastructure to properly handle bapm

bapm is a pm feature for sharing the power budget between
the GPU and the CPU on APUs. It needs to be enabled or
disabled in certain circumstances. For now, disable it
when on battery and enable it when on AC power.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: fix typo in PG flags

s/CG/PG/ in the GFX powergating flag name.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add spinlocks for indirect register accesss

This adds spinlocks to protect access to other
indirect register apertures. These indirect spaces are
used pretty infrequently and we haven't had an reported
problems, but better safe than sorry.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: protect concurrent smc register access with a spinlock

smc registers are access indirectly via the main mmio aperture, so
there may be problems with concurrent access. This adds a spinlock
to protect access to this register space.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/si: restructure cg code (v3)

Resturcture clockgating code so that it can be
enabled/disabled from other components such as
dpm.

v2: make function static
v3: add fine grained cg controls

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add cg and pg flags

This commits adds flags for supported clockgating and
powergating features. This allows us to more easily
track which features are supported on a particular
asic and to enable/disable features for debugging.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add audio support for DCE6/8 GPUs (v12)

Similar to DCE4/5, but supports multiple audio pins
which can be assigned per afmt block.

v2: rework the driver to handle more than one audio
pin.
v3: try different dto reg
v4: properly program dto
v5 (ck): change dto programming order
v6: program speaker allocation block
v7: rebase
v8: rebase on Rafał's changes
v9: integrated Rafał's comments, update to latest
drm_edid_to_speaker_allocation API
v10: add missing line break in error message
v11: add back audio enabled messages
v12: fix copy paste typo in r600_audio_enable

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Rafał Miłecki <zajec5@gmail.com>

drm/radeon: remove special handling for the DMA ring

Now that we have callbacks for [rw]ptr handling we can
remove the special handling for the DMA rings and use
the callbacks instead.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: rework UVD writeback & [rw]ptr handling

The hardware just doesn't support this correctly.
Disable it before we accidentally write anywhere we shouldn't.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: rework ring function handling

Give the ring functions a separate structure and let the asic
structure point to the ring specific functions. This simplifies
the code and allows us to make changes at only one point.

No change in functionality.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add new callback for powergating UVD (v4)

Starting on CIK, multi-media blocks like UVD no longer
have special power state. Rather they have their own
DPM implementation which adjusts their clocks dynamically
when active. When they are not active, the blocks are
powergated to save power.

v2: add missing pm locks
v3: rebase on uvd state selection rework
v4: fix inverted logic typo noticed by Christian

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add dpm support for CI dGPUs (v2)

This adds dpm support for btc asics. This includes:
- dynamic engine clock scaling
- dynamic memory clock scaling
- dynamic voltage scaling
- dynamic pcie gen switching

Set radeon.dpm=1 to enable.

v2: remove unused radeon_atombios.c changes,
make missing smc ucode non-fatal

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add vce clocks to radeon_ps

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add clock voltage dep tables for acp, samu

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add structs to store vce clock voltage deps

Used for vce power management.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: grab mvdd_dependency_on_mclk info from vbios

Required for dpm on CI.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add support for parsing the atom powertune table

Needed for DPM on CI.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: update cac leakage table parsing for CI

Uses a different table format if the board supports EVV.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add support for thermal controller on KB/KV

No support for reading temperature back yet.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add structs to store uvd clock voltage deps

Used for uvd power management.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add indirect accessors for dift registers on CIK

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: implement clock and power gating for CIK (v3)

Only the APUs support power gating.

v2: disable cgcg for now
v3: workaround hw issue in mgcg

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: convert SI,CIK to use sumo_rlc functions

and remove duplicate si_rlc functions.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: rework thermal state handling

1. Handle the the thermal state directly in the work handler.
Remove the state selection function since nothing else uses it now.
2. On some asics there is no thermal state, so we just use a regular
state and force the low performance state.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: use multiple UVD power states (v3)

Use the UVD handle information to determine which
which power states to select when using UVD. For
example, decoding a single SD stream requires much
lower clocks than multiple HD streams.

v2: switch to a cleaner dpm/uvd interface
v3: change the uvd power state while streams
are active if need be

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add UVD->DPM helper function (v5)

Add a helper function for counting the number of open stream handles.

v2: fix copy-pasta in comments and whitespace error
v3: make function static since it's only used in radeon_uvd.c
at the moment
v4: make non-static again for future changes
v5: make static again for new rework of dpm uvd changes

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/kms: remove r6xx+ blit copy routines

No longer used now that we use the async dma engines or
CP DMA for bo copies.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: fix WREG32_OR macro setting bits in a register

This bug (introduced in 3.10) in WREG32_OR made
commit d3418eacad403033e95e49dc14afa37c2112c134
"drm/radeon/evergreen: setup HDMI before enabling it"
cause a regression. Sometimes audio over HDMI wasn't working, sometimes
display was corrupted.

This fixes:
https://bugzilla.kernel.org/show_bug.cgi?id=60687
https://bugzilla.kernel.org/show_bug.cgi?id=60709
https://bugs.freedesktop.org/show_bug.cgi?id=67767

Signed-off-by: Rafał Miłecki <zajec5@gmail.com>
Cc: stable@vger.kernel.org
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/cik: use a mutex to properly lock srbm instanced registers

We need proper locking in the driver when accessing instanced
registers on CIK.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: only save UVD bo when we have open handles

Otherwise just reinitialize from scratch on resume,
and so make it more likely to succeed.

Signed-off-by: Christian König <christian.koenig@amd.com>
Cc: stable@vger.kernel.org
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/gem: create drm_gem_dumb_destroy

All the gem based kms drivers really want the same function to
destroy a dumb framebuffer backing storage object.

So give it to them and roll it out in all drivers.

This still leaves the option open for kms drivers which don't use GEM
for backing storage, but it does decently simplify matters for gem
drivers.

Acked-by: Inki Dae <inki.dae@samsung.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Intel Graphics Development <intel-gfx@lists.freedesktop.org>
Cc: Ben Skeggs <skeggsb@gmail.com>
Reviwed-by: Rob Clark <robdclark@gmail.com>
Cc: Alex Deucher <alexdeucher@gmail.com>
Acked-by: Patrik Jakobsson <patrik.r.jakobsson@gmail.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: add a module parameter to disable aspm

Can cause hangs when enabled in certain motherboards.
Set radeon.aspm=0 to disable aspm.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: align VM PTBs (Page Table Blocks) to 32K

Covers requirements of all current asics.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: allow selection of alignment in the sub-allocator

There are cases where we need more than 4k alignment. No
functional change with this commit.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: never unpin UVD bo v3

Changing the UVD BOs offset on suspend/resume doesn't work because the VCPU
internally keeps pointers to it. Just keep it always pinned and save the
content manually.

Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=66425

v2: fix compiler warning
v3: fix CIK support

Note: a version of this patch needs to go to stable.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add checks against vblank time

If the vblank time is too short to adjust mclk,
assume multiple displays (no mclk adjustments).

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add infrastructure to force performance levels

This allows you to force specific power levels within a power
state. Due to hardware restrictions between generations, the
interface is limited to the following 3 selections:

auto: all levels enabled
low: forced to the lowest power level
high: forced to the highest power level

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add support for 3d perf states on older asics

Certain older rv770 asics have both a performance and
a 3D performance state rather than just multiple performance
levels in the state power state. The current code would
select the performance state rather than the 3D performance
state when the "performance" profile was selected. This change
switches to the "balanced" profile by default which ends up being
the internal performance profile. When the user selects the
"performance" profile, it selects the internal 3D performance
state so the user can select the higher performance modes.

For most asics this changes nothing. For certain rv770 asics
with static performance and 3D performance states, this allows
you to select between then using by selecting the "balanced"
and "performance" dpm profiles.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add infrastructure to support debugfs info

This lays the frameworks to report realtime power level
feedback.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: make ttm reservation calls behave like reservation calls

This commit converts the source of the val_seq counter to
the ww_mutex api. The reservation objects are converted later,
because there is still a lockdep splat in nouveau that has to
resolved first.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon/kms: add dpm support for SI (v7)

This adds dpm support for SI asics. This includes:
- dynamic engine clock scaling
- dynamic memory clock scaling
- dynamic voltage scaling
- dynamic pcie gen1/gen2/gen3 switching
- power containment
- shader power scaling

Set radeon.dpm=1 to enable.

v2: enable hainan support, rebase
v3: guard acpi stuff
v4: fix 64 bit math
v5: fix 64 bit div harder
v6: fix thermal interrupt check noticed by Jerome
v7: attempt fix state enable

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: validate voltages against dispclk requirements

Validate the voltages against the voltage requirements of the
dispclk. We currently don't adjust the disp clock so it never
changes, but we need to filter out voltage levels that are too
low none the less.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: pull in ppm info from atom

Used by SI dpm.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: pull in phase shedding limits from atom

Required for dpm on SI.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add an enum for pcie gen selection

This makes it easier the understand what the code is
doing.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add indirect accessors for UVD CTX registers

These are needed for certain UVD power saving features.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add atom get leakage vddc function

Required for DPM on SI.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add accessors of pif_phy indirect register space

Required for accessing certain pcie related registers.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: update radeon_atom_get_voltage_table() for SI

SI uses a new atom table revision. Required for DPM on SI.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: update radeon_atom_is_voltage_gpio() for SI

SI uses a new atom table. Required for DPM on SI.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/atom: add helper to calcuate mpll params

There's a new table for calculating the memory pll
parameters on SI. Required for SI DPM support.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: implement apci perf request

These functions use acpi methods to adjust the pcie
gen speed. Used by DPM.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: remove broken dyn state remnants

Now that the proper fix has been implemented I can
remove the last remnants of the initial implementation.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add new pre/post_set_power_state callbacks

Needed to properly handle dynamic state adjustment.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: fixup dynamic state adjust for btc (v2)

Use a dedicated copy of the current power state since
we may have to adjust it on the fly.

v2: fix up redundant state sets

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: fixup dynamic state adjust for sumo

Use a dedicated copy of the current power state since
we may have to adjust it on the fly.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: track whether we are on AC or battery

Driver needs this information to validate power states.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/dpm: add helpers for extended power tables (v2)

This data will be needed for dpm on newer asics.

v2: fix typo in rebase

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/kms: enable UVD as needed (v9)

When using UVD, the driver must switch to a special UVD power
state. In the CS ioctl, switch to the power state and schedule
work to change the power state back, when the work comes up,
check if uvd is still busy and if not, switch back to the user
state, otherwise, reschedule the work.

Note: We really need some better way to decide when to
switch out of the uvd power state. Switching power states
while playback is active make uvd angry.

V2: fix locking.

V3: switch from timer to delayed work

V4: check fence driver for UVD jobs, reduce timeout to
1 second and rearm timeout on activity

v5: rebase on new dpm tree

v6: rebase on interim uvd on demand changes

v7: fix UVD when DPM is disabled

v8: unify non-DPM and DPM UVD handling

v9: remove leftover idle work struct

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <deathsimple@vodafone.de>

drm/radeon/kms: add dpm support for evergreen (v4)

This adds dpm support for evergreen asics. This includes:
- clockgating
- dynamic engine clock scaling
- dynamic memory clock scaling
- dynamic voltage scaling
- dynamic pcie gen1/gen2 switching (requires additional acpi support)

Set radeon.dpm=1 to enable.

v2: reduce stack usage, rename ulv struct
v3: fix thermal interrupt check notices by Jerome
v4: fix state enable

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/kms: add dpm support for rv7xx (v4)

This adds dpm support for rv7xx asics. This includes:
- clockgating
- dynamic engine clock scaling
- dynamic memory clock scaling
- dynamic voltage scaling
- dynamic pcie gen1/gen2 switching

Set radeon.dpm=1 to enable.

v2: reduce stack usage
v3: fix 64 bit div
v4: fix state enable

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/kms: add dpm support for rv6xx (v3)

This adds dpm support for rv6xx asics. This includes:
- clockgating
- dynamic engine clock scaling
- dynamic memory clock scaling
- dynamic voltage scaling
- dynamic pcie gen1/gen2 switching

Set radeon.dpm=1 to enable.

v2: remove duplicate line
v3: fix thermal interrupt check noticed by Jerome

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>

drm/radeon/kms: add common r600 dpm functions

These are shared by rs780/rs880, rv6xx, and newer chips.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/kms: add common dpm infrastructure

This adds the common dpm (dynamic power management)
infrastructure:
- dpm callbacks
- dpm init/fini/suspend/resume
- dpm power state selection

No device specific code is enabled yet.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/kms: add atom helper functions for dpm (v3)

dpm needs access to atombios data and command tables
for setup and calculation of a number of parameters.

v2: endian fix
v3: fix mc reg table bug

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: properly set up the RLC on ON/LN/TN (v3)

This is required for certain advanced functionality.

v2: save/restore list takes dword offsets
v3: rebase on gpu reset changes

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: make get_temperature functions a callback

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/evergreen: add indirect register accessors for CG registers

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/kms: add accessors for RCU indirect space

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add cik tile mode array query

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/cik: Add support for compute queues (v4)

On CIK, the compute rings work slightly differently than
on previous asics, however the basic concepts are the same.

The main differences:
- New MEC engines for compute queues
- Multiple queues per MEC:
- CI/KB: 1 MEC, 4 pipes per MEC, 8 queues per pipe = 32 queues
- KV: 2 MEC, 4 pipes per MEC, 8 queues per pipe = 64 queues
- Queues can be allocated and scheduled by another queue
- New doorbell aperture allows you to assign space in the aperture
for the wptr which allows for userspace access to queues

v2: add wptr shadow, fix eop setup
v3: fix comment
v4: switch to new callback method

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>

drm/radeon: implement simple doorbell page allocator

The doorbell aperture is a PCI BAR whose pages can be
mapped to compute resources for things like wptrs
for userspace queues.

This patch maps the BAR and sets up a simple allocator
to allocate pages from the BAR.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: use callbacks for ring pointer handling (v3)

Add callbacks to the radeon_asic struct to handle
rptr/wptr fetchs and wptr updates.
We currently use one version for all rings, but this
allows us to override with a ring specific versions.

Needed for compute rings on CIK.

v2: udpate as per Christian's comments
v3: fix some rebase cruft

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add indirect register accessors for SMC registers

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/cik: add hw cursor support (v2)

CIK (DCE8) hw cursors are programmed the same as evergreen
(DCE4) with the following caveats:
- cursors are now 128x128 pixels
- new alpha blend enable bit

v2: rebase

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: handle the integrated thermal controller on CI

No support for reading the temperature yet.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/cik: add support for sDMA dma engines (v8)

CIK has new asynchronous DMA engines called sDMA
(system DMA). Each engine supports 1 ring buffer
for kernel and gfx and 2 userspace queues for compute.

TODO: fill in the compute setup.

v2: update to the latest reset code
v3: remove ib_parse
v4: fix copy_dma()
v5: drop WIP compute sDMA queues
v6: rebase
v7: endian fixes for IB
v8: cleanup for release

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add support for interrupts on CIK (v5)

Todo:
- handle interrupts for compute queues

v2: add documentation
v3: update to latest reset code
v4: update to latest illegal CP handling
v5: fix missing break in interrupt handler switch statement

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add initial ucode loading for CIK (v5)

Currently the driver required 6 sets of ucode:
1. pfp - pre-fetch parser, part of the GFX CP
2. me - micro engine, part of the GFX CP
3. ce - constant engine, part of the GFX CP
4. rlc - interrupt, etc. controller
5. mc - memory controller (discrete cards only)
6. mec - compute engines, part of Compute CP

V2: add documentation
V3: update MC ucode
V4: rebase
V5: update mc ucode

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add gpu init support for CIK (v9)

v2: tiling fixes
v3: more tiling fixes
v4: more tiling fixes
v5: additional register init
v6: rebase
v7: fix gb_addr_config for KV/KB
v8: drop wip KV bits for now, add missing config reg
v9: fix cu count on Bonaire

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add DCE8 macro for CIK

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add a reset work handler

New asics support non-privileged IBs. This allows us
to skip IB checking in the driver since the hardware
will check the command buffers for us. When using
non-privileged IBs, if the CP encounters an illegal
register in the command stream, it will halt and generate
an interrupt. The CP needs to be reset to continue. For now
just do a full GPU reset when this happens.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: track which asics have UVD

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add chip family for Hainan

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>

drm/radeon: consolidate UVD clock programming

Instead of duplicating the code over and over again, just use a single
function to handle the clock calculations.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

radeon: add bo tracking debugfs

This is to allow debugging of userspace program not freeing buffer
after, which is basicly a memory leak. This print the list of all
gem object along with their size and placement (VRAM,GTT,CPU) and
with the pid of the task that created them.

agd5f: add warning fix

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add helper function to support golden registers

Golden registers are arrays of register settings from the
hw team that need to be initialized at asic startup.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: switch audio handling to use callbacks

Register audio callbacks for asic where we support
audio. Cleans up the code and makes it easier to
add support for newer asics.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: raise UVD clocks only on demand

That not only saves some power, but also solves problems with
older chips where an idle UVD block on higher clocks can
cause problems.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add helpers for masking and setting bits in regs

Signed-off-by: Rafał Miłecki <zajec5@gmail.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add si tile mode array query v3

Allow userspace to query for the tile mode array so userspace can properly
compute surface pitch and alignment requirement depending on tiling.

v2: Make strict aliasing safer by casting to char when copying
v3: merge fix from Christian

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/kms: replace *REG32_PCIE_P with *REG32_PCIE_PORT

Avoid confusion with the *REG32_P mask macro.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add radeon_atom_get_clock_dividers helper

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add pm callback for setting uvd clocks

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: UVD bringup v8

Just everything needed to decode videos using UVD.

v6: just all the bugfixes and support for R7xx-SI merged in one patch
v7: UVD_CGC_GATE is a write only register, lockup detection fix
v8: split out VRAM fallback changes, remove support for RV770,
add support for HEMLOCK, add buffer sizes checks

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: rework fallback handling v2

Let the CS module decide if we can fall back to VRAM or not.

v2: remove unintended change

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Use direct mapping for fast fb access on RS690

This patch allows the CPU to map the stolen vram segment
directly rather than going through the PCI BAR. This
significantly improves performance for certain workloads with
a properly patched ddx.

Use radeon.fastfb=1 to enable it (disabled by default).
Currently only supported on RS690, but support for RS780/880
and newer APUs may be added eventually.

Signed-off-by: Samuel Li <samuel.li@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: clean up vram/gtt location handling

Add a per-asic MC (memory controller) mask which holds the
mak address mask the asic is capable of. Use this when
calculating the vram and gtt locations rather using asic
specific functions or limiting everything to 32 bits.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: switch get_gpu_clock() to a callback (v2)

Cleans up the code for future asics

v2: rebase, fix some missing radeon_asic updates

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add a asic callback to get the xclk

This is required to get the reference clock used
by the gfx engine for things like timestamps. Fixes
support for GL extensions the use timestamps on
certain boards.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: use prime helpers

Simplify the Radeon prime implementation by using the default behavior provided
by drm_gem_prime_import and drm_gem_prime_export.

v2:
- Rename functions to radeon_gem_prime_get_sg_table and
radeon_gem_prime_import_sg_table.
- Delete the now-unused vmapping_count variable.

Signed-off-by: Aaron Plattner <aplattner@nvidia.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: David Airlie <airlied@linux.ie>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: add Oland chip family

Oland is a new asic in the SI family.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: use IBs for VM page table updates v2

For very large page table updates, we can exceed the
size of the ring. To avoid this, use an IB to perform
the page table update.

v2(ck): cleanup the IB infrastructure and the use it instead
of filling the struct ourself.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>

drm/radeon: add a bios scratch asic hung helper

Used by all asic families from r600+.
Flag for the vbios and later instances of the driver
that the GPU is hung.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add additional reset flags

This adds further flags for fine grained reset.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: pull out common next_reloc function

next_reloc function does the same thing in all ASICs with
the exception of R600 which has a special case in legacy mode.
Pull out the common function in preparation for refactoring.

Signed-off-by: Ilija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: rename r100_cs_dump_packet to radeon_cs_dump_packet

This function is not limited to r100, but it can dump a
(raw) packet for any ASIC. Rename it accordingly and move
its declaration to radeon.h

Signed-off-by: Ilija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: refactor vline packet parsing function

vline packet parsing function for R600 and Evergreen+ are
the same, except that they use different registers. Factor
out the algorithm into a common function that uses register
table passed from ASIC-specific caller.

This reduces ASIC-specific function to (trivial) setup
of register table and call into the common function.

Signed-off-by: Ilija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: factor out cs_next_is_pkt3_nop function

Once we factored out radeon_cs_packet_parse function,
evergreen_cs_next_is_pkt3_nop and r600_cs_next_is_pkt3_nop
functions became identical, so they can be factored out
into a common function.

Signed-off-by: Ilija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: use common cs packet parse function

We now have a common radeon_cs_packet_parse function
that is good for all ASICs. Hook it up and eliminate
ASIC-specific versions.

Signed-off-by: Ilija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Revert "drm/radeon: do not move bo to different placement at each cs"

This reverts commit d025e9e2b890db679f1246037bf65bd4be512627.

This causes corruption for a number of users and needs further
investigation in the next cycle.
https://bugzilla.kernel.org/show_bug.cgi?id=52491
https://bugs.freedesktop.org/show_bug.cgi?id=58659
http://lists.freedesktop.org/archives/dri-devel/2013-January/032961.html

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>