drm/ttm: replace busy placement with flags v6

Instead of a list of separate busy placement add flags which indicate
that a placement should only be used when there is room or if we need to
evict.

v2: add missing TTM_PL_FLAG_IDLE for i915
v3: fix auto build test ERROR on drm-tip/drm-tip
v4: fix some typos pointed out by checkpatch
v5: cleanup some rebase problems with VMWGFX
v6: implement some missing VMWGFX functionality pointed out by Zack,
rename the flags as suggested by Michel, rebase on drm-tip and
adjust XE as well

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Somalapuram Amaranath <Amaranath.Somalapuram@amd.com>
Reviewed-by: Zack Rusin <zack.rusin@broadcom.com>
Reviewed-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20240112125158.2748-4-christian.koenig@amd.com

drm/radeon/radeon_ttm: Remove unused variable 'rbo' from radeon_bo_move()

Fixes the following W=1 kernel build warning(s):

drivers/gpu/drm/radeon/radeon_ttm.c: In function ‘radeon_bo_move’:
drivers/gpu/drm/radeon/radeon_ttm.c:201:27: warning: variable ‘rbo’ set but not used [-Wunused-but-set-variable]

Signed-off-by: Lee Jones <lee@kernel.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

mm/gup: remove unused vmas parameter from get_user_pages()

Patch series "remove the vmas parameter from GUP APIs", v6.

(pin_/get)_user_pages[_remote]() each provide an optional output parameter
for an array of VMA objects associated with each page in the input range.

These provide the means for VMAs to be returned, as long as mm->mmap_lock
is never released during the GUP operation (i.e. the internal flag
FOLL_UNLOCKABLE is not specified).

In addition, these VMAs can only be accessed with the mmap_lock held and
become invalidated the moment it is released.

The vast majority of invocations do not use this functionality and of
those that do, all but one case retrieve a single VMA to perform checks
upon.

It is not egregious in the single VMA cases to simply replace the
operation with a vma_lookup(). In these cases we duplicate the (fast)
lookup on a slow path already under the mmap_lock, abstracted to a new
get_user_page_vma_remote() inline helper function which also performs
error checking and reference count maintenance.

The special case is io_uring, where io_pin_pages() specifically needs to
assert that the VMAs underlying the range do not result in broken
long-term GUP file-backed mappings.

As GUP now internally asserts that FOLL_LONGTERM mappings are not
file-backed in a broken fashion (i.e. requiring dirty tracking) - as
implemented in "mm/gup: disallow FOLL_LONGTERM GUP-nonfast writing to
file-backed mappings" - this logic is no longer required and so we can
simply remove it altogether from io_uring.

Eliminating the vmas parameter eliminates an entire class of danging
pointer errors that might have occured should the lock have been
incorrectly released.

In addition, the API is simplified and now clearly expresses what it is
intended for - applying the specified GUP flags and (if pinning) returning
pinned pages.

This change additionally opens the door to further potential improvements
in GUP and the possible marrying of disparate code paths.

I have run this series against gup_test with no issues.

Thanks to Matthew Wilcox for suggesting this refactoring!


This patch (of 6):

No invocation of get_user_pages() use the vmas parameter, so remove it.

The GUP API is confusing and caveated. Recent changes have done much to
improve that, however there is more we can do. Exporting vmas is a prime
target as the caller has to be extremely careful to preclude their use
after the mmap_lock has expired or otherwise be left with dangling
pointers.

Removing the vmas parameter focuses the GUP functions upon their primary
purpose - pinning (and outputting) pages as well as performing the actions
implied by the input flags.

This is part of a patch series aiming to remove the vmas parameter
altogether.

Link: https://lkml.kernel.org/r/cover.1684350871.git.lstoakes@gmail.com
Link: https://lkml.kernel.org/r/589e0c64794668ffc799651e8d85e703262b1e9d.1684350871.git.lstoakes@gmail.com
Signed-off-by: Lorenzo Stoakes <lstoakes@gmail.com>
Suggested-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Christian König <christian.koenig@amd.com> (for radeon parts)
Acked-by: Jarkko Sakkinen <jarkko@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Acked-by: Sean Christopherson <seanjc@google.com> (KVM)
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
Cc: Janosch Frank <frankja@linux.ibm.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

drm/radeon: stop including swiotlb.h

radeon does not need swiotlb.h, so stop including it.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Remove unnecessary (void*) conversions

No need cast (void*) to (struct radeon_device *)
or (struct radeon_ring *).

Signed-off-by: Su Hui <suhui@nfschina.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: handle NULL bo->resource in move callback

The ttm bo now initially has NULL bo->resource, and leaves the driver
the handle that. However it looks like we forgot to handle that for
radeon. It looks like this will just null-ptr-deref in
radeon_bo_move(), if bo->resource is NULL.

Fix this by calling move_null().

Fixes: 180253782038 ("drm/ttm: stop allocating dummy resources during BO creation")
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230208145319.397235-4-matthew.auld@intel.com
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>

drm/ttm: prevent moving of pinned BOs

We have checks for this in the individual drivers move callback, but
it's probably better to generally forbid that on a higher level.

Also stops exporting ttm_resource_compat() since that's not necessary
any more after removing the extra checks in vmwgfx.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230130120636.63765-6-matthew.auld@intel.com

drm/ttm: revert "prevent moving of pinned BOs"

This reverts commit b49323aa35d502b0d9a7950327f30a1a52eae534.

This still seems to break i915.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Matthew Auld <matthew.william.auld@gmail.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230125155023.105584-1-christian.koenig@amd.com

drm/ttm: prevent moving of pinned BOs

We have checks for this in the individual drivers move callback, but
it's probably better to generally forbid that on a higher level.

Also stops exporting ttm_resource_compat() since that's not necessary
any more after removing the extra checks in vmwgfx.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230124125726.13323-4-christian.koenig@amd.com

drm/ttm: merge ttm_bo_api.h and ttm_bo_driver.h v2

Merge and cleanup the two headers into a single description of the
object API. Also move all the documentation to the implementation and
drop unnecessary includes from the header.

No functional change.

v2: minimal checkpatch.pl cleanup

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221125102137.1801-4-christian.koenig@amd.com

drm/ttm: rework on ttm_resource to use size_t type

Change ttm_resource structure from num_pages to size_t size in bytes.
v1 -> v2: change PFN_UP(dst_mem->size) to ttm->num_pages
v1 -> v2: change bo->resource->size to bo->base.size at some places
v1 -> v2: remove the local variable
v1 -> v2: cleanup cmp_size_smaller_first()
v2 -> v3: adding missing PFN_UP in ttm_bo_vm_fault_reserved

Signed-off-by: Somalapuram Amaranath <Amaranath.Somalapuram@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221027091237.983582-1-Amaranath.Somalapuram@amd.com
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>

drm/radeon: Replace kmap() with kmap_local_page()

The use of kmap() is being deprecated in favor of kmap_local_page().

There are two main problems with kmap(): (1) It comes with an overhead as
the mapping space is restricted and protected by a global lock for
synchronization and (2) it also requires global TLB invalidation when the
kmap’s pool wraps and it might block when the mapping space is fully
utilized until a slot becomes available.

With kmap_local_page() the mappings are per thread, CPU local, can take
page faults, and can be called from any context (including interrupts).
It is faster than kmap() in kernels with HIGHMEM enabled. Furthermore,
the tasks can be preempted and, when they are scheduled to run again, the
kernel virtual addresses are restored and still valid.

Therefore, replace kmap() with kmap_local_page() in radeon_ttm_gtt_read().

Cc: "Venkataramanan, Anirudh" <anirudh.venkataramanan@intel.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Christian König <christian.koenig@amd.com>
Suggested-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Fabio M. De Francesco <fmdefrancesco@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Use TTM builtin resource manager debugfs code

Switch to using the TTM resource manager debugfs helpers. The
functionality is largely the same.
The TTM resource managers need to stay valid for as long as the
drm debugfs_root is valid.

Signed-off-by: Zack Rusin <zackr@vmware.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: "Pan, Xinhui" <Xinhui.Pan@amd.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: amd-gfx@lists.freedesktop.org
Link: https://patchwork.freedesktop.org/patch/msgid/20220412033526.369115-6-zack@kde.org
Reviewed-by: Christian König <christian.koenig@amd.com>

drm/radeon: remove resource accounting v2

Use the one provided by TTM instead.

v2: drop new_mem parameter as well

Signed-off-by: Christian König <christian.koenig@amd.com>
Tested-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220214093439.2989-5-christian.koenig@amd.com

drm/radeon: use ttm_resource_manager_debug

Instead of calling the debug operation directly.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211124124430.20859-8-christian.koenig@amd.com

drm/ttm: s/FLAG_SG/FLAG_EXTERNAL/

It covers more than just ttm_bo_type_sg usage, like with say dma-buf,
since one other user is userptr in amdgpu, and in the future we might
have some more. Hence EXTERNAL is likely a more suitable name.

v2(Christian):
- Rename these to TTM_TT_FLAGS_*
- Fix up all the holes in the flag values

Suggested-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Christian König <christian.koenig@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210929132629.353541-1-matthew.auld@intel.com
Signed-off-by: Christian König <christian.koenig@amd.com>

drm/ttm: remove ttm_tt_destroy_common v2

Move the functionality into ttm_tt_fini and ttm_bo_tt_destroy instead.

We don't need this any more since we removed the unbind from the destroy
code paths in the drivers.

Also add a warning to ttm_tt_fini() if we try to fini a still populated TT
object.

v2: instead of reverting the patch move the functionality to different
places.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210728130552.2074-5-christian.koenig@amd.com

drm/radeon: unbind in radeon_ttm_tt_unpopulate()

Doing this in radeon_ttm_tt_destroy() is to late.

It turned out that this is not a good idea at all because it leaves pointers
to freed up system memory pages in the GART tables of the drivers.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210728130552.2074-4-christian.koenig@amd.com

drm/radeon: Fix NULL dereference when updating memory stats

radeon_ttm_bo_destroy() is attempting to access the resource object to
update memory counters. However, the resource object is already freed when
ttm calls this function via the destroy callback. This causes an oops when
a bo is freed:

BUG: kernel NULL pointer dereference, address: 0000000000000010
RIP: 0010:radeon_ttm_bo_destroy+0x2c/0x100 [radeon]
Call Trace:
radeon_bo_unref+0x1a/0x30 [radeon]
radeon_gem_object_free+0x33/0x50 [radeon]
drm_gem_object_release_handle+0x69/0x70 [drm]
drm_gem_handle_delete+0x62/0xa0 [drm]
? drm_mode_destroy_dumb+0x40/0x40 [drm]
drm_ioctl_kernel+0xb2/0xf0 [drm]
drm_ioctl+0x30a/0x3c0 [drm]
? drm_mode_destroy_dumb+0x40/0x40 [drm]
radeon_drm_ioctl+0x49/0x80 [radeon]
__x64_sys_ioctl+0x8e/0xd0

Avoid the issue by updating the counters in the delete_mem_notify callback
instead. Also, fix memory statistic updating in radeon_bo_move() to
identify the source type correctly. The source type needs to be saved
before the move, because the moved from object may be altered by the move.

Fixes: bfa3357ef9ab ("drm/ttm: allocate resource object instead of embedding it v2")
Signed-off-by: Mikel Rychliski <mikel@mikelr.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210624045121.15643-1-mikel@mikelr.com

drm/radeon: Fix NULL dereference when updating memory stats

radeon_ttm_bo_destroy() is attempting to access the resource object to
update memory counters. However, the resource object is already freed when
ttm calls this function via the destroy callback. This causes an oops when
a bo is freed:

BUG: kernel NULL pointer dereference, address: 0000000000000010
RIP: 0010:radeon_ttm_bo_destroy+0x2c/0x100 [radeon]
Call Trace:
radeon_bo_unref+0x1a/0x30 [radeon]
radeon_gem_object_free+0x33/0x50 [radeon]
drm_gem_object_release_handle+0x69/0x70 [drm]
drm_gem_handle_delete+0x62/0xa0 [drm]
? drm_mode_destroy_dumb+0x40/0x40 [drm]
drm_ioctl_kernel+0xb2/0xf0 [drm]
drm_ioctl+0x30a/0x3c0 [drm]
? drm_mode_destroy_dumb+0x40/0x40 [drm]
radeon_drm_ioctl+0x49/0x80 [radeon]
__x64_sys_ioctl+0x8e/0xd0

Avoid the issue by updating the counters in the delete_mem_notify callback
instead. Also, fix memory statistic updating in radeon_bo_move() to
identify the source type correctly. The source type needs to be saved
before the move, because the moved from object may be altered by the move.

Fixes: bfa3357ef9ab ("drm/ttm: allocate resource object instead of embedding it v2")
Signed-off-by: Mikel Rychliski <mikel@mikelr.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210624045121.15643-1-mikel@mikelr.com

drm/ttm: flip over the range manager to self allocated nodes

Start with the range manager to make the resource object the base
class for the allocated nodes.

While at it cleanup a lot of the code around that.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210602100914.46246-2-christian.koenig@amd.com

drm/ttm: allocate resource object instead of embedding it v2

To improve the handling we want the establish the resource object as base
class for the backend allocations.

v2: add missing error handling

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210602100914.46246-1-christian.koenig@amd.com

drm/ttm: rename bo->mem and make it a pointer

When we want to decouble resource management from buffer management we need to
be able to handle resources separately.

Add a resource pointer and rename bo->mem so that all code needs to
change to access the pointer instead.

No functional change.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210430092508.60710-4-christian.koenig@amd.com

drm/radeon: Implement mmap as GEM object function

Moving the driver-specific mmap code into a GEM object function allows
for using DRM helpers for various mmap callbacks.

This change also allows to support prime-based mmap via DRM's helper
drm_gem_prime_mmap().

Permission checks are implemented by drm_gem_mmap(), with an additional
check for radeon_ttm_tt_has_userptr() in the GEM object function. The
function radeon_verify_access() is now unused and has thus been removed.

As a side effect, radeon_ttm_vm_ops and radeon_ttm_fault() are now
implemented in amdgpu's GEM code.

v3:
* remove unnecessary checks from mmap (Christian)
v2:
* rename radeon_ttm_vm_ops and radeon_ttm_fault() to
radeon_gem_vm_ops and radeon_gem_fault() (Christian)
* fix commit description (Alex)

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210525151055.8174-4-tzimmermann@suse.de

drm/radeon: Move AGP data structures into radeon

With the AGP code already duplicated, move over the AGP structures
from the legacy code base in to radeon. The AGP data structures that
are required by radeon are now declared within the driver. The AGP
instance is stored in struct radeon_device.agp.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210507185709.22797-3-tzimmermann@suse.de

drm/radeon: Fix size overflow

ttm->num_pages is uint32. Hit overflow when << PAGE_SHIFT directly

Fixes: 230c079fdcf4 ("drm/ttm: make num_pages uint32_t")
Signed-off-by: xinhui pan <xinhui.pan@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: avoid potential null pointer access

Leverage the same logic from amdgpu_ttm_tt_unpin_userptr.

Signed-off-by: Guchun Chen <guchun.chen@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/ttm: Fix memory leak userptr pages

If userptr pages have been pinned but not bounded,
they remain uncleared.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Daniel Gomez <daniel@qtec.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: keep __user during cast

Silence static checker warning.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/ttm: constify static vm_operations_struct

The only usage of radeon_ttm_vm_ops is to assign its address to the
vm_ops field in the vm_area_struct struct. Make it const to allow the
compiler to put it in read-only memory

Signed-off-by: Rikard Falkeborn <rikard.falkeborn@gmail.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210209234817.55112-3-rikard.falkeborn@gmail.com
Signed-off-by: Christian König <christian.koenig@amd.com>

drm/radeon: do not use drm middle layer for debugfs (v2)

Use debugfs API directly instead of drm middle layer.

v2: squash in build fix (Alex)

Signed-off-by: Nirmoy Das <nirmoy.das@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: cleanup debugfs

Remove unnecessary debugfs dentries and also radeon_ttm_debugfs_fini()
as drm_debugfs_cleanup() will recursively remove debugfs files.

Signed-off-by: Nirmoy Das <nirmoy.das@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: device naming cleanup

Rename ttm_bo_device to ttm_device.
Rename ttm_bo_driver to ttm_device_funcs.
Rename ttm_bo_global to ttm_global.

Move global and device related functions to ttm_device.[ch].

No functional change.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/415222/

drm/radeon: Fix size overflow

ttm->num_pages is uint32. Hit overflow when << PAGE_SHIFT directly

Fixes: 230c079fdcf4 ("drm/ttm: make num_pages uint32_t")
Signed-off-by: xinhui pan <xinhui.pan@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: Store PCI controller in struct radeon_device.hose

Moves struct drm_device.hose into struct radeon_device. The field in
struct DRM device is only for legacy drivers.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210112081035.6882-6-tzimmermann@suse.de

drm/ttm: cleanup BO size handling v3

Based on an idea from Dave, but cleaned up a bit.

We had multiple fields for essentially the same thing.

Now bo->base.size is the original size of the BO in
arbitrary units, usually bytes.

bo->mem.num_pages is the size in number of pages in the
resource domain of bo->mem.mem_type.

v2: use the GEM object size instead of the BO size
v3: fix printks in some places

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com> (v1)
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/406831/

drm/ttm/drivers: remove unecessary ttm_module.h include v2

ttm_module.h deals with internals of TTM and should never
be include outside of it.

v2: also move the file around

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/404885/

drm/prime: split array import functions v4

Mapping the imported pages of a DMA-buf into an userspace process
doesn't work as expected.

But we have reoccurring requests on this approach, so split the
functions for this and document that dma_buf_mmap() needs to be used
instead.

v2: split it into two functions
v3: rebased on latest changes
v4: update commit message a bit

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/403838/

drm/radeon: stop using pages with drm_prime_sg_to_page_addr_arrays v2

This is deprecated.

v2: also use ttm_sg_tt_init to avoid allocating the page array.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/403832/

drm/radeon: fix check order in radeon_bo_move

Reorder the code to fix checking if blitting is available.

Signed-off-by: Christian König <christian.koenig@amd.com>
Fixes: 28a68f828266 ("drm/radeon/ttm: use multihop")
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/403847/

drm/radeon: stop re-init the TTM page pool

Drivers are not supposed to init the page pool directly any more.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/412153/

drm/radeon: fix check order in radeon_bo_move

Reorder the code to fix checking if blitting is available.

Signed-off-by: Christian König <christian.koenig@amd.com>
Fixes: 28a68f828266 ("drm/radeon/ttm: use multihop")
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/403847/
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

drm/radeon: fix check order in radeon_bo_move

Reorder the code to fix checking if blitting is available.

Signed-off-by: Christian König <christian.koenig@amd.com>
Fixes: 28a68f828266 ("drm/radeon/ttm: use multihop")
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/403847/

drm/radeon: Move radeon_ttm{init, fini} to shared location

Fixes the following W=1 kernel build warning(s):

drivers/gpu/drm/radeon/radeon_ttm.c: At top level:
drivers/gpu/drm/radeon/radeon_ttm.c:817:5: warning: no previous prototype for ‘radeon_ttm_init’ [-Wmissing-prototypes]
817 | int radeon_ttm_init(struct radeon_device *rdev)
| ^~~~~~~~~~~~~~~
drivers/gpu/drm/radeon/radeon_ttm.c:878:6: warning: no previous prototype for ‘radeon_ttm_fini’ [-Wmissing-prototypes]
878 | void radeon_ttm_fini(struct radeon_device *rdev)
| ^~~~~~~~~~~~~~~

Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: amd-gfx@lists.freedesktop.org
Cc: dri-devel@lists.freedesktop.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Signed-off-by: Lee Jones <lee.jones@linaro.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/radeon_ttm: Place declaration of 'rdev' in same clause as its use

Fixes the following W=1 kernel build warning(s):

drivers/gpu/drm/radeon/radeon_ttm.c: In function ‘radeon_ttm_tt_create’:
drivers/gpu/drm/radeon/radeon_ttm.c:611:24: warning: variable ‘rdev’ set but not used [-Wunused-but-set-variable]

Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Jerome Glisse <glisse@freedesktop.org>
Cc: amd-gfx@lists.freedesktop.org
Cc: dri-devel@lists.freedesktop.org
Signed-off-by: Lee Jones <lee.jones@linaro.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon/ttm: use multihop

This removes the code to move resources directly between
SYSTEM and VRAM in favour of using the core ttm mulithop code.

Signed-off-by: Dave Airlie <airlied@redhat.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201109005432.861936-5-airlied@gmail.com

drm/ttm: add multihop infrastrucutre (v3)

Currently drivers get called to move a buffer, but if they have to
move it temporarily through another space (SYSTEM->VRAM via TT)
then they can end up with a lot of ttm->driver->ttm call stacks,
if the temprorary space moves requires eviction.

Instead of letting the driver do all the placement/space for the
temporary, allow it to report back (-EMULTIHOP) and a placement (hop)
to the move code, which will then do the temporary move, and the
correct placement move afterwards.

This removes a lot of code from drivers, at the expense of
adding some midlayering. I've some further ideas on how to turn
it inside out, but I think this is a good solution to the call
stack problems.

v2: separate out the driver patches, add WARN for getting
MULTHOP in paths we shouldn't (Daniel)
v3: use memset (Christian)

Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: hristian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201109005432.861936-2-airlied@gmail.com

drm/radeon: switch to new allocator v2

It should be able to handle all cases here.

v2: fix debugfs as well

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Madhav Chauhan <madhav.chauhan@amd.com>
Tested-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/397088/?series=83051&rev=1

drm/ttm: wire up the new pool as default one v2

Provide the necessary parameters by all drivers and use the new pool alloc
when no driver specific function is provided.

v2: fix the GEM VRAM helpers

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Madhav Chauhan <madhav.chauhan@amd.com>
Tested-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/397081/?series=83051&rev=1

drm/ttm: merge ttm_dma_tt back into ttm_tt

It makes no difference to kmalloc if the structure
is 48 or 64 bytes in size.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/396950/

drm/ttm: make num_pages uint32_t

We can still allocate 16TiB with that.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/396946/

drm/ttm: nuke ttm_tt_set_(un)populated again

Neither page allocation backend nor the driver should mess with that.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Madhav Chauhan <madhav.chauhan@amd.com>
Link: https://patchwork.freedesktop.org/patch/396948/

drm/ttm: replace last move_notify with delete_mem_notify

The move notify callback is only used in one place, this should
be removed in the future, but for now just rename it to the use
case which is to notify the driver that the GPU memory is to be
deleted.

Drivers can be cleaned up after this separately.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201021044031.1752624-2-airlied@gmail.com

drm/ttm: move last binding into the drivers.

This moves the call to tt binding into the driver move,
and drops the driver callback.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201020010319.1692445-8-airlied@gmail.com

drm/ttm: drop move notify around move.

The drivers now do this in the move callback.

move_notify is still needed in the destroy path.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201020010319.1692445-7-airlied@gmail.com

drm/ttm: drop unbind callback.

The drivers now control this, so drop unbinding.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201020010319.1692445-5-airlied@gmail.com

drm/ttm: add move to system into drivers

This moves the to system move into the drivers, and moves all
the unbinds in the move path under driver control

Note: radeon/nouveau already wait so don't duplicate it.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201020010319.1692445-4-airlied@gmail.com

drm/ttm: move some move binds into the drivers

This just gives the driver control over some of the bind paths.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201020010319.1692445-2-airlied@gmail.com

drm/ttm: add move old to system to drivers.

Uninline ttm_bo_move_ttm. Eventually want to unhook the unbind out.

Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201019071314.1671485-5-airlied@gmail.com

drm/ttm: use new move interface for known system->ttm moves

In all 3 drivers there is a case where the driver knows the
bo is in SYSTEM so don't call the api that checks that.

Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201019071314.1671485-4-airlied@gmail.com

drm/ttm: nuke caching placement flags

Changing the caching on the fly never really worked
flawlessly.

So stop this completely and just let drivers specific the
desired caching in the tt or bus object.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com>
Link: https://patchwork.freedesktop.org/patch/394256/

drm/ttm: add caching state to ttm_bus_placement

And implement setting it up correctly in the drivers.

This allows getting rid of the placement flags for this.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com>
Link: https://patchwork.freedesktop.org/patch/394254/

drm/ttm: set the tt caching state at creation time

All drivers can determine the tt caching state at creation time,
no need to do this on the fly during every validation.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com>
Link: https://patchwork.freedesktop.org/patch/394253/

drm/radeon: stop using TTMs fault callback

We already implemented the fault handler ourself,
just open code what is necessary here.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/392322/

drm/radeon/ttm: handle ttm moves properly

The core move code currently handles use_tt moves, for radeon
this was being handled also in the driver, but not using the same
paths.

If moving between TT/SYSTEM (all the use_tt paths on radeon) use
the core move function.

Eventually the core will be flipped over to calling the driver.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200924051845.397177-5-airlied@gmail.com

drm/radeon: switch over to the new pin interface

Stop using TTM_PL_FLAG_NO_EVICT.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/391610/?series=81973&rev=1

drm/ttm: add bo wait that takes a ctx wrapper.

I'm thinking of pushing the wait into the drivers.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200923030454.362731-6-airlied@gmail.com

drm/radeon: cleanup ttm operation ctx usage.

Just pass it around move, and remove unused pieces

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200923030454.362731-4-airlied@gmail.com

drm/ttm: drop special pipeline accel cleanup function.

The two accel cleanup paths were mostly the same once refactored.

Just pass a bool to say if the evictions are to be pipelined.

Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200917064132.148521-2-airlied@gmail.com

drm/ttm/drivers: call the bind function directly.

Now the bind functions have all the protection explicitly the
drivers can just call them directly, and the api can be unexported

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200917043040.146575-5-airlied@gmail.com

drm/ttm: move unbind into the tt destroy.

This moves unbind into the driver side on destroy paths.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200917043040.146575-4-airlied@gmail.com

drm/ttm: flip tt destroy ordering.

Call the driver first and have it call the common code cleanup.

This is useful later to fix unbind.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200917043040.146575-3-airlied@gmail.com

drm/ttm: protect against reentrant bind in the drivers

This moves the generic tracking into the drivers and protects
against reentrancy in the drivers. It fixes up radeon and agp
to be able to query the bound status as that is required.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200917043040.146575-2-airlied@gmail.com

drm/ttm: move ttm binding/unbinding out of ttm_tt paths.

Move these up to the bo level, moving ttm_tt to just being
backing store. Next step is to move the bound flag out.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200915024007.67163-6-airlied@gmail.com

drm/ttm: split populate out from binding.

Drivers have to call populate themselves now before binding.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200915024007.67163-5-airlied@gmail.com

drm/ttm/tt: add wrappers to set tt state.

This adds 2 getters and 4 setters, however unbound and populated
are currently the same thing, this will change, it also drops
a BUG_ON that seems not that useful.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200915024007.67163-2-airlied@gmail.com

drm/ttm: remove available_caching

Instead of letting TTM make an educated guess based on
some mask all drivers should just specify what caching
they want for their CPU mappings.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/390207/

drm/ttm: remove default caching

As far as I can tell this was never used either and we just
always fallback to the order cached > wc > uncached anyway.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/390142/

drm/ttm: nuke memory type flags

It's not supported to specify more than one of those flags.
So it never made sense to make this a flag in the first place.

Nuke the flags and specify directly which memory type to use.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/389826/?series=81551&rev=1

drm/ttm/agp: remove bdev from agp helpers

Since the agp bind/unbind/destroy are now getting called from drivers
rather than via the func table, drop the bdev parameter.

Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200907204630.1406528-13-airlied@gmail.com

drm/ttm: get rid of agp specific populate/unpopulate paths.

Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200907204630.1406528-12-airlied@gmail.com

drm/radeon/ttm: move to driver binding/destroy functions. (v2)

Do agp decision in the driver, instead of special binding funcs

v2: use container_of, drop some {}.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200907204630.1406528-6-airlied@gmail.com

drm/ttm: introduce ttm_bo_move_null

This pattern is cut-n-pasted across 4 drivers, switch it to
a WARN_ON instead, as BUG_ON is considered a bad idea usually.

Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200907204630.1406528-2-airlied@gmail.com

drm/ttm: merge offset and base in ttm_bus_placement

This is used by TTM to communicate the physical address
which should be used with ioremap(), ioremap_wc(). We don't
need to separate the base and offset in any way here.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/389457/

drm/ttm: remove bdev from ttm_tt

I want to split this structure up and use it differently,
step one remove bdev pointer from it and pass it explicitly.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200826014428.828392-4-airlied@gmail.com

drm/radeon/ttm: don't store driver copy of device pointer.

This can be gotten back from bdev.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200826014428.828392-3-airlied@gmail.com

drm/ttm: drop bus.size from bus placement.

This is always calculated the same, and only used in a couple of places.

Signed-off-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200811074658.58309-2-airlied@gmail.com

drm/ttm: init mem->bus in common code.

The drivers all do the same thing here.

Reviewed-by: Christian König <christian.koenig@amd.com> for both.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200811074658.58309-1-airlied@gmail.com

drm/radeon: drop superflous AGP handling

The object flags created in radeon_ttm_placement_from_domain take care that
we use the correct caching for AGP, this is just superflous.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/384336/?series=80346&rev=1

drm/ttm: give resource functions their own [ch] files

This is a separate object we work within TTM.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/384338/?series=80346&rev=1

drm/ttm: rename ttm_mem_reg to ttm_resource.

This name better reflects what the object does. I didn't rename
all the pointers it seemed too messy.

Signed-off-by: Dave Airlie <airlied@redhat.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-60-airlied@gmail.com

drm/ttm: rename ttm_mem_type_manager -> ttm_resource_manager.

This name makes a lot more sense, since these are about managing
driver resources rather than just memory ranges.

Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-59-airlied@gmail.com

drm/ttm: make ttm_range_man_init/takedown take type + args

This makes it easier to move these to a driver allocated system

Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Ben Skeggs <bskeggs@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-47-airlied@gmail.com

drm/radeon/ttm: use wrapper to access memory manager

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-42-airlied@gmail.com

drm/radeon/ttm: use new takedown paths

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-32-airlied@gmail.com

drm/radeon: use new ttm man init path

Use the new common manager init path.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-17-airlied@gmail.com

drm/ttm: remove the init_mem_type callback

It is a very strange concept to call a function which just
calls back the caller for the functions parameters.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/382085/

drm/radeon: stop implementing init_mem_type

Instead just initialize the memory type parameters
before calling ttm_bo_init_mm.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/382080/

drm/ttm: remove TTM_MEMTYPE_FLAG_FIXED v2

Instead use a boolean field in the memory manager structure.

Also invert the meaning of the field since the use of a TT
structure is the special case here.

v2: cleanup zero init.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/382079/

drm/ttm: initialize the system domain with defaults v2

Instead of repeating that in each driver.

v2: keep the caching limitation for VMWGFX for now.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Thomas Zimmermann <tzimmermann@suse.de>
Link: https://patchwork.freedesktop.org/patch/382078/

drm/ttm: make ttm_tt unbind function return void.

The return value just led to BUG_ON, I think if a driver wants
to BUG_ON here it can do it itself. (don't BUG_ON).

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200728040003.20398-1-airlied@gmail.com

drm/radeon: stop using TTM_MEMTYPE_FLAG_MAPPABLE

The driver doesn't expose any not-mapable memory resources.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/378239/

drm/ttm: remove TTM_MEMTYPE_FLAG_CMA

The original intention was to avoid CPU page table unmaps
when BOs move between the GTT and SYSTEM domain.

The problem is that this never correctly handled changes
in the caching attributes or backing pages.

Just drop this for now and simply unmap the CPU page
tables in all cases.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/378240/

drm: remove optional dummy function from drivers using TTM

Implementing those is completely unnecessary.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Madhav Chauhan <madhav.chauhan@amd.com>
Link: https://patchwork.freedesktop.org/patch/378236/

drm: radeon: fix common struct sg_table related issues

The Documentation/DMA-API-HOWTO.txt states that the dma_map_sg() function
returns the number of the created entries in the DMA address space.
However the subsequent calls to the dma_sync_sg_for_{device,cpu}() and
dma_unmap_sg must be called with the original number of the entries
passed to the dma_map_sg().

struct sg_table is a common structure used for describing a non-contiguous
memory buffer, used commonly in the DRM and graphics subsystems. It
consists of a scatterlist with memory pages and DMA addresses (sgl entry),
as well as the number of scatterlist entries: CPU pages (orig_nents entry)
and DMA mapped pages (nents entry).

It turned out that it was a common mistake to misuse nents and orig_nents
entries, calling DMA-mapping functions with a wrong number of entries or
ignoring the number of mapped entries returned by the dma_map_sg()
function.

To avoid such issues, lets use a common dma-mapping wrappers operating
directly on the struct sg_table objects and use scatterlist page
iterators where possible. This, almost always, hides references to the
nents and orig_nents entries, making the code robust, easier to follow
and copy/paste safe.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/371172/
Signed-off-by: Christian König <christian.koenig@amd.com>

drm/radeon: don't use ttm bo->offset

Calculate GPU offset in radeon_bo_gpu_offset without depending on
bo->offset.

Signed-off-by: Nirmoy Das <nirmoy.das@amd.com>
Reviewed-and-tested-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/372935/

drm/radeon: fix scatter-gather mapping with user pages

Calls to dma_map_sg may return less segments / entries than requested
if they fall on page bounderies. The old implementation did not
support this use case.

Fixes: be62dbf554c5 ("iommu/amd: Convert AMD iommu driver to the dma-iommu api")
Bug: https://bugzilla.kernel.org/show_bug.cgi?id=206461
Bug: https://bugzilla.kernel.org/show_bug.cgi?id=206895
Bug: https://gitlab.freedesktop.org/drm/amd/issues/1056
Signed-off-by: Shane Francis <bigbeeshane@gmail.com>
Reviewed-by: Michael J. Ruhl <michael.j.ruhl@intel.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200325090741.21957-4-bigbeeshane@gmail.com
Cc: stable@vger.kernel.org

drm/ttm: nuke invalidate_caches callback

Another completely unused feature.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/348265/

remove ioremap_nocache and devm_ioremap_nocache

ioremap has provided non-cached semantics by default since the Linux 2.6
days, so remove the additional ioremap_nocache interface.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Arnd Bergmann <arnd@arndb.de>

drm/radeon: Don't include <drm/drm_pci.h>

Including <drm/drm_pci.h> is unnecessary in most cases. Replace
these instances.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20191203100406.9674-9-tzimmermann@suse.de

drm/radeon: finally fix the racy VMA setup

Finally clean up the VMA setup for radeon now that TTM exports the
necessary functions.

No functional change, but only compile tested.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/339354/

drm/radeon: switch to gem vma offset manager

Pass gem vma_offset_manager to ttm_bo_device_init(), so ttm uses it
instead of its own embedded struct. This makes some gem functions
(specifically drm_gem_object_lookup) work on ttm objects.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20190905070509.22407-5-kraxel@redhat.com

drm/ttm: turn ttm_bo_device.vma_manager into a pointer

Rename the embedded struct vma_offset_manager, new name is _vma_manager.
ttm_bo_device.vma_manager changed to a pointer.

The ttm_bo_device_init() function gets an additional vma_manager
argument which allows to initialize ttm with a different vma manager.
When passing NULL the embedded _vma_manager is used.

All callers are updated to pass NULL, so the behavior doesn't change.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20190905070509.22407-2-kraxel@redhat.com

drm/radeon: handle PCIe root ports with addressing limitations

radeon uses a need_dma32 flag to indicate to the drm core that some
allocations need to be done using GFP_DMA32, but it only checks the
device addressing capabilities to make that decision. Unfortunately
PCIe root ports that have limited addressing exist as well. Use the
dma_addressing_limited instead to also take those into account.

Reviewed-by: Christian König <christian.koenig@amd.com>
Reported-by: Atish Patra <Atish.Patra@wdc.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: switch driver from bo->resv to bo->base.resv

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20190805140119.7337-12-kraxel@redhat.com

drm/radeon: use embedded gem object

Drop drm_gem_object from radeon_bo, use the
ttm_buffer_object.base instead.

Build tested only.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20190805140119.7337-5-kraxel@redhat.com

radeon: no need to check return value of debugfs_create functions

When calling debugfs functions, there is no need to ever check the
return value. The function can work or not, but the code logic should
never do something different based on this.

Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: "David (ChunMing) Zhou" <David1.Zhou@amd.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: amd-gfx@lists.freedesktop.org
Cc: dri-devel@lists.freedesktop.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: drop use of drmP.h (1/2)

Drop use of drmP.h in all .c files named radeon*c.
To ease review a little drmP.h removal was divided in two commits.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: "David (ChunMing) Zhou" <David1.Zhou@amd.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20190608080241.4958-7-sam@ravnborg.org

drm/ttm: remove set but not used variable 'rdev'

Fixes gcc '-Wunused-but-set-variable' warning:

drivers/gpu/drm/radeon/radeon_ttm.c: In function 'radeon_move_vram_ram':
drivers/gpu/drm/radeon/radeon_ttm.c:254:24: warning:
variable 'rdev' set but not used [-Wunused-but-set-variable]

drivers/gpu/drm/radeon/radeon_ttm.c: In function 'radeon_move_ram_vram':
drivers/gpu/drm/radeon/radeon_ttm.c:301:24: warning:
variable 'rdev' set but not used [-Wunused-but-set-variable]

It's not used since a987fcaa805f ("ttm: Make parts of a struct ttm_bo_device
global.")

Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: Quick-test mmap offset in ttm_bo_mmap()

A BO's address has to be at least the minimum offset. Sharing this
test in ttm_bo_mmap() removes code from drivers. A full buffer-address
validation is still done within drm_vma_offset_lockup_locked().

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: Remove file_page_offset parameter from ttm_bo_device_init()

The parameter file_page_offset is a constant shared by all drivers. Just
replace it with the constant itself.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: Define a single DRM_FILE_PAGE_OFFSET constant

Most TTM drivers define the constant DRM_FILE_PAGE_OFFSET of the same
value. The only exception is vboxvideo, which is being converted to the
new offset by this patch. Unifying the constants in a single place
simplifies the driver code.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: initialize globals during device init (v2)

Make sure that the global BO state is always correctly initialized.

This allows removing all the device code to initialize it.

v2: fix up vbox (Alex)

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: use a static ttm_mem_global instance

As the name says we only need one global instance of ttm_mem_global.

Drop all the driver initialization and just use a single exported
instance which is initialized during BO global initialization.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: Rename ttm_bo_global_{init,release}() to ttm_bo_global_ref_{,}()

The functions ttm_bo_global_init() and ttm_bo_global_release() do not
receive an argument of type struct ttm_bo_global. Both take a struct
drm_global_reference that contains points to a struct ttm_bo_global_ref.
Renaming them reflects this.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

gpu: drm: radeon: Adding new typedef vm_fault_t

Use new return type vm_fault_t for fault handler
in vm_operations_struct.

Signed-off-by: Souptick Joarder <jrdr.linux@gmail.com>
Reviewed-by: Matthew Wilcox <mawilcox@microsoft.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: add bo as parameter to the ttm_tt_create callback

Instead of calculating the size in bytes just to recalculate the number
of pages from it pass the BO directly to the function.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: drop ttm->dummy_read_page

Only used by the AGP backend and there it can be easily accessed using
ttm->bdev->glob.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: remove extra TT unpopulated check

The subsystem should check that, not the driver.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: only enable swiotlb path when need v2

swiotlb expands our card accessing range, but its path always is slower
than ttm pool allocation.
So add condition to use it.
v2: move a bit later

Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20180209024410.1469-3-david1.zhou@amd.com

drm/ttm: use an operation ctx for ttm_tt_bind

forward the operation context to ttm_tt_bind as well,
and the ultimate goal is swapout enablement for reserved BOs.

v2: use common term rather than amd specific

Reviewed-by: Thomas Hellström <thellstrom@vmware.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chuming Zhou <david1.zhou@amd.com>
Signed-off-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: use an operation ctx for ttm_tt_populate in ttm_bo_driver (v2)

forward the operation context to ttm_tt_populate as well,
and the ultimate goal is swapout enablement for reserved BOs.

v2: squash in fix for vboxvideo

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Roger He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: remove the default io_mem_pfn set

The default interface situation has been taken into the framework, so
remove the default set of each module.

Signed-off-by: Tan Xiaojun <tanxiaojun@huawei.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: use an ttm operation ctx for ttm_bo_move_xxx

include ttm_bo_move_memcpy and ttm_bo_move_ttm

Signed-off-by: Roger He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: add context to driver move callback as well

Instead of passing the parameters manually.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Tested-by: Dieter Nützel <Dieter@nuetzel-hh.de>
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: use an operation context for ttm_bo_mem_space v2

Instead of specifying interruptible and no_wait_gpu manually.

v2: rebase

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Tested-by: Dieter Nützel <Dieter@nuetzel-hh.de>
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

mm: remove cold parameter for release_pages

All callers of release_pages claim the pages being released are cache
hot. As no one cares about the hotness of pages being released to the
allocator, just ditch the parameter.

No performance impact is expected as the overhead is marginal. The
parameter is removed simply because it is a bit stupid to have a useless
parameter copied everywhere.

Link: http://lkml.kernel.org/r/20171018075952.10627-7-mgorman@techsingularity.net
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

drm/radeon: use new TTM populate/dma map helper functions

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: fix incorrect use of the lru_lock

The BO manager has its own lock and doesn't use the lru_lock.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Fix stolen typo

Change "stollen" to "stolen"

Signed-off-by: Kent Russell <kent.russell@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm: radeon: radeon_ttm: constify ttm_place structures.

ttm_place are not supposed to change at runtime. All functions
working with ttm_place provided by <drm/ttm/ttm_placement.h> work
with const ttm_place. So mark the non-const structs as const.

File size before:
text data bss dec hex filename
9235 344 136 9715 25f3 drivers/gpu/drm/radeon/radeon_ttm.o

File size After adding 'const':
text data bss dec hex filename
9267 312 136 9715 25f3 drivers/gpu/drm/radeon/radeon_ttm.o

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Arvind Yadav <arvind.yadav.cs@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: fix include notation and remove -Iinclude/drm flag

Include <drm/*.h> instead of relative path from include/drm, then
remove the -Iinclude/drm compiler flag.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1493009447-31524-14-git-send-email-yamada.masahiro@socionext.com

drm/ttm: add io_mem_pfn callback

This allows the driver to handle io_mem mappings on their own.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Override fpfn for all VRAM placements in radeon_evict_flags

We were accidentally only overriding the first VRAM placement. For BOs
with the RADEON_GEM_NO_CPU_ACCESS flag set,
radeon_ttm_placement_from_domain creates a second VRAM placment with
fpfn == 0. If VRAM is almost full, the first VRAM placement with
fpfn > 0 may not work, but the second one with fpfn == 0 always will
(the BO's current location trivially satisfies it). Because "moving"
the BO to its current location puts it back on the LRU list, this
results in an infinite loop.

Fixes: 2a85aedd117c ("drm/radeon: Try evicting from CPU accessible to
inaccessible VRAM first")
Reported-by: Zachary Michaels <zmichaels@oblong.com>
Reported-and-Tested-by: Julien Isorce <jisorce@oblong.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

mm, fs: reduce fault, page_mkwrite, and pfn_mkwrite to take only vmf

->fault(), ->page_mkwrite(), and ->pfn_mkwrite() calls do not need to
take a vma and vmf parameter when the vma already resides in vmf.

Remove the vma parameter to simplify things.

[arnd@arndb.de: fix ARM build]
Link: http://lkml.kernel.org/r/20170125223558.1451224-1-arnd@arndb.de
Link: http://lkml.kernel.org/r/148521301778.19116.10840599906674778980.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Darrick J. Wong <darrick.wong@oracle.com>
Cc: Matthew Wilcox <mawilcox@microsoft.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Jan Kara <jack@suse.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

drm: radeon: radeon_ttm: Handle return NULL error from ioremap_nocache

Here, If ioremap_nocache will fail. It will return NULL.
Kernel can run into a NULL-pointer dereference.
This error check will avoid NULL pointer dereference.

Signed-off-by: Arvind Yadav <arvind.yadav.cs@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: revert "implement LRU add callbacks v2"

The additional housekeeping had too much CPU overhead,
let's use the BO priorities instead.

agd: also revert hibmc changes

Reviewed-by: Sinclair Yeh <syeh@vmware.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-and-Tested-by: Roger.He <Hongbo.He@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/mm: Convert to drm_printer

Including all drivers. I thought about keeping small compat functions
to avoid having to change all drivers. But I really like the
drm_printer idea, so figured spreading it more widely is a good thing.

v2: Review from Chris:
- Natural argument order and better name for drm_mm_print.
- show_mm() macro in the selftest.

Cc: Rob Clark <robdclark@gmail.com>
Cc: Russell King <rmk+kernel@armlinux.org.uk>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Tomi Valkeinen <tomi.valkeinen@ti.com>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Jyri Sarha <jsarha@ti.com>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1483009764-8281-1-git-send-email-daniel.vetter@ffwll.ch

drm/ttm: make eviction decision a driver callback v2

This way the driver can decide if it is valuable to evict a BO or not.

The current implementation is added as default to all existing drivers.

v2: fix some typos found during internal testing

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

mm: replace get_user_pages() write/force parameters with gup_flags

This removes the 'write' and 'force' from get_user_pages() and replaces
them with 'gup_flags' to make the use of FOLL_FORCE explicit in callers
as use of this flag can result in surprising behaviour (and hence bugs)
within the mm subsystem.

Signed-off-by: Lorenzo Stoakes <lstoakes@gmail.com>
Acked-by: Christian König <christian.koenig@amd.com>
Acked-by: Jesper Nilsson <jesper.nilsson@axis.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

drm: use drm_file to tag vm-bos

Rather than using "struct file*", use "struct drm_file*" as tag VM tag for
BOs. This will pave the way for "struct drm_file*" without any "struct
file*" back-pointer.

Signed-off-by: David Herrmann <dh.herrmann@gmail.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20160901124837.680-3-dh.herrmann@gmail.com

drm/radeon: fix radeon_move_blit on 32bit systems

This bug seems to be present for a very long time.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/ttm: Remove unused parameter evict from ttm_bo_move_memcpy

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: Remove unused parameter evict from ttm_bo_move_ttm

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: Wait for a BO to become idle before unbinding it from GTT

Fixes hangs under memory pressure, e.g. running the piglit test
tex3d-maxsize concurrently with other tests.

Fixes: 17d33bc9d6ef ("drm/ttm: drop waiting for idle in ttm_bo_evict.")
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: Wait for a BO to become idle before unbinding it from GTT

Fixes hangs under memory pressure, e.g. running the piglit test
tex3d-maxsize concurrently with other tests.

Fixes: 17d33bc9d6ef ("drm/ttm: drop waiting for idle in ttm_bo_evict.")
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: remove no_gpu_wait param from ttm_bo_move_accel_cleanup

It isn't used and not waiting for the GPU after scheduling a move is
actually quite dangerous.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: wait for BO idle in ttm_bo_move_memcpy

When we want to pipeline accelerated moves we need to wait in the fallback path.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: add wait for idle in all drivers bo_move functions

Wait for idle before moving the BO in all drivers implementing
an accelerated move function.

This should keep the current behavior when removing the pre move wait.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: implement LRU add callbacks v2

This allows fine grained control for the driver where to add a BO into the LRU.

v2: fix typo in comment

Reviewed-by: Sinclair Yeh <syeh@vmware.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: forbid mapping of userptr bo through radeon device file

Allowing userptr bo which are basicly a list of page from some vma
(so either anonymous page or file backed page) would lead to serious
corruption of kernel structures and counters (because we overwrite
the page->mapping field when mapping buffer).

This will already block if the buffer was populated before anyone does
try to mmap it because then TTM_PAGE_FLAG_SG would be set in in the
ttm_tt flags. But that flag is check before ttm_tt_populate in the ttm
vm fault handler.

So to be safe just add a check to verify_access() callback.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

mm, fs: get rid of PAGE_CACHE_* and page_cache_{get,release} macros

PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} macros were introduced *long* time
ago with promise that one day it will be possible to implement page
cache with bigger chunks than PAGE_SIZE.

This promise never materialized. And unlikely will.

We have many places where PAGE_CACHE_SIZE assumed to be equal to
PAGE_SIZE. And it's constant source of confusion on whether
PAGE_CACHE_* or PAGE_* constant should be used in a particular case,
especially on the border between fs and mm.

Global switching to PAGE_CACHE_SIZE != PAGE_SIZE would cause to much
breakage to be doable.

Let's stop pretending that pages in page cache are special. They are
not.

The changes are pretty straight-forward:

- <foo> << (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;

- <foo> >> (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;

- PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} -> PAGE_{SIZE,SHIFT,MASK,ALIGN};

- page_cache_get() -> get_page();

- page_cache_release() -> put_page();

This patch contains automated changes generated with coccinelle using
script below. For some reason, coccinelle doesn't patch header files.
I've called spatch for them manually.

The only adjustment after coccinelle is revert of changes to
PAGE_CAHCE_ALIGN definition: we are going to drop it later.

There are few places in the code where coccinelle didn't reach. I'll
fix them manually in a separate patch. Comments and documentation also
will be addressed with the separate patch.

virtual patch

@@
expression E;
@@
- E << (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E

@@
expression E;
@@
- E >> (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E

@@
@@
- PAGE_CACHE_SHIFT
+ PAGE_SHIFT

@@
@@
- PAGE_CACHE_SIZE
+ PAGE_SIZE

@@
@@
- PAGE_CACHE_MASK
+ PAGE_MASK

@@
expression E;
@@
- PAGE_CACHE_ALIGN(E)
+ PAGE_ALIGN(E)

@@
expression E;
@@
- page_cache_get(E)
+ get_page(E)

@@
expression E;
@@
- page_cache_release(E)
+ put_page(E)

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

drm/radeon: Don't move pinned BOs

The purpose of pinning is to prevent a buffer from moving.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: use post-decrement in error handling

We need to use post-decrement to get the pci_map_page undone also for
i==0, and to avoid some very unpleasant behaviour if pci_map_page
failed already at i==0.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

mm/gup: Switch all callers of get_user_pages() to not pass tsk/mm

We will soon modify the vanilla get_user_pages() so it can no
longer be used on mm/tasks other than 'current/current->mm',
which is by far the most common way it is called. For now,
we allow the old-style calls, but warn when they are used.
(implemented in previous patch)

This patch switches all callers of:

get_user_pages()
get_user_pages_unlocked()
get_user_pages_locked()

to stop passing tsk/mm so they will no longer see the warnings.

Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Hansen <dave@sr71.net>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: jack@suse.cz
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20160212210156.113E9407@viggo.jf.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>

drm: Remove __OS_HAS_AGP

We already express the drm/agp depencies correctly in Kconfig, so we
can rip this remnant from the shared drm core days.

Aside: Pretty much all the #ifdefs in radeon/nouveau could be killed
if ttm would provide dummy functions. I'm not going to volunteer for
that though.

v2: Use IS_ENABLED(CONFIG_AGP) as suggested by Ville

v3: Polish from Ville's review.

Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: Christian König <christian.koenig@amd.com> (v2)
Reviewed-by: David Herrmann <dh.herrmann@gmail.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

drm/radeon: use kzalloc for allocating one thing

Use kzalloc for allocating one thing rather than
kcalloc(1...

The semantic patch that makes this change is as follows:

// <smpl>
@@
@@

- kcalloc(1,
+ kzalloc(
...)
// </smpl>

Signed-off-by: Maninder Singh <maninder1.s@samsung.com>
Reviewed-by: Vaneet Narang <v.narang@samsung.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: fix userptr BO unpin bug v3

Fixing a memory leak with userptrs.

v2: clean up the loop, use an iterator instead
v3: remove unused variable

Signed-off-by: monk.liu <monk.liu@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
CC: stable@vger.kernel.org
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add extra check in radeon_ttm_tt_unpin_userptr

We somehow try to free the SG table twice.

Bugs: https://bugs.freedesktop.org/show_bug.cgi?id=89734

Signed-off-by: Christian König <christian.koenig@amd.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: check the right ring in radeon_evict_flags()

Check the that ring we are using for copies is functional
rather than the GFX ring. On newer asics we use the DMA
ring for bo moves.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: Try evicting from CPU accessible to inaccessible VRAM first

Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: export reservation_object from dmabuf to ttm

Adds an extra argument to radeon_bo_create, which is only used in radeon_prime.c.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/<ttm-based-drivers>: Don't call drm_mmap

Really, the legacy buffer api should be dead, especially for all these
newfangled drivers. I suspect this is copypasta from the transitioning
days, which probably originated in radeon.

Cc: "Christian König" <christian.koenig@amd.com>
Cc: David Herrmann <dh.herrmann@gmail.com>
Cc: Rashika <rashika.kheria@gmail.com>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Fabian Frederick <fabf@skynet.be>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Alexandre Courbot <acourbot@nvidia.com>
Cc: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Cc: Christian Engelmayer <cengelma@gmx.at>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Reviewed-by: David Herrmann <dh.herrmann@gmail.com>
Acked-by: Ben Skeggs <bskeggs@redhat.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: fix AGP userptr handling

AGP mappings are not cache coherent, so userptr support
won't work. Additional to that the AGP implementation uses
a different ttm_tt container structure so we run into
problems if we cast the pointer without checking if it's
the right type.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add the infrastructure for concurrent buffer access

This allows us to specify if we want to sync to
the shared fences of a reservation object or not.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: flip the switch, and convert to dma_fence

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>

drm/ttm: move fpfn and lpfn into each placement v2

This allows us to more fine grained specify where to place the buffer object.

v2: rebased on drm-next, add bochs changes as well

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add userptr flag to limit it to anonymous memory v2

Avoid problems with writeback by limiting userptr to anonymous memory.

v2: add commit and code comments

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add userptr support v8

This patch adds an IOCTL for turning a pointer supplied by
userspace into a buffer object.

It imposes several restrictions upon the memory being mapped:

1. It must be page aligned (both start/end addresses, i.e ptr and size).

2. It must be normal system memory, not a pointer into another map of IO
space (e.g. it must not be a GTT mmapping of another object).

3. The BO is mapped into GTT, so the maximum amount of memory mapped at
all times is still the GTT limit.

4. The BO is only mapped readonly for now, so no write support.

5. List of backing pages is only acquired once, so they represent a
snapshot of the first use.

Exporting and sharing as well as mapping of buffer objects created by
this function is forbidden and results in an -EPERM.

v2: squash all previous changes into first public version
v3: fix tabs, map readonly, don't use MM callback any more
v4: set TTM_PAGE_FLAG_SG so that TTM never messes with the pages,
pin/unpin pages on bind/unbind instead of populate/unpopulate
v5: rebased on 3.17-wip, IOCTL renamed to userptr, reject any unknown
flags, better handle READONLY flag, improve permission check
v6: fix ptr cast warning, use set_page_dirty/mark_page_accessed on unpin
v7: add warning about it's availability in the API definition
v8: drop access_ok check, fix VM mapping bits

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v4)
Reviewed-by: Jérôme Glisse <jglisse@redhat.com> (v4)
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Allow write-combined CPU mappings of BOs in GTT (v2)

v2: fix rebase onto drm-fixes

Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Pass GART page flags to radeon_gart_set_page() explicitly

Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm: init TTM dev_mapping in ttm_bo_device_init()

With dev->anon_inode we have a global address_space ready for operation
right from the beginning. Therefore, there is no need to do a delayed
setup with TTM. Instead, set dev_mapping during initialization in
ttm_bo_device_init() and remove any "if (dev_mapping)" conditions.

Cc: Dave Airlie <airlied@redhat.com>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Cc: Alex Deucher <alexdeucher@gmail.com>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: David Herrmann <dh.herrmann@gmail.com>

drm: use anon-inode instead of relying on cdevs

DRM drivers share a common address_space across all character-devices of a
single DRM device. This allows simple buffer eviction and mapping-control.
However, DRM core currently waits for the first ->open() on any char-dev
to mark the underlying inode as backing inode of the device. This delayed
initialization causes ugly conditions all over the place:
if (dev->dev_mapping)
do_sth();

To avoid delayed initialization and to stop reusing the inode of the
char-dev, we allocate an anonymous inode for each DRM device and reset
filp->f_mapping to it on ->open().

Signed-off-by: David Herrmann <dh.herrmann@gmail.com>

drm/radeon: silence GCC warning on 32 bit

Building radeon_ttm.o on 32 bit x86 triggers a warning:
In file included from include/asm-generic/bug.h:13:0,
from [...]/arch/x86/include/asm/bug.h:38,
from include/linux/bug.h:4,
from include/drm/drm_mm.h:39,
from include/drm/drm_vma_manager.h:26,
from include/drm/ttm/ttm_bo_api.h:35,
from drivers/gpu/drm/radeon/radeon_ttm.c:32:
drivers/gpu/drm/radeon/radeon_ttm.c: In function 'radeon_ttm_gtt_read':
include/linux/kernel.h:712:17: warning: comparison of distinct pointer types lacks a cast [enabled by default]
(void) (&_min1 == &_min2); \
^
drivers/gpu/drm/radeon/radeon_ttm.c:938:22: note: in expansion of macro 'min'
ssize_t cur_size = min(size, PAGE_SIZE - off);
^

Silence this warning by using min_t(). Since cur_size will never be
negative and its upper bound is PAGE_SIZE, we can change its type to
size_t and use min_t(size_t, [...]) here.

Signed-off-by: Paul Bolle <pebolle@tiscali.nl>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>

drm/radeon: TTM must be init with cpu-visible VRAM, v2

Without this, a bo may get created in the cpu-inaccessible vram.
Before the CP engines get setup, all copies are done via cpu memcpy.

This means that the cpu tries to read from inaccessible memory, fails,
and the radeon module proceeds to disable acceleration.

Doing this has no downsides, as the real VRAM size gets set as soon as the
CP engines get init.

This is a candidate for 3.14 fixes.

v2: Add comment on why the function is used

Signed-off-by: Lauri Kasanen <cand@gmx.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Cc: stable@vger.kernel.org

drm/radeon: track memory statistics about VRAM and GTT usage and buffer moves v2

The statistics are:
- VRAM usage in bytes
- GTT usage in bytes
- number of bytes moved by TTM

The last one is actually a counter, so you need to sample it before and after
command submission and take the difference.

This is useful for finding performance bottlenecks. Userspace queries are
also added.

v2: use atomic64_t

Signed-off-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>

drm/radeon: add GART debugfs access v3

v2: add default_llseek
v3: set inode size in the open callback

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: add VRAM debugfs access v3

Not very fast, but makes it possible to access even the
normally inaccessible parts of VRAM from userspace.

v2: use MM_INDEX_HI for >2GB mem access, add default_llseek
v3: set inode size in the open callback

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: cleanup radeon_ttm debugfs handling

Otherwise we not necessary export the right information.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm: rip out drm_core_has_AGP

Most place actually want to just check for dev->agp (most do, but a
few don't so this fixes a few potential NULL derefs). The only
exception is the agp init code which should check for the AGP driver
feature flag.

Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm: verify vma access in TTM+GEM drivers

GEM does already a good job in tracking access to gem buffers via handles
and drm_vma access management. However, TTM drivers currently do not
verify this during mmap().

TTM provides the verify_access() callback to test this. So fix all drivers
to actually call into gem+vma to verify access instead of always returning
0.

All drivers assume that user-space can only get access to TTM buffers via
GEM handles. So whenever the verify_access() callback is called from
ttm_bo_mmap(), the buffer must have a valid embedded gem object. This is
true for all TTM+GEM drivers. But that's why this patch doesn't touch pure
TTM drivers (ie, vmwgfx).

v2: Switch to drm_vma_node_verify_access() to correctly return -EACCES if
access was denied.

Cc: Dave Airlie <airlied@redhat.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Cc: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: David Herrmann <dh.herrmann@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: Fix VRAM size calculation for VRAM >= 4GB

Add ULL prefix to avoid overflow.

Signed-off-by: Niels Ole Salscheider <niels_ole@salscheider-online.de>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

drm: fix compile failure by including <linux/swiotlb.h>

On tile architecture (with "make allyesconfig") including
<linux/swiotlb.h> is required to call swiotlb_nr_tbl().

Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
Acked-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm: fix compile failure by including <linux/swiotlb.h>

On tile architecture (with "make allyesconfig") including
<linux/swiotlb.h> is required to call swiotlb_nr_tbl().

Signed-off-by: Chris Metcalf <cmetcalf@tilera.com>
Acked-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>

drm/ttm: remove no_wait_reserve, v3

All items on the lru list are always reservable, so this is a stupid
thing to keep. Not only that, it is used in a way which would
guarantee deadlocks if it were ever to be set to block on reserve.

This is a lot of churn, but mostly because of the removal of the
argument which can be nested arbitrarily deeply in many places.

No change of code in this patch except removal of the no_wait_reserve
argument, the previous patch removed the use of no_wait_reserve.

v2:
- Warn if -EBUSY is returned on reservation, all objects on the list
should be reservable. Adjusted patch slightly due to conflicts.
v3:
- Focus on no_wait_reserve removal only.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/ttm: remove sync_arg from driver functions

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-By: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/ttm: remove sync_obj_arg from ttm_bo_move_accel_cleanup

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-By: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: Mark all possible functions / structs as static

Let's allow GCC to optimize better.

This exposed some five unused functions, but this patch doesn't remove them.

Signed-off-by: Lauri Kasanen <cand@gmx.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm: track dev_mapping in more robust and flexible way

Setting dev_mapping (pointer to the address_space structure
used for memory mappings) to the address_space of the first
opener's inode and then failing if other openers come in
through a different inode has a few restrictions that are
eliminated by this patch.

If we already have valid dev_mapping and we spot an opener
with different i_node, we force its i_mapping pointer to the
already established address_space structure (first opener's
inode). This will make all mappings from drm device hang off
the same address_space object.

Some benefits (things that now work and didn't work
before) of this patch are:

* user space can mknod and use any number of device
nodes and they will all work fine as long as the major
device number is that of the drm module.
* user space can even remove the first opener's device
nodes and mknod the new one and the applications and
windowing system will still work.
* GPU drivers can safely assume that dev->dev_mapping is
correct address_space and just blindly copy it
into their (private) bdev.dev_mapping

For reference, some discussion that lead to this patch can
be found here:

http://lists.freedesktop.org/archives/dri-devel/2012-April/022283.html

Signed-off-by: Ilija Hadzic <ihadzic@research.bell-labs.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: replace vmram_mutex with mclk_lock v2

It is a rw_semaphore now and only write locked
while changing the clock. Also the lock is renamed
to better reflect what it is protecting.

v2: Keep the ttm_vm_ops on IGPs

Signed-off-by: Christian König <deathsimple@vodafone.de>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>

drm/radeon: rework ring syncing code

Move inter ring syncing with semaphores into the
existing ring allocations, with that we need to
lock the ring mutex only once.

Signed-off-by: Christian König <deathsimple@vodafone.de>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>

drm/radeon: remove radeon_fence_create

It is completely unnecessary to create fences
before they are emitted, so remove it and a bunch
of checks if fences are emitted or not.

Signed-off-by: Christian König <deathsimple@vodafone.de>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>

drm/radeon: add PRIME support (v2)

This adds prime->fd and fd->prime support to radeon.
It passes the sg object to ttm and then populates
the gart entries using it.

Compile tested only.

v2: stub kmap + use new helpers + add reimporting

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: immediately free ttm-move semaphore

We can now protected the semaphore ram by a
fence, so free it immediately.

Signed-off-by: Christian König <deathsimple@vodafone.de>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: fix a bug with the ring syncing code

Rings need to lock in order, otherwise
the ring subsystem can deadlock.

v2: fix error handling and number of locked doublewords.
v3: stop creating unneeded semaphores.

Signed-off-by: Christian König <deathsimple@vodafone.de>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon/kms: reorganize copy callbacks

tidy up the radeon_asic struct, handle multiple
rings better.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König<christian.koenig@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

ttm: fix agp since ttm tt rework

ttm tt rework modified the way we allocate and populate the
ttm_tt structure, the AGP side was missing some bit to properly
work. Fix those and fix radeon and nouveau AGP support.

Tested on radeon only so far.

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon/kms: sync across multiple rings when doing bo moves v3

We need to synchronize across rings when doing a bo move to make
sure we the buffer is idle if it's in use by a different ring than
the ring doing the move.

v2: fix fence setup for bo moves

v3: add missing ring lock/unlock

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: rename struct radeon_cp to radeon_ring

That naming seems to make more sense, since we not
only want to run PM4 rings with it.

Signed-off-by: Christian König <deathsimple@vodafone.de>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: make cp variable an array

Replace cp, cp1 and cp2 members with just an array
of radeon_cp structs.

Signed-off-by: Christian König <deathsimple@vodafone.de>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon/kms: add support for multiple fence queues v2

For supporting multiple CP ring buffers, async DMA
engines and UVD. We still need a way to synchronize
between engines.

v2 initialize unused fence driver ring to avoid issue in
suspend/unload

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <deathsimple@vodafone.de>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/ttm: isolate dma data from ttm_tt V4

Move dma data to a superset ttm_dma_tt structure which herit
from ttm_tt. This allow driver that don't use dma functionalities
to not have to waste memory for it.

V2 Rebase on top of no memory account changes (where/when is my
delorean when i need it ?)
V3 Make sure page list is initialized empty
V4 typo/syntax fixes

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>

drm/radeon/kms: enable the ttm dma pool if swiotlb is on V4

With the exception that we do not handle the AGP case. We only
deal with PCIe cards such as ATI ES1000 or HD3200 that have been
detected to only do DMA up to 32-bits.

V2 force dma32 if we fail to set bigger dma mask
V3 Rebase on top of no memory account changes (where/when is my
delorean when i need it ?)
V4 add debugfs entry is swiotlb is active not only if we are
on dma 32bits only gpu

CC: Dave Airlie <airlied@redhat.com>
CC: Alex Deucher <alexdeucher@gmail.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>

drm/ttm: introduce callback for ttm_tt populate & unpopulate V4

Move the page allocation and freeing to driver callback and
provide ttm code helper function for those.

Most intrusive change, is the fact that we now only fully
populate an object this simplify some of code designed around
the page fault design.

V2 Rebase on top of memory accounting overhaul
V3 New rebase on top of more memory accouting changes
V4 Rebase on top of no memory account changes (where/when is my
delorean when i need it ?)

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>

drm/ttm: merge ttm_backend and ttm_tt V5

ttm_backend will only exist with a ttm_tt, and ttm_tt
will only be of interest when bound to a backend. Merge them
to avoid code and data duplication.

V2 Rebase on top of memory accounting overhaul
V3 Rebase on top of more memory accounting changes
V4 Rebase on top of no memory account changes (where/when is my
delorean when i need it ?)
V5 make sure ttm is unbound before destroying, change commit
message on suggestion from Tormod Volden

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>

drm/ttm: remove unused backend flags field

This field is not use by any of the driver just drop it.

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>

drm/radeon/kms: Make GPU/CPU page size handling consistent in blit code (v2)

The BO blit code inconsistenly handled the page size. This wasn't
an issue on system with 4k pages since the GPU's page size is 4k as
well. Switch the driver blit callbacks to take num pages in GPU
page units.

Fixes lemote mipsel systems using AMD rs780/rs880 chipsets.

v2: incorporate suggestions from Michel.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Cc: stable@kernel.org
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon/alpha: Add Alpha support to Radeon DRM code

Alpha needs to have available the system bus address for the Radeon's
local memory, so that it can be used in ttm_bo_vm_fault(), when building
the PTEs for accessing that VRAM. So, we make bus.addr hold the ioremap()
return, and then we can modify bus.base appropriately for use during page
fault processing.

Signed-off-by: Jay Estabrook <jay.estabrook@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: fix problem with changing active VRAM size. (v2)

So we used to use lpfn directly to restrict VRAM when we couldn't
access the unmappable area, however this was removed in
93225b0d7bc030f4a93165347a65893685822d70 as it also restricted
the gtt placements. However it was only later noticed that this
broke on some hw.

This removes the active_vram_size, and just explicitly sets it
when it changes, TTM/drm_mm will always use the real_vram_size,
and the active vram size will change the TTM size used for lpfn
setting.

We should re-work the fpfn/lpfn to per-placement at some point
I suspect, but that is too late for this kernel.

Hopefully this addresses:
https://bugs.freedesktop.org/show_bug.cgi?id=35254

v2: fix reported useful VRAM size to userspace to be correct.

Signed-off-by: Dave Airlie <airlied@redhat.com>

Revert "ttm: Include the 'struct dev' when using the DMA API."

This reverts commit 5a893fc28f0393adb7c885a871b8c59e623fd528.

This causes a use after free in the ttm free alloc pages path,
when it tries to get the be after the be has been destroyed.

Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: embed struct drm_gem_object

Unconditionally initialize the drm gem object - it's not
worth the trouble not to for the few kernel objects.

This patch only changes the place of the drm gem object,
access is still done via pointers.

v2: Uncoditionally align the size in radeon_bo_create. At
least the r600/evergreen blit code didn't to this, angering
the paranoid gem code.

Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Dave Airlie <airlied@redhat.com>

ttm: Include the 'struct dev' when using the DMA API.

This makes the accounting when using 'debug_dma_dump_mappings()'
and CONFIG_DMA_API_DEBUG=y be assigned to the correct device
instead of 'fallback'.

No functional change - just cosmetic.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

radeon/ttm/PCIe: Use dma_addr if TTM has set it.

If the TTM layer has used the DMA API to setup pages that are
TTM_PAGE_FLAG_DMA32 (look at patch titled: "ttm: Utilize the dma_addr_t
array for pages that are to in DMA32 pool."), lets use it
when programming the GART in the PCIe type cards.

This patch skips doing the pci_map_page (and pci_unmap_page) if
there is a DMA addresses passed in for that page. If the dma_address
is zero (or DMA_ERROR_CODE), then we continue on with our old
behaviour.

[v2: Fixed an indentation problem, added reviewed-by tag]
[v3: Added Acked-by Jerome]

Acked-by: Jerome Glisse <j.glisse@gmail.com>
Reviewed-by: Thomas Hellstrom <thomas@shipmail.org>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Tested-by: Ian Campbell <ian.campbell@citrix.com>

drm/radeon: fix memory debugging since d961db75ce86a84f1f04e91ad1014653ed7d9f46

The old code dereferenced a value, the new code just needs to pass
the ptr.

fixes an oops looking at files in debugfs.

cc: stable@kernel.org
Signed-off-by: Dave Airlie <airlied@redhat.com>

ttm: Expand (*populate) to support an array of DMA addresses.

We pass in the array of ttm pages to be populated in the GART/MM
of the card (or AGP). Patch titled: "ttm: Utilize the DMA API for
pages that have TTM_PAGE_FLAG_DMA32 set." uses the DMA API to make
those pages have a proper DMA addresses (in the situation where
page_to_phys or virt_to_phys do not give use the DMA (bus) address).

Since we are using the DMA API on those pages, we should pass in the
DMA address to this function so it can save it in its proper fields
(later patches use it).

[v2: Added reviewed-by tag]

Reviewed-by: Thomas Hellstrom <thellstrom@shipmail.org>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Tested-by: Ian Campbell <ian.campbell@citrix.com>

drm/radeon/kms: fix alignment when allocating buffers

We were previously dropping alignment requests on the floor
when allocating buffers so we always ended up page aligned.
Certain tiling modes on 6xx+ require larger alignment which
wasn't happening before.

Signed-off-by: Alex Deucher <alexdeucher@gmail.com>
Cc: Jerome Glisse <j.glisse@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drivers/gpu/drm: Update WARN uses

Coalesce long formats.
Align arguments.
Add missing newlines.

Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/ttm: restructure to allow driver to plug in alternate memory manager

Nouveau will need this on GeForce 8 and up to account for the GPU
reordering physical VRAM for some memory types.

Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Acked-by: Thomas Hellström <thellstrom@vmware.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/ttm: introduce utility function to free an allocated memory node

Existing core code/drivers call drm_mm_put_block on ttm_mem_reg.mm_node
directly. Future patches will modify TTM behaviour in such a way that
ttm_mem_reg.mm_node doesn't necessarily belong to drm_mm.

Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Acked-by: Thomas Hellström <thellstrom@vmware.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drivers/gpu/drm: Remove unnecessary casts of private_data

Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: Jiri Kosina <jkosina@suse.cz>

drm: move ttm global code to core drm

I wrote this for the prime sharing work, but I also noticed other external
non-upstream drivers from a large company carrying a similiar patch, so I
may as well ship it in master.

Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: AGP memory is only I/O if the aperture can be mapped by the CPU.

Fixes AGP initialization failure with Apple UniNorth bridges due to trying to
ioremap() normal RAM.

Signed-off-by: Michel Dänzer <daenzer@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

radeon: Unmap vram pages when reclocking

Touching vram while the card is reclocking can lead to lockups. Unmap
any pages that could be touched by the CPU and block any accesses to
vram until the reclocking is complete.

Signed-off-by: Matthew Garrett <mjg@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/ttm: remove io_ field from TTM V6

All TTM driver have been converted to new io_mem_reserve/free
interface which allow driver to choose and return proper io
base, offset to core TTM for ioremapping if necessary. This
patch remove what is now deadcode.

V2 adapt to match with change in first patch of the patchset
V3 update after io_mem_reserve/io_mem_free callback balancing
V4 adjust to minor cleanup
V5 remove the needs ioremap flag
V6 keep the ioremapping facility in TTM

[airlied- squashed driver removals in here also]

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon/kms: add support for new fault callback V7

This add the support for the new fault callback and also the
infrastructure for supporting unmappable VRAM.

V2 validate BO with no_wait = true
V3 don't derefence bo->mem.mm_node as it's not NULL only for
VRAM or GTT
V4 update to splitted no_wait ttm change
V5 update to new balanced io_mem_reserve/free change
V6 callback is responsible for iomapping memory
V7 move back iomapping to ttm

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/ttm: split no_wait argument in 2 GPU or reserve wait

There is case where we want to be able to wait only for the
GPU while not waiting for other buffer to be unreserved. This
patch split the no_wait argument all the way down in the whole
ttm path so that upper level can decide on what to wait on or
not.

[airlied: squashed these 4 for bisectability reasons.]
drm/radeon/kms: update to TTM no_wait splitted argument
drm/nouveau: update to TTM no_wait splitted argument
drm/vmwgfx: update to TTM no_wait splitted argument
[vmwgfx patch: Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>]

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Acked-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon/kms: Add ttm page pool debugfs file.

ttm_page_pool file is hooked ttm_page_alloc_debugfs for pool
allocator state.

Signed-off-by: Pauli Nieminen <suokkos@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h

percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.

2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).

* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.

Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>

drm/radeon/kms: simplify memory controller setup V2

Get rid of _location and use _start/_end also simplify the
computation of vram_start|end & gtt_start|end. For R1XX-R2XX
we place VRAM at the same address of PCI aperture, those GPU
shouldn't have much memory and seems to behave better when
setup that way. For R3XX and newer we place VRAM at 0. For
R6XX-R7XX AGP we place VRAM before or after AGP aperture this
might limit to limit the VRAM size but it's very unlikely.
For IGP we don't change the VRAM placement.

Tested on (compiz,quake3,suspend/resume):
PCI/PCIE:RV280,R420,RV515,RV570,RV610,RV710
AGP:RV100,RV280,R420,RV350,RV620(RPB*),RV730
IGP:RS480(RPB*),RS690,RS780(RPB*),RS880

RPB: resume previously broken

V2 correct commit message to reflect more accurately the bug
and move VRAM placement to 0 for most of the GPU to avoid
limiting VRAM.

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon/kms: only evict to GTT if CP is ready

Testing GTT ready might be more correct but cp.ready
works fine and has been tested on irc by 2-3 ppl.

fixes bug k.org 15035 and fd.o 25733

Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon/kms: Avoid crash when trying to cleanup uninitialized structure

Add boolean to record if some part of the driver are initialized or
not this allow to avoid a crash when trying to cleanup uninitialized
structure members.

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon/kms: Check if bo we got from ttm are radeon object or not

If they are not radeon object don't do anythings special for them,
this avoid rare oops than can happen in a complex use case.

[airlied: additional fixups]

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon/kms: actualy set the eviction placements we choose

Stupid bug, somehow copying the eviction placements into the
result structure was missing.

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@linux.ie>

drm/radeon/kms: Convert radeon to new TTM validation API (V2)

This convert radeon to use new TTM validation API, it doesn't
really take advantage of it beside in the eviction case.

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

tree-wide: fix assorted typos all over the place

That is "success", "unknown", "through", "performance", "[re|un]mapping"
, "access", "default", "reasonable", "[con]currently", "temperature"
, "channel", "[un]used", "application", "example","hierarchy", "therefore"
, "[over|under]flow", "contiguous", "threshold", "enough" and others.

Signed-off-by: André Goddard Rosa <andre.goddard@gmail.com>
Signed-off-by: Jiri Kosina <jkosina@suse.cz>

drm/radeon/kms: Rework radeon object handling

The locking & protection of radeon object was somewhat messy.
This patch completely rework it to now use ttm reserve as a
protection for the radeon object structure member. It also
shrink down the various radeon object structure by removing
field which were redondant with the ttm information. Last it
converts few simple functions to inline which should with
performances.

airlied: rebase on top of r600 and other changes.

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon/kms: fix coherency issues on AGP cards.

When we are evicting from VRAM->RAM we allocate the ttm object,
but we don't set the caching policy on it before blitting into it.
This means on AGP we end up blitting into cached pages, and
the CPU later flushes out on top of them. This was mostly seen as
font corruption.

The other question is why we don't evict VRAM->GTT in a lot of cases,
this would save us some cache transitions since a lot of objects
that are evicted from VRAM will probably end up being pulled back in
a few operations later, and evicting them to system memory involves
2 unnecessary cache transitions.

Signed-off-by: Dave Airlie <airlied@redhat.com>

drm: fix radeon DRM warnings when !CONFIG_DEBUG_FS

Compiling the radeon DRM driver with !CONFIG_DEBUG_FS
throws the following warnings:

drivers/gpu/drm/radeon/radeon_ttm.c: In function 'radeon_ttm_debugfs_init':
drivers/gpu/drm/radeon/radeon_ttm.c:714: warning: unused variable 'i'
drivers/gpu/drm/radeon/radeon_ttm.c: At top level:
drivers/gpu/drm/radeon/radeon_ttm.c:692: warning: 'radeon_mem_types_list' defined but not used
drivers/gpu/drm/radeon/radeon_ttm.c:693: warning: 'radeon_mem_types_names' defined but not used

Fix: move these variables inside the #if defined(CONFIG_DEBUG_FS)
block in radeon_ttm_debugsfs_init(), which is the only place using them.

Signed-off-by: Mikael Pettersson <mikpe@it.uu.se>
Signed-off-by: Dave Airlie <airlied@linux.ie>

const: mark struct vm_struct_operations

* mark struct vm_area_struct::vm_ops as const
* mark vm_ops in AGP code

But leave TTM code alone, something is fishy there with global vm_ops
being used.

Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

drm/radeon/kms: add r600 KMS support

This adds the r600 KMS + CS support to the Linux kernel.

The r600 TTM support is quite basic and still needs more
work esp around using interrupts, but the polled fencing
should work okay for now.

Also currently TTM is using memcpy to do VRAM moves,
the code is here to use a 3D blit to do this, but
isn't fully debugged yet.

Authors:
Alex Deucher <alexdeucher@gmail.com>
Dave Airlie <airlied@redhat.com>
Jerome Glisse <jglisse@redhat.com>

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/mm: add ability to dump mm lists via debugfs

This adds code to the drm_mm to talk to debugfs, and adds
support to radeon to add the VRAM and GTT mm lists to debugfs.

I tested with spinlock debugging and it doesn't give out.

Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/ttm: Fixes for "Make parts of a struct ttm_bo_device global"

ttm:
Remove a stray debug printout.
Remove a re-init of the lru spinlock at device init.

radeon:
Fix the size of the bo_global allocation.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

ttm: Make parts of a struct ttm_bo_device global.

Common resources, like memory accounting and swap lists should be
global and not per device. Introduce a struct ttm_bo_global to
accomodate this, and register it with sysfs. Add a small sysfs interface
to return the number of active buffer objects.

Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: Dave Airlie <airlied@linux.ie>

drm/radeon/kms: vram sizing on certain r100 chips needs workaround.

If an rn50/r100/m6/m7 GPU has < 64MB RAM, i.e. 8/16/32, the
aperture used to calculate the MC_FB_LOCATION needs to be worked
out from the CONFIG_APER_SIZE register, and not the actual vram size.

TTM VRAM size was also being initialised wrong, use actual vram size
to initialise it.

Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: Fall back to evicting BOs with memcpy if necessary.

Otherwise if there's no GTT space we would fail the eviction, leading to
cascaded failure.

Signed-off-by: Michel Dänzer <daenzer@vmware.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon/kms: add initial colortiling support.

This adds new set/get tiling interfaces where the pitch
and macro/micro tiling enables can be set. Along with
a flag to decide if this object should have a surface when mapped.

The only thing we need to allocate with a mapped surface should be
the frontbuffer. Note rotate scanout shouldn't require one, and
back/depth shouldn't either, though mesa needs some fixes.

It fixes the TTM interfaces along Thomas's suggestions, and I've tested
the surface stealing code with two X servers and not seen any lockdep issues.

I've stopped tiling the fbcon frontbuffer, as I don't see there being
any advantage other than testing, I've left the testing commands in there,
just flip the fb_tiled to true in radeon_fb.c

Open: Can we integrate endian swapping in with this?

Future features:
texture tiling - need to relocate texture registers TXOFFSET* with tiling info.

This also merges Michel's cleanup surfaces regs at init time patch
even though it makes sense on its own, this patch really relies on it.

Some PowerMac firmwares set up a tiling surface at the beginning of VRAM
which messes us up otherwise.
that patch is:
Signed-off-by: Michel Dänzer <daenzer@vmware.com>

Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/ttm/radeon: add dma32 support.

This add support for using dma32 memory on gpus that really need it.

Currently IGPs are left without DMA32 but we might need to change
that unless we can fix rs690.

Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: Respect AGP cant_use_aperture flag.

Some AGP devices can't map the aperture, radeon needs to tell TTM this.

Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/radeon: introduce kernel modesetting for radeon hardware

Add kernel modesetting support to radeon driver, use the ttm memory
manager to manage memory and DRM/GEM to provide userspace API.
In order to avoid backward compatibility issue and to allow clean
design and code the radeon kernel modesetting use different code path
than old radeon/drm driver.

When kernel modesetting is enabled the IOCTL of radeon/drm
driver are considered as invalid and an error message is printed
in the log and they return failure.

KMS enabled userspace will use new API to talk with the radeon/drm
driver. The new API provide functions to create/destroy/share/mmap
buffer object which are then managed by the kernel memory manager
(here TTM). In order to submit command to the GPU the userspace
provide a buffer holding the command stream, along this buffer
userspace have to provide a list of buffer object used by the
command stream. The kernel radeon driver will then place buffer
in GPU accessible memory and will update command stream to reflect
the position of the different buffers.

The kernel will also perform security check on command stream
provided by the user, we want to catch and forbid any illegal use
of the GPU such as DMA into random system memory or into memory
not owned by the process supplying the command stream. This part
of the code is still incomplete and this why we propose that patch
as a staging driver addition, future security might forbid current
experimental userspace to run.

This code support the following hardware : R1XX,R2XX,R3XX,R4XX,R5XX
(radeon up to X1950). Works is underway to provide support for R6XX,
R7XX and newer hardware (radeon from HD2XXX to HD4XXX).

Authors:
Jerome Glisse <jglisse@redhat.com>
Dave Airlie <airlied@redhat.com>
Alex Deucher <alexdeucher@gmail.com>

Signed-off-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Alex Deucher <alexdeucher@gmail.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>