Revert r330897:

This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.

Revert with prejudice.

This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.

Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.

Requested by: gjb (re)

Partial merge of the SPDX changes

These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.

No objections from: pfg

MFC r323868
Modernize calls to vm_page_unwire(). As of r288122, vm_page_unwire()
accepts PQ_NONE as the specified queue and returns a Boolean indicating
whether the page's wire count transitioned to zero. Use these features
in dev/drm2.

MFC r323880:
Simplify i915_gem_wire_page() and avoid unneeded page-busying.

MFC r309712:
Use the populate() driver paging method for i915 driver.

MFC r310027:
Fix bug in r309712, do not leak gem object pin count in case of error
or retry.

MFC r307218:
Fix a race in vm_page_busy_sleep(9).

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

etc: minor spelling fixes.

Mostly comments but also some user-visible strings.

MFC after: 2 weeks

Handle the driver KPI change from r292373. Ensure that managed device
pagers fault routines always return with a result page, be it the
proper and valid result page, or initially passed freshly allocated
placeholder. Do not free the passed in page until we are able to
provide the replacement, and do not assign NULL to *mres.

Reported and tested by: dumbbell
Reviewed by: royger (who also verified that Xen code is safe)
Sponsored by: The FreeBSD Foundation

drm/i915: Fix page fault handler failure

... when __wait_seqno() is interrupted by a signal. In this case,
__wait_seqno() returns -ERESTARTSYS. Like we already do in drm_ioctl(),
we need to convert this error to a common code such as -EINTR, so the
page fault handler is restarted.

Reported by: Frederic Chardon <chardon.frederic@gmail.com>
Tested by: Frederic Chardon <chardon.frederic@gmail.com>

drm/i915: Fix malloc type in i915_gem_object_bind_to_gtt()

drm_mm.c expects DRM_MEM_MM, not DRM_I915_GEM.

drm/i915: Update to match Linux 3.8.13

This update brings initial support for Haswell GPUs.

Tested by: Many users of FreeBSD, PC-BSD and HardenedBSD
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D5554

drm/i915: Further reduce the diff with Linux 3.8

There is no functional change. The goal is to ease the future update to
Linux 3.8's i915 driver.

A change to KPI of vm_pager_get_pages() and underlying VOP_GETPAGES().

o With new KPI consumers can request contiguous ranges of pages, and
unlike before, all pages will be kept busied on return, like it was
done before with the 'reqpage' only. Now the reqpage goes away. With
new interface it is easier to implement code protected from race
conditions.

Such arrayed requests for now should be preceeded by a call to
vm_pager_haspage() to make sure that request is possible. This
could be improved later, making vm_pager_haspage() obsolete.

Strenghtening the promises on the business of the array of pages
allows us to remove such hacks as swp_pager_free_nrpage() and
vm_pager_free_nonreq().

o New KPI accepts two integer pointers that may optionally point at
values for read ahead and read behind, that a pager may do, if it
can. These pages are completely owned by pager, and not controlled
by the caller.

This shifts the UFS-specific readahead logic from vm_fault.c, which
should be file system agnostic, into vnode_pager.c. It also removes
one VOP_BMAP() request per hard fault.

Discussed with: kib, alc, jeff, scottl
Sponsored by: Nginx, Inc.
Sponsored by: Netflix

drm/i915: Remove "Attempting to unbind pinned buffer" message

This error message is removed in later versions of Linux and currently,
it spams users.

PR: 200712
MFC after: 1 week

Back out a change which should not have been committed yet

Reduce diff against linux 3.8

Reviewed by: dumbbell
Differential Revision: https://reviews.freebsd.org/D3492

drm/i915: Sort functions in i915_gem.c to match Linux 3.8's ordering

While here, reduce the style diff with Linux.

There is no functional change. The goal is to ease the future update to
Linux 3.8's i915 driver.

MFC after: 2 months

A miss from r284310. vm_pager_get_pages() updates the array, so there is
no need for vm_page_lookup().

Implement lockless resource limits.

Use the same scheme implemented to manage credentials.

Code needing to look at process's credentials (as opposed to thred's) is
provided with *_proc variants of relevant functions.

Places which possibly had to take the proc lock anyway still use the proc
pointer to access limits.

drm: Update the device-independent code to match Linux 3.8.13

This update brings few features:
o Support for the setmaster/dropmaster ioctls. For instance, they
are used to run multiple X servers simultaneously.
o Support for minor devices. The only user-visible change is a new
entry in /dev/dri but it is useless at the moment. This is a
first step to support render nodes [1].

The main benefit is to greatly reduce the diff with Linux (at the
expense of an unreadable commit diff). Hopefully, next upgrades will be
easier.

No updates were made to the drivers, beside adapting them to API
changes.

[1] https://en.wikipedia.org/wiki/Direct_Rendering_Manager#Render_nodes

Tested by: Many people
MFC after: 1 month
Relnotes: yes

An update for the i915 GPU driver, which brings the code up to Linux
commit 4d93914ae3db4a897ead4b. Some related drm infrastructure
changes are imported as needed.

Biggest update is the rewrite of the i915 gem io to more closely
follow Linux model, althought the mechanism used by FreeBSD port is
different.

Sponsored by: The FreeBSD Foundation
MFC after: 2 month

drm/i915: Add HW context support

This feature is required by Mesa 9.2+. Without this, a GL application
crashes with the following message:
# glxinfo
name of display: :0.0
Gen6+ requires Kernel 3.6 or later.
Assertion failed: (ctx->Version > 0), function handle_first_current,
file ../../src/mesa/main/context.c, line 1498.
Abort (core dumped)

Now, Mesa 10.2.4 and 10.3-rc3 works fine:
# glxinfo
name of display: :0
display: :0 screen: 0
direct rendering: Yes
...
OpenGL renderer string: Mesa DRI Intel(R) 965GM
OpenGL version string: 2.1 Mesa 10.2.4
...

The code was imported from Linux 3.8.13.

Reviewed by: kib@
Tested by: kwm@, danfe@, Henry Hu,
Lundberg, Johannes <johannes@brilliantservice.co.jp>,
Johannes Dieterich <dieterich.joh@gmail.com>,
Lutz Bichler <lutz.bichler@gmail.com>,
MFC after: 3 days
Relnotes: yes

drm: fix usage of vm_phys_fictitious_to_vm_page

vm_phys_fictitious_to_vm_page should not be called directly, even when
operating on a range that has been registered using
vm_phys_fictitious_reg_range. PHYS_TO_VM_PAGE should be used instead
because on arches that use VM_PHYSSEG_DENSE the page might come
directly from vm_page_array.

Reported by: nwhitehorn
Tested by: nwhitehorn, David Mackay <davidm.jx8p@gmail.com>
Sponsored by: Citrix Systems R&D

- Modify vm_page_unwire() and vm_page_enqueue() to directly accept
the queue where to enqueue pages that are going to be unwired.
- Add stronger checks to the enqueue/dequeue for the pagequeues when
adding and removing pages to them.

Of course, for unmanaged pages the queue parameter of vm_page_unwire() will
be ignored, just as the active parameter today.
This makes adding new pagequeues quicker.

This change effectively modifies the KPI. __FreeBSD_version will be,
however, bumped just when the full cache of free pages will be
evicted.

Sponsored by: EMC / Isilon storage division
Reviewed by: alc
Tested by: pho

Fix two cases of recursive acquisitions of the vm object lock, only
possible in rare failure situations.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week

Add a mmap flag (MAP_32BIT) on 64-bit platforms to request that a mapping use
an address in the first 2GB of the process's address space. This flag should
have the same semantics as the same flag on Linux.

To facilitate this, add a new parameter to vm_map_find() that specifies an
optional maximum virtual address. While here, fix several callers of
vm_map_find() to use a VMFS_* constant for the findspace argument instead of
TRUE and FALSE.

Reviewed by: alc
Approved by: re (kib)

Correct atomic operations in i915.

Remove the deprecated VM_ALLOC_RETRY flag for the vm_page_grab(9).
The flag was mandatory since r209792, where vm_page_grab(9) was
changed to only support the alloc retry semantic.

Suggested and reviewed by: alc
Sponsored by: The FreeBSD Foundation

On all the architectures, avoid to preallocate the physical memory
for nodes used in vm_radix.
On architectures supporting direct mapping, also avoid to pre-allocate
the KVA for such nodes.

In order to do so make the operations derived from vm_radix_insert()
to fail and handle all the deriving failure of those.

vm_radix-wise introduce a new function called vm_radix_replace(),
which can replace a leaf node, already present, with a new one,
and take into account the possibility, during vm_radix_insert()
allocation, that the operations on the radix trie can recurse.
This means that if operations in vm_radix_insert() recursed
vm_radix_insert() will start from scratch again.

Sponsored by: EMC / Isilon storage division
Reviewed by: alc (older version)
Reviewed by: jeff
Tested by: pho, scottl

The soft and hard busy mechanism rely on the vm object lock to work.
Unify the 2 concept into a real, minimal, sxlock where the shared
acquisition represent the soft busy and the exclusive acquisition
represent the hard busy.
The old VPO_WANTED mechanism becames the hard-path for this new lock
and it becomes per-page rather than per-object.
The vm_object lock becames an interlock for this functionality:
it can be held in both read or write mode.
However, if the vm_object lock is held in read mode while acquiring
or releasing the busy state, the thread owner cannot make any
assumption on the busy state unless it is also busying it.

Also:
- Add a new flag to directly shared busy pages while vm_page_alloc
and vm_page_grab are being executed. This will be very helpful
once these functions happen under a read object lock.
- Move the swapping sleep into its own per-object flag

The KPI is heavilly changed this is why the version is bumped.
It is very likely that some VM ports users will need to change
their own code.

Sponsored by: EMC / Isilon storage division
Discussed with: alc
Reviewed by: jeff, kib
Tested by: gavin, bapt (older version)
Tested by: pho, scottl

Use the right name for the vm_map_find(9) flag to specify shared
mapping. The MAP_SHARED and MAP_INHERIT_SHARE values are identical.

Noted by: alc
MFC after: 3 days

Be more aggressive in using superpages in all mappings of objects:
- Add a new address space allocation method (VMFS_OPTIMAL_SPACE) for
vm_map_find() that will try to alter the alignment of a mapping to match
any existing superpage mappings of the object being mapped. If no
suitable address range is found with the necessary alignment,
vm_map_find() will fall back to using the simple first-fit strategy
(VMFS_ANY_SPACE).
- Change mmap() without MAP_FIXED, shmat(), and the GEM mapping ioctl to
use VMFS_OPTIMAL_SPACE instead of VMFS_ANY_SPACE.

Reviewed by: alc (earlier version)
MFC after: 2 weeks

Since the gem pagefault handler relocks the vm object lock, other
thread might fault on the same GTT offset meantime and instantiate the
mapping. Recheck that the mgt device object still does not have a
page at the current offset after relocking, and return a possibly
installed page.

Reported by: Oleg Sidorkin <osidorkin@gmail.com>
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Remove stray empty line.

MFC after: 3 days

Don't busy the page unless we are likely to release the object lock.

Reviewed by: kib
Sponsored by: EMC / Isilon Storage Division

The vm_page lock is not needed around the call to vm_page_insert().

Submitted by: alc
MFC after: 1 week

Switch the vm_object mutex to be a rwlock. This will enable in the
future further optimizations where the vm_object lock will be held
in read mode most of the time the page cache resident pool of pages
are accessed for reading purposes.

The change is mostly mechanical but few notes are reported:
* The KPI changes as follow:
- VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK()
- VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK()
- VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK()
- VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED()
(in order to avoid visibility of implementation details)
- The read-mode operations are added:
VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(),
VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED()
* The vm/vm_pager.h namespace pollution avoidance (forcing requiring
sys/mutex.h in consumers directly to cater its inlining functions
using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h
consumers now must include also sys/rwlock.h.
* zfs requires a quite convoluted fix to include FreeBSD rwlocks into
the compat layer because the name clash between FreeBSD and solaris
versions must be avoided.
At this purpose zfs redefines the vm_object locking functions
directly, isolating the FreeBSD components in specific compat stubs.

The KPI results heavilly broken by this commit. Thirdy part ports must
be updated accordingly (I can think off-hand of VirtualBox, for example).

Sponsored by: EMC / Isilon storage division
Reviewed by: jeff
Reviewed by: pjd (ZFS specific review)
Discussed with: alc
Tested by: pho

Use right size when freeing unneeded GTT mapping.

MFC after: 3 days

A small simplification to i915_gem_pager_fault().

Reviewed by: kib

Add the code for new Intel GPU driver, which supports GEM, KMS and
works with new generations of GPUs (IronLake, SandyBridge and
supposedly IvyBridge).

The driver is not connected to the build yet.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week