MFC r283291: don't use CALLOUT_MPSAFE with callout_init()

The main purpose of this MFC is to reduce conflicts for other merges.
Parts of the original change have already "trickled down" via individual MFCs.

MFC r307218:
Fix a race in vm_page_busy_sleep(9).

drm/i915: Restore pci_enable_busmaster() call in the init path

This fixes a GPU hang on i945GM.

While here, merge some minor fixes to DRM core and i915:
* Remove obsolete drm_agp_*_memory() prototypes
* Fix comment in drm_fops.c (outisde -> outside)
* Fix some formatting issues in drm_stub.c (spaces -> tabs)

Approved by: re (marius)
MFC of: r288653, r288952, r293851
Submitted by: <s3erios@gmail.com>
Differential Revision: https://reviews.freebsd.org/D3413

MFC 292409:
It seems certain Intel GPUs use GPIO bitbanging over a child device
instead of GMBUS access for I2C transfers. The GMBUS driver falls back
to this mode when a transfer times out. However, the first transfer to
timeout was sending the request back to itself resulting in an panic due
to recursing on a lock. Fix it to forward the request on to the proper
device. This appears to have been accidentally changed in r277487.

drm/i915: Remove "Attempting to unbind pinned buffer" message

This error message is removed in later versions of Linux and currently,
it spams users.

PR: 200712
MFC of: r289109

MFC 288452,289719:

288452:
Most error cases in i915_gem_do_execbuffer() jump to one of two labels to
release resources (such as unholding pages) when errors occur. Some
recently added error checks return immediately instead of jumping to a
label resulting in leaks. Fix these to jump to a label to do cleanup
instead.

Note that stable/9 does not have the "recently added" error checks, but
it does have some older error checks (that were are no longer present
in stable/10 and head) that have the same bug and this fixes those
instead.

289719:
i915_gem_do_execbuffer() holds the pages backing each relocation region for
various reasons while executing user commands. After these commands are
completed, the pages backing the relocation regions are unheld.

Since relocation regions do not have to be page aligned, the code in
validate_exec_list() allocates 2 extra page pointers in the array of
held pages populated by vm_fault_quick_hold_pages(). However, the cleanup
code that unheld the pages always assumed that only the buffer size /
PAGE_SIZE pages were used. This meant that non-page aligned buffers would
not unheld the last 1 or 2 pages in the list. Fix this by saving the
number of held pages returned by vm_fault_quick_hold_pages() for each
relocation region and using this count during cleanup.

drm: Update the device-independent code to match Linux 3.8.13

This update brings few features:
o Support for the setmaster/dropmaster ioctls. For instance, they
are used to run multiple X servers simultaneously.
o Support for minor devices. The only user-visible change is a new
entry in /dev/dri but it is useless at the moment. This is a
first step to support render nodes [1].

The main benefit is to greatly reduce the diff with Linux (at the
expense of an unreadable commit diff). Hopefully, next upgrades will be
easier.

No updates were made to the drivers, beside adapting them to API
changes.

[1] https://en.wikipedia.org/wiki/Direct_Rendering_Manager#Render_nodes

r280814 is merged at the same time to avoid a short window where RANDR
might be broken:

drm: Import Linux commit 9bc3cd5673d84d29272fa7181a4dfca83cbb48c1

Author: Ville Syrjälä <ville.syrjala@linux.intel.com>
Date: Fri May 31 12:17:08 2013 +0000

drm: Sort connector modes based on vrefresh

Keeping the modes sorted by vrefresh before the pixel clock makes the
mode list somehow more pleasing to the eye.

Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

PR: 198936 (r280814)
Tested by: Many people
MFC of: r280183, r280187 (original commit by glebius), r280814
Relnotes: yes

MFC r277487:
An update for the i915 GPU driver, which brings the code up to Linux
commit 4d93914ae3db4a897ead4b.

MFC r277959 (by adrian):
Fix backlight for ivybridge based laptops (and whatever else comes through
this codepath.)

MFC r278146:
Do not attach to the unsupported chipsets, unless magic tunable is
frobbed.

MFC r278147, r278148:
Fix sign for the error code returned from the driver-specific code.

MFC r278152:
Do not access gmbus_ports array past its end.

MFC r278159 (by emaste):
Remove duplicate intel_fbc_enabled prototype.

MFC 270516:
i915 driver - enable opregion handle; program CADL.

add opregion handling for drm2 - which exposes some ACPI video configuration
pieces that some Lenovo laptop models use to flesh out which video device
to speak to. This enables the brightness control in ACPI to work these models.

The CADL bits are also important - it's used to figure out which ACPI
events to hook the brightness buttons into. It doesn't yet seem to work
for me, but it does for the OP.

PR: 190186, 198551
Submitted by: Henry Hu <henry.hu.sh@gmail.com>

MFC r269634:

drm: fix usage of vm_phys_fictitious_to_vm_page

vm_phys_fictitious_to_vm_page should not be called directly, even when
operating on a range that has been registered using
vm_phys_fictitious_reg_range. PHYS_TO_VM_PAGE should be used instead
because on arches that use VM_PHYSSEG_DENSE the page might come
directly from vm_page_array.

Reported by: nwhitehorn
Tested by: nwhitehorn, David Mackay <davidm.jx8p@gmail.com>
Sponsored by: Citrix Systems R&D

MFC r272761:
Add an argument to the x86 pmap_invalidate_cache_range() to request
forced invalidation of the cache range regardless of the presence of
self-snoop feature.

MFC r272943:
MFi386 r272761.

drm/i915: Add HW context support

This feature is required by Mesa 9.2+. Without this, a GL application
crashes with the following message:
# glxinfo
name of display: :0.0
Gen6+ requires Kernel 3.6 or later.
Assertion failed: (ctx->Version > 0), function handle_first_current,
file ../../src/mesa/main/context.c, line 1498.
Abort (core dumped)

Now, Mesa 10.2.4 and 10.3-rc3 works fine:
# glxinfo
name of display: :0
display: :0 screen: 0
direct rendering: Yes
...
OpenGL renderer string: Mesa DRI Intel(R) 965GM
OpenGL version string: 2.1 Mesa 10.2.4
...

The code was imported from Linux 3.8.13.

This an MFC of r271705.

Approved by: re (glebius)
Reviewed by: kib@
Tested by: kwm@, danfe@, Henry Hu,
Lundberg, Johannes <johannes@brilliantservice.co.jp>,
Johannes Dieterich <dieterich.joh@gmail.com>,
Lutz Bichler <lutz.bichler@gmail.com>,
Relnotes: yes

MFC r268947: Hide syscons-specific workaround under DEV_SC

MFC r265102:
Fix two cases of recursive acquisitions of the vm object lock, only
possible in rare failure situations.

MFC 259016,259019,259049,259071,259102,259110,259129,259130,259178,259179,
259203,259221,259261,259532,259615,259650,259651,259667,259680,259727,
259761,259772,259776,259777,259830,259882,259915,260160,260449,260450,
260688,260888,260953,261269,261547,261551,261552,261553,261585:
Merge the vt(4) driver (newcons) to stable/10.

Approved by: ray

MFC r258779,r258780,r258787,r258822:

Fix undefined behavior: (1 << 31) is not defined as 1 is an int and this
shifts into the sign bit. Instead use (1U << 31) which gets the
expected result.

Similar to the (1 << 31) case it is not defined to do (2 << 30).

This fix is not ideal as it assumes a 32 bit int, but does fix the issue
for most cases.

A similar change was made in OpenBSD.

MFC r256848:
Use plain register read for waiting of the reset completion
notification, to avoid gt_lock recursion.

Approved by: re (glebius)

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Add a mmap flag (MAP_32BIT) on 64-bit platforms to request that a mapping use
an address in the first 2GB of the process's address space. This flag should
have the same semantics as the same flag on Linux.

To facilitate this, add a new parameter to vm_map_find() that specifies an
optional maximum virtual address. While here, fix several callers of
vm_map_find() to use a VMFS_* constant for the findspace argument instead of
TRUE and FALSE.

Reviewed by: alc
Approved by: re (kib)

Correct atomic operations in i915.

drm: Move definition of EREMOTEIO to drmP.h

It will be used by both i915 and radeon drivers.

Add ERESTARTSYS definition at the same time.

drm: Const'ify the 1st "drm_display_mode" passed to "mode_fixup" callbacks

This will be needed by the Radeon KMS driver.

drm/i915: Import Linux commit 71244653a8fb0f46bc12ae421f1d5f72af6a75da

Author: Daniel Vetter <daniel.vetter@ffwll.ch>
Date: Mon Jun 4 18:39:20 2012 +0200

drm/i915: adjusted_mode->clock in the dp mode_fixup

... instead of changing mode->clock, which we should leave as-is.

After the previous patch we only touch that if it's a panel, and then
adjusted mode->clock equals adjusted_mode->clock. Outside of
intel_dp.c we only use ajusted_mode->clock in the mode_set functions.

Within intel_dp.c we only use it to calculate the dp dithering
and link bw parameters, so that's the only thing we need to fix
up.

As a temporary ugliness (until the cleanup in the next patch) we
pass the adjusted_mode into dp_dither for both parameters (because
that one still looks at mode->clock).

Note that we do overwrite adjusted_mode->clock with the selected dp
link clock, but that only happens after we've calculated everything we
need based on the dotclock of the adjusted output configuration.

Outside of intel_dp.c only intel_display.c uses adjusted_mode->clock,
and that stays the same after this patch (still equals the selected dp
link clock). intel_display.c also needs the actual dotclock (as
target_clock), but that has been fixed up in the previous patch.

v2: Adjust the debug message to also use adjusted_mode->clock.

Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

Remove the deprecated VM_ALLOC_RETRY flag for the vm_page_grab(9).
The flag was mandatory since r209792, where vm_page_grab(9) was
changed to only support the alloc retry semantic.

Suggested and reviewed by: alc
Sponsored by: The FreeBSD Foundation

On all the architectures, avoid to preallocate the physical memory
for nodes used in vm_radix.
On architectures supporting direct mapping, also avoid to pre-allocate
the KVA for such nodes.

In order to do so make the operations derived from vm_radix_insert()
to fail and handle all the deriving failure of those.

vm_radix-wise introduce a new function called vm_radix_replace(),
which can replace a leaf node, already present, with a new one,
and take into account the possibility, during vm_radix_insert()
allocation, that the operations on the radix trie can recurse.
This means that if operations in vm_radix_insert() recursed
vm_radix_insert() will start from scratch again.

Sponsored by: EMC / Isilon storage division
Reviewed by: alc (older version)
Reviewed by: jeff
Tested by: pho, scottl

The soft and hard busy mechanism rely on the vm object lock to work.
Unify the 2 concept into a real, minimal, sxlock where the shared
acquisition represent the soft busy and the exclusive acquisition
represent the hard busy.
The old VPO_WANTED mechanism becames the hard-path for this new lock
and it becomes per-page rather than per-object.
The vm_object lock becames an interlock for this functionality:
it can be held in both read or write mode.
However, if the vm_object lock is held in read mode while acquiring
or releasing the busy state, the thread owner cannot make any
assumption on the busy state unless it is also busying it.

Also:
- Add a new flag to directly shared busy pages while vm_page_alloc
and vm_page_grab are being executed. This will be very helpful
once these functions happen under a read object lock.
- Move the swapping sleep into its own per-object flag

The KPI is heavilly changed this is why the version is bumped.
It is very likely that some VM ports users will need to change
their own code.

Sponsored by: EMC / Isilon storage division
Discussed with: alc
Reviewed by: jeff, kib
Tested by: gavin, bapt (older version)
Tested by: pho, scottl

Replace kernel virtual address space allocation with vmem. This provides
transparent layering and better fragmentation.

- Normalize functions that allocate memory to use kmem_*
- Those that allocate address space are named kva_*
- Those that operate on maps are named kmap_*
- Implement recursive allocation handling for kmem_arena in vmem.

Reviewed by: alc
Tested by: pho
Sponsored by: EMC / Isilon Storage Division

The pmap_qenter() and pmap_qremove() perform TLB invalidation on its
own, no need to call pmap_invalidate_range() one more time.

Noted by: alc
MFC after: 1 week

Use the right name for the vm_map_find(9) flag to specify shared
mapping. The MAP_SHARED and MAP_INHERIT_SHARE values are identical.

Noted by: alc
MFC after: 3 days

Be more aggressive in using superpages in all mappings of objects:
- Add a new address space allocation method (VMFS_OPTIMAL_SPACE) for
vm_map_find() that will try to alter the alignment of a mapping to match
any existing superpage mappings of the object being mapped. If no
suitable address range is found with the necessary alignment,
vm_map_find() will fall back to using the simple first-fit strategy
(VMFS_ANY_SPACE).
- Change mmap() without MAP_FIXED, shmat(), and the GEM mapping ioctl to
use VMFS_OPTIMAL_SPACE instead of VMFS_ANY_SPACE.

Reviewed by: alc (earlier version)
MFC after: 2 weeks

On some generations of the Intel GPU, disabling of the VGA Display
stops updating the vertical retrace indicator. The text mouse
renderer in syscons is executing from the callout and spins waiting
for the start of next frame. As result, after the X server finishes,
since the VGA cannot be turned on, but syscons does not know about
this, the clock swi spins forever.

Hack around the problem by disabling wait for the retrace if KMS is
activated.

Diagnosed and tested by: Michiel Boland <boland37@xs4all.nl>
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Since the gem pagefault handler relocks the vm object lock, other
thread might fault on the same GTT offset meantime and instantiate the
mapping. Recheck that the mgt device object still does not have a
page at the current offset after relocking, and return a possibly
installed page.

Reported by: Oleg Sidorkin <osidorkin@gmail.com>
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

Remove stray empty line.

MFC after: 3 days

Don't busy the page unless we are likely to release the object lock.

Reviewed by: kib
Sponsored by: EMC / Isilon Storage Division

The vm_page lock is not needed around the call to vm_page_insert().

Submitted by: alc
MFC after: 1 week

drm and i915: Left-shift iic_msg.slave at creation time

This is required because, in the radeon driver, we can't left-shift in a
central place, like it was done in the i915 driver.

Reviewed by: kib@, kan@, avg@
Tested by: kib@, avg@

Switch the vm_object mutex to be a rwlock. This will enable in the
future further optimizations where the vm_object lock will be held
in read mode most of the time the page cache resident pool of pages
are accessed for reading purposes.

The change is mostly mechanical but few notes are reported:
* The KPI changes as follow:
- VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK()
- VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK()
- VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK()
- VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED()
(in order to avoid visibility of implementation details)
- The read-mode operations are added:
VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(),
VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED()
* The vm/vm_pager.h namespace pollution avoidance (forcing requiring
sys/mutex.h in consumers directly to cater its inlining functions
using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h
consumers now must include also sys/rwlock.h.
* zfs requires a quite convoluted fix to include FreeBSD rwlocks into
the compat layer because the name clash between FreeBSD and solaris
versions must be avoided.
At this purpose zfs redefines the vm_object locking functions
directly, isolating the FreeBSD components in specific compat stubs.

The KPI results heavilly broken by this commit. Thirdy part ports must
be updated accordingly (I can think off-hand of VirtualBox, for example).

Sponsored by: EMC / Isilon storage division
Reviewed by: jeff
Reviewed by: pjd (ZFS specific review)
Discussed with: alc
Tested by: pho

Reduce delays in several wait loops from 10ms to 10us, same is it is done
in Linux. This substantially increases graphics performance on Ivy Bridge.

Submitted by: avg@
Reviewed by: kib@

Prefer __containerof() above member2struct().

The first does proper checking of the argument types, while the latter
does not.

Add drm and i915 ioctl translations for 32 bit process on 64 bit host.

Submitted by: meowthink@gmail.com
MFC after: 2 weeks

Use right size when freeing unneeded GTT mapping.

MFC after: 3 days

A small simplification to i915_gem_pager_fault().

Reviewed by: kib

Disable end of buffer fixup by default. New DDX does not need this, and
since batch_len is unused by Linux driver, it seems that it is sometimes
gets passed wrong. This causes command buffer corruption and GPU hung.

Old GEMified DDX drivers that needs this workaround are not supported.

MFC after: 1 month

Fix calculation of the execution buffer end in the mapped pages
when it is spilled into the next page.

MFC after: 1 month

Fix the following clang warning in drm2:

sys/dev/drm2/i915/intel_display.c:8861:3: error: expression result unused [-Werror,-Wunused-value]
_intel_wait_for(dev,
^~~~~~~~~~~~~~~~~~~~
@/dev/drm2/i915/intel_drv.h:55:2: note: expanded from macro '_intel_wait_for'
ret; \
^~~

Reviewed by: kib
MFC after: 1 week

Add the code for new Intel GPU driver, which supports GEM, KMS and
works with new generations of GPUs (IronLake, SandyBridge and
supposedly IvyBridge).

The driver is not connected to the build yet.

Sponsored by: The FreeBSD Foundation
MFC after: 1 week