MFC r342630:
Add 'v' modifier to the ddb 'show pginfo' command to display vm_page
backing the provided kernel virtual address.

MFC r341247:
Update the free page count when blacklisting pages.

PR: 231296

MFC r332658:
Ensure that m and skip_m belong to the same object.

MFC r332182:
Handle Skylake-X errata SKZ63.

Revert r330897:

This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.

Revert with prejudice.

This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.

Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.

Requested by: gjb (re)

MFC r331128:
Have vm_page_replace() assert that the new page is not enqueued.

MFC r331425:
Correct a couple of assertion messages in vm_page_reclaim_run().

Partial merge of the SPDX changes

These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.

No objections from: pfg

MFC r324610:

Reduce traffic on vm_cnt.v_free_count

The variable is modified with the highly contended page free queue lock.
It unnecessarily shares a cacheline with purely read-only fields and is
re-read after the lock is dropped in the page allocation code making the
hold time longer.

Pad the variable just like the others and store the value as found with
the lock held instead of re-reading.

Provides a modest 1%-ish speed up in concurrent page faults.

Due to KBI constraints the field is not moved in this commit, only re-read is
avoided.

MFC r327168, r327213:
Fix two problems with the page daemon control loop.

MFC r325530 (jeff), r325566 (kib), r325588 (kib):
Replace many instances of VM_WAIT with blocking page allocation flags.

MFC r322547:
Add vm_page_alloc_after().

MFC r323234,r323305,r323306,r324044:

Start annotating global _padalign locks with __exclusive_cache_line

While these locks are guarnteed to not share their respective cache lines,
their current placement leaves unnecessary holes in lines which preceeded them.

For instance the annotation of vm_page_queue_free_mtx allows 2 neighbour
cachelines (previously separate by the lock) to be collapsed into 1.

The annotation is only effective on architectures which have it implemented in
their linker script (currently only amd64). Thus locks are not converted to
their not-padaligned variants as to not affect the rest.

=============

Annotate global process locks with __exclusive_cache_line

=============

Annotate Giant with __exclusive_cache_line

=============

Annotate sysctlmemlock with __exclusive_cache_line.

MFC r326234, r326235, r326284:
vm_page_array initialization improvements.

MFC r326055:
Allow for fictitious physical pages in vm_page_scan_contig().

MFC r324824:
Check that the page which is freed as zeroed, indeed has all-zero content.

MFC r324793:
In vm_page_free_phys_pglist(), do not take vm_page_queue_free_mtx if
there is nothing to do.

MFC r320980,321377
Generalize vm_page_ps_is_valid() to support testing other predicates on
the (super)page, renaming the function to vm_page_ps_test().

In vm_page_ps_test(), always check that the base pages within the specified
superpage all belong to the same object. To date, that check has not been
needed, but upcoming changes require it.

MFC r323973,324087
Optimize vm_page_try_to_free(). Specifically, the call to pmap_remove_all()
can be avoided when the page's containing object has a reference count of
zero. (If the object has a reference count of zero, then none of its pages
can possibly be mapped.)

Address nearby style issues in vm_page_try_to_free(), and change its
return type to "bool".

Optimize vm_object_page_remove() by eliminating pointless calls to
pmap_remove_all(). If the object to which a page belongs has no
references, then that page cannot possibly be mapped.

MFC r323290:
Speed up vm_page_array initialization.

MFC r323544:
Fix a logic error in the item size calculation for internal UMA zones.

MFC r323562:
Remove inline specifier from vm_page_free_wakeup(), do not
micro-manage compiler.

MFC r323559:
Split vm_page_free_toq().

MFC r322405, r322406:
Modify vm_page_grab_pages() to handle VM_ALLOC_NOWAIT, use it in
sendfile_swapin().

MFC r322296
Introduce vm_page_grab_pages(), which is intended to replace loops calling
vm_page_grab() on consecutive page indices. Besides simplifying the code
in the caller, vm_page_grab_pages() allows for batching optimizations.
For example, the current implementation replaces calls to vm_page_lookup()
on consecutive page indices by cheaper calls to vm_page_next().

MFC r323368:
Add a vm_page_change_lock() helper.

MFC r322383:
Make vm_page_sunbusy() assert that the page is unlocked.

MFC r312208, r312994:
Optimize vm_object_madvise().

MFC r309017 (partially):
Omit v_cache_count when computing the number of free pages, since its
value is always 0.

MFC 319702: Fix an off-by-one error in the VM page array on some systems.

r313186 changed how the size of the VM page array was calculated to be
less wasteful. For most systems, the amount of memory is divided by
the overhead required by each page (a page of data plus a struct vm_page)
to determine the maximum number of available pages. However, if the
remainder for the first non-available page was at least a page of data
(so that the only memory missing was a struct vm_page), this last page
was left in phys_avail[] but was not allocated an entry in the VM page
array. Handle this case by explicitly excluding the page from
phys_avail[].

Approved by: re (kib)

Null pointer must be checked before use. This fixes a regression introduced
in r318716.

Note it is a direct commit to stable/11 because head removed support for
idle page zeroing in r305362.

PR: 219994
Reviewed by: markj
Approved by: re (gjb)

MFC r308474, r308691, r309203, r309365, r309703, r309898, r310720,
r308489, r308706:
Add PQ_LAUNDRY and remove PG_CACHED pages.

MFC r313186
Over the years, the code and comments in vm_page_startup() have diverged
in one respect. When determining how many page structures to allocate,
contrary to what the comments say, the code does not account for the
overhead of a page structure per page of physical memory. This revision
changes the code to match the comments.

MFC r310834:
Assert that the pages found on the object queue by vm_page_next() and
vm_page_prev() have correct ownership.

MFC r308288:
Do not sleep in vm_wait() if pagedaemon did not yet started. Panic instead.

MFC r306712
Make the page daemon's notion of what kind of pass is being performed
by vm_pageout_scan() local to vm_pageout_worker(). There is no reason
to store the pass in the NUMA domain structure.

MFC r307499:
Export vm_page_xunbusy_maybelocked().

MFC r307218:
Fix a race in vm_page_busy_sleep(9).

MFC r307064:
When downgrading exclusively busied page to shared-busy state, wakeup
waiters.

MFC r305056, r305367:
Restore swap pager readahead.

MFC r304053, r304054:
Initialize busy lock state and strengthen busy lock assertions.

MFC r303243
Update a comment in vm_page_advise() to match behaviour after r290529.

MFC r303516
Use vm_page_undirty() instead of manually setting a page field.