MFC r356563:
UMA: Don't destroy zones after the system shutdown process starts.

PR: 242427

MFC r348764: Allow UMA hash tables to expand faster then 2x in 20 seconds.

ZFS ABD allocates tons of 4KB chunks via UMA, requiring huge hash tables.
With initial hash table size of only 32 elements it takes ~20 expansions
or ~400 seconds to adapt to handling 220GB ZFS ARC. During that time not
only the hash table is highly inefficient, but also each of those expan-
sions takes significant time with the lock held, blocking operation.

On my test system with 256GB of RAM and ZFS pool of 28 HDDs this change
reduces time needed to first time read 240GB from ~300-400s, during which
system is quite busy and unresponsive, to only ~150s with light CPU load
and just 5 sub-second CPU spikes to expand the hash table.

MFC r344042:
UMA: unsign some variables related to allocation in hash_alloc().

As a followup to r343673, unsign some variables related to allocation
since the hashsize cannot be negative. This gives a bit more space to
handle bigger allocations and avoid some implicit casting.

While here also unsign uh_hashmask, it makes little sense to keep it
signed.

Differential Revision: https://reviews.freebsd.org/D19148

MFC r343673: Fix integer math overflow in UMA hash_alloc().

512GB of ZFS ABD ARC means abd_chunk zone of 128M 4KB items. To manage
them UMA tries to allocate 2GB hash table, which size does not fit into
the int variable, causing later allocation failure, which makes ARC shrink
back below the 512GB, not letting it to use more RAM. With this change I
easily reached >700GB ARC size on 768GB RAM machine.

Sponsored by: iXsystems, Inc.

MFC r332968:
Add a UMA zone flag to disable the use of buckets.

Merge r331871:
Handle a special case when a slab can fit only one allocation,
and zone has a large alignment. With alignment taken into
account uk_rsize will be greater than space in a slab. However,
since we have only one item per slab, it is always naturally
aligned.

Code that will panic before this change with 4k page:

z = uma_zcreate("test", 3984, NULL, NULL, NULL, NULL, 31, 0);
uma_zalloc(z, M_WAITOK);

A practical scenario to hit the panic is a machine with 56 CPUs
and 2 NUMA domains, which yields in zone size of 3984 (on head).

PR: 227116

MFC r331489:
For vm_zone_stats() sysctl handler, do not drain sbuf calling copyout(9)
while owning zone lock.

MFC r325530 (jeff), r325566 (kib), r325588 (kib):
Replace many instances of VM_WAIT with blocking page allocation flags.

MFC r323234,r323305,r323306,r324044:

Start annotating global _padalign locks with __exclusive_cache_line

While these locks are guarnteed to not share their respective cache lines,
their current placement leaves unnecessary holes in lines which preceeded them.

For instance the annotation of vm_page_queue_free_mtx allows 2 neighbour
cachelines (previously separate by the lock) to be collapsed into 1.

The annotation is only effective on architectures which have it implemented in
their linker script (currently only amd64). Thus locks are not converted to
their not-padaligned variants as to not affect the rest.

=============

Annotate global process locks with __exclusive_cache_line

=============

Annotate Giant with __exclusive_cache_line

=============

Annotate sysctlmemlock with __exclusive_cache_line.

MFC r323544:
Fix a logic error in the item size calculation for internal UMA zones.

MFC 316493: Assert that the align parameter to uma_zcreate() is valid.

MFC r315078: uma: fix pages <-> items conversions at several places

MFC r315077: uma: eliminate uk_slabsize field

MFC r314272: call vm_lowmem hook in uma_reclaim_worker

MFC r307693:
Simplify keg_drain() a bit by using LIST_FOREACH_SAFE.

MFC r303059
Release the second critical section in uma_zfree_arg() slightly earlier.