Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC: r251782, r251783, r253994

- Remove conflicting macros from SPARC64's atomic(9) header.
- Add MD (for now) atomic_store_acq_<type>() and use it in pmap_activate()
to get the semantics when setting the PMAP right.

MFC: r241374

Add an unified macro to deny ability from the compiler to reorder
instruction loads/stores at its will.
The macro __compiler_membar() is currently supported for both gcc and
clang, but kernel compilation will fail otherwise.

Reviewed by: bde, kib
Discussed with: dim, theraven

MFC: r225890

- Add protective parentheses to macros as far as possible.
- Move {r,w,}mb() to the top of this file where they live on most of the
other architectures.

MFC: r225889, r228222

In total store which we use for running the kernel and all of the userland
atomic operations behave as if they were followed by a CPU memory barrier
so there's no need to include ones in the acquire variants of atomic(9) and
it's sufficient to just use include compiler memory barriers to satisfy
the requirements of atomic(9). Removing the CPU memory barriers results in
a small performance improvement, specifically this is sufficient to
compensate the performance loss seen in the worldstone benchmark seen when
using SCHED_ULE instead of SCHED_4BSD.
This change is inspired by Linux even more radically doing the equivalent
thing some time ago.
Thanks go to Peter Jeremy for additional testing.

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

- bump __FreeBSD version to reflect added buf_ring, memory barriers,
and ifnet functions

- add memory barriers to <machine/atomic.h>
- update drivers to only conditionally define their own

- add lockless producer / consumer ring buffer
- remove ring buffer implementation from cxgb and update its callers

- add if_transmit(struct ifnet *ifp, struct mbuf *m) to ifnet to
allow drivers to efficiently manage multiple hardware queues
(i.e. not serialize all packets through one ifq)
- expose if_qflush to allow drivers to flush any driver managed queues

This work was supported by Bitgravity Inc. and Chelsio Inc.

Oops. Use atomic_add_long() for atomic_fetchadd_long() (not atomic_add_int())
for sparc64 and sun4v.

Noticed by: marius

Implement atomic_fetchadd_long() for all architectures and document it.

Reviewed by: attilio, jhb, jeff, kris (as a part of the uidinfo_waitfree.patch)

Add a new atomic_fetchadd() primitive that atomically adds a value to a
variable and returns the previous value of the variable.

Tested on: i386, alpha, sparc64, arm (cognet)
Reviewed by: arch@
Submitted by: cognet (arm)
MFC after: 1 week

Convert the atomic_ptr() operations over to operating on uintptr_t
variables rather than void * variables. This makes it easier and simpler
to get asm constraints and volatile keywords correct.

MFC after: 3 days
Tested on: i386, alpha, sparc64
Compiled on: ia64, powerpc, amd64
Kernel toolchain busted on: arm

Use unsigned types for the arguments of the atomic(9) operations,
like described in the man page and done on all other architectures.

OK'ed by: tmm

Switch from BSD-style u_intXX_t to ISO C99 uintXX_t.

Make the atomic arithmetic functions return the old value, since they're
all implemented with cas anyway.

Use ASI_P instead of ASI_N if _KERNEL isn't defined so that these
can be used in userland.

Submitted by: tmm

Add atomic_load and store functions without membars, fwiw.

The author isn't a [UC] Regents. Correct the copyright language.

Flesh out the sparc64 port considerably. This contains:
- mostly complete kernel pmap support, and tested but currently turned
off userland pmap support
- low level assembly language trap, context switching and support code
- fully implemented atomic.h and supporting cpufunc.h
- some support for kernel debugging with ddb
- various header tweaks and filling out of machine dependent structures

Add skeleton machine dependent headers and c files for a port of freebsd
to a new architecture. This is the base of the sparc64 port, but contains
limited machine dependent code, and can be used a base for ports. Included
are:
- standard machine dependent headers, tweaked for a 64 bit, big endian
architecture, including empty versions of all the machine dependent
structures
- a machine independent atomic.h, which can be used until a port has
support for interrupts and the operations really need to be atomic
- stub versions of all the machine dependent functions, which panic
when called and print out the name of the function that needs to
be implemented. functions which are normally in assembly files are
not included, but this should reduce the number of different undefined
references on the first few compiles from hundreds to 5 or 6
Given minimal startup code and console support it should be trivial to
make this compile and run the first few sysinits on almost any architecture.

Requested by: alfred, imp, jhb