Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Retire the INTR_FAST flag as it was obsoleted by the introduction of the
filter argument to bus_setup_intr().

Add some cross references to locking.9 from related pages.

Allow threads to atomically release rw and sx locks while waiting for an
event. Locking primitives that support this (mtx, rw, and sx) now each
include their own foo_sleep() routine.
- Rename msleep() to _sleep() and change it's 'struct mtx' object to a
'struct lock_object' pointer. _sleep() uses the recently added
lc_unlock() and lc_lock() function pointers for the lock class of the
specified lock to release the lock while the thread is suspended.
- Add wrappers around _sleep() for mutexes (mtx_sleep()), rw locks
(rw_sleep()), and sx locks (sx_sleep()). msleep() still exists and
is now identical to mtx_sleep(), but it is deprecated.
- Rename SLEEPQ_MSLEEP to SLEEPQ_SLEEP.
- Rewrite much of sleep.9 to not be msleep(9) centric.
- Flesh out the 'RETURN VALUES' section in sleep.9 and add an 'ERRORS'
section.
- Add __nonnull(1) to _sleep() and msleep_spin() so that the compiler will
warn if you try to pass a NULL wait channel. The functions already have
a KASSERT to that effect.

Place mtx_destroy() and rw_destroy() right after mtx_init() and rw_init()
to match sx(9) and condvar(9).

Document MTX_NOPROFILE flag.

Replace a reference to no longer existing MUTEX_PROFILING(9) with
a LOCK_PROFILING(9).

o Fix MTX_SYSINIT prototype.

PR: kern/99956
Submitted by: Arne Woerner
MFC after: 1 week

Xref rwlock(9).

Reviewed by: ru

Various and sundry improvements:
- Replace 'process' with 'thread' everywhere.
- Update several places to note that that the fact that default mutexes
may adaptively spin isn't necessarily MD, but is just part of the
implementation as a whole.
- Clarify the text about MTX_SPIN mutexes only being appropriate for
INTR_FAST interrupts or other low level scheduler code to make the
jargon more FreeBSD-ish rather than BSD/OS-ish.
- Also, note that it is possible that interrupts aren't blocked but just
deferred when a spin lock is held (the whole blocked vs. deferred bit is
an MD implementation detail).
- Remove statements saying that spin locks must be released in the exact
opposite order that they were acquired. This stopped being true several
years ago when we first added critical sections that stored their state
in the current thread rather than in struct mtx.
- Note that a mutex must be initialized before it is passed to any other
mutex function, not just mtx_lock.
- Clarify that mtx_trylock() only operates on MTX_DEF mutexes.
- Simplify the text about possible preemption during a mtx_unlock().
- Use complete English sentences in place of phrases in a few places.
- Clarify that it isn't ever safe to sleep with a mutex held. The kernel
tends to panic when you do that.

Requested by: scottl (7)
MFC after: 3 days

Don't say that mtx_lock() will "sleep" if another kernel thread is
holding the mutex, say it will "block". Later in this manual page
we say that sleeping while holding a mutex isn't allowed, and this
can be confusing.

Submitted by: jhb

Expand *n't contractions.

Fixed the misplaced $FreeBSD$.

"non-spin" is better with a dash, than without.

Fix the NAME section making whatis(1) happy in particular.

Mechanically kill hard sentence breaks and double whitespaces.

Assorted markup, spelling, and grammar fixes.

Add a cross-reference to MUTEX_PROFILING(9).

Add "options INVARIANTS" and "options INVARIANT_SUPPORT" to the
synopsis, with supporting text in the body of the manual page.

Add a cross-reference to panic(9) for completeness.

Reviewed by: ru (synopsis changes)

Use of the MTX_SYSINIT macro requires the inclusion of <sys/kernel.h>.

Typo fix.

Assorted mdoc(7), grammar, spelling, and punctuation fixes.

Approved by: re (blanket)

Back out revision 1.34 until we can come up with a more accurate patch.

Add a section on Giant lock manipulation macros.

Submitted by: Hiten M. Pandya <hiten@angelica.unixdaemons.com>
Reviewed by: bmilekic

Add some documentation of FreeBSD's special synchronization quirks
which may surprise developers coming from Solaris, or other platforms
which have a similar interface, but slightly different rules.

Reviewed by: jhb, ru

Remove all mention of MTX_NOSWITCH. It was obsoleted a long time ago.

Document MTX_DUPOK

Requested by: jeff (long ago)

mdoc(7) police: lint.

Remove a duplicate description of the type parameter for mtx_init.

Noticed by: scottl

Add text describing the new type parameter, and rename description
to name as that is what it is called in mutex.h.

Document the lock type argument of mtx_init().

Update the .Fn line for MTX_SYSINIT to include the parameter types.

- Add notes about SX_SYSINIT and MTX_SYSINIT in order to document the
recent additions of these macros.
- Add in a MLINK to create SX_SYSINIT.9 and MTX_SYSINIT.9.

Add mtx_pool(9) to the SEE ALSO section.

Submitted by: Garrett Rooney <rooneg@electricjellyfish.net>

ispell sweep of share/man/man9/*.

mdoc(7) police: Use the new .In macro for #include statements.

Common Mike adds see alsos from all of the synchronization
man pages to their brethren.

mdoc(7) police: Please don't commit when you are drunk!

(Committer rules candidate.)

From Mike Silbersack's message on cvs-commits:

> > Doh! There is a sx man page, the mutex man page just doesn't have a SEE
> > ALSO pointing to sx(9). Could some mdoc madman go add one?

mdoc(7) police: s;BSD/OS;.Bsx; where appropriate.

can not -> cannot

Removed whitespace at end-of-line; no content changes. I simply did
cd src/share; find man[1-9] -type f|xargs perl -pi -e 's/[ \t]+$//'

BTW, what editors are the culprits? I'm using vim and it shows
me whitespace at EOL in troff files with a thick blue block...

Reviewed by: Silence from cvs diff -b
MFC after: 7 days

s/mtx_try_enter/mtx_trylock/

- Document mtx_initialized().
- Document that MTX_QUIET can be passed to mtx_init() and the new flag
MTX_NOWITNESS.

Fixed wrong include in synopsis. <sys/param.h> is a prerequisite for
<sys/mutex.h> due to #include spam in <sys/mutex.h>. (More precisely,
<sys/time.h> is the prerequisite, but that is provided by standard
#include spam in <sys/param.h>.)

Fixed bitrot in prototype for mtx_init().

- Both <sys/sx.h> and <sys/mutex.h> depend on <sys/types.h> and
<sys/lock.h>.
- <sys/sx.h> depends on <sys/mutex.h>.

mdoc(7) cleanup.

Catch up to mutex API changes.

mtx_try_enter() returns an int, not void.

Remove MUTEX_DECLARE() and MTX_COLD. Instead, postpone full mutex
initialization until after malloc() is safe to call, then iterate through
all mutexes and complete their initialization.

This change is necessary in order to avoid some circular bootstrapping
dependencies.

mdoc(7) police: add a missing .Os, change the list type to ``tagged''.

Document the MA_RECURSED and MA_NOTRECURSED assertion flags as well as
the MTX_QUIET flag.

Catch up to mutexes becoming MI: machine/mutex.h -> sys/mutex.h

Whitespace only:

* Remove hard sentence breaks.
* Remove trailing whitespace.
* Separate macro arguments from trailing punctuation with whitespace.

- Document MUTEX_DECLARE and MTX_COLD
- Clean up some minor nits

Huge improvement to the mutex man page.

o Document all the mutex calls, not just the entry and exit.

o Fix the mtx_t typedef (now is struct mtx).

Major update to the way synchronization is done in the kernel. Highlights
include:

* Mutual exclusion is used instead of spl*(). See mutex(9). (Note: The
alpha port is still in transition and currently uses both.)

* Per-CPU idle processes.

* Interrupts are run in their own separate kernel threads and can be
preempted (i386 only).

Partially contributed by: BSDi (BSD/OS)
Submissions by (at least): cp, dfr, dillon, grog, jake, jhb, sheldonh