kernel_lock isn't needed to synchronise kthread_exit() and kthread_join().

kthread(9): Fix nested kthread_join.

No reason for one kthread_join to interfere with another, or to cause
non-cyclic dependencies to get stuck.

Uses struct lwp::l_private for this purpose, which for user threads
stores the tls pointer. I don't think anything in kthread(9) uses
l_private -- generally kernel threads will use lwp specificdata. But
maybe this should use a new member instead, or a union member with an
existing pointer for the purpose.

kthread(9): Attribute uarea to caller for kmsan.

New functions kthread_fpu_enter/exit.

The MI definitions don't do anything but maintain a flag, but MD code
can define kthread_fpu_enter/exit_md to actually enable/disable the
FPU. (These are almost pcu_load/discard on systems that use pcu(9),
except they apply to all PCUs.)

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2020/06/20/msg026524.html

The proposed kthread flag KTHREAD_FPU is not included because I
couldn't find any particular need for it that would not be covered by
just calling kthread_fpu_enter/exit in the kthread function.

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

branches: 1.44.2;
Minor scheduler cleanup:

- Adapt to cpu_need_resched() changes. Avoid lost & duplicate IPIs and ASTs.
sched_resched_cpu() and sched_resched_lwp() contain the logic for this.
- Changes for LSIDL to make the locking scheme match the intended design.
- Reduce lock contention and false sharing further.
- Numerous small bugfixes, including some corrections for SCHED_FIFO/RT.
- Use setrunnable() in more places, and merge cut & pasted code.

branches: 1.43.4;
KASSERT() that kthread_join()'s target is expecting to be joined.

As discussed on IRC.

- Propagate the signal mask from the ucontext_t to the newly created thread
as specified by _lwp_create(2)
- Reset the signal stack for threads created with _lwp_create(2)

branches: 1.41.2; 1.41.4;
Remove the unused and all-but-unusable kthread_destroy(). Also makes
it more obvious that the opposite of kthread_create() is kthread_exit().

Don't check if constant-sized KM_SLEEP allocations succeeded.

branches: 1.39.2; 1.39.16;
Add a __HAVE_CPU_UAREA_IDLELWP hook so that the MD code can allocate
special UAREAs for idle lwp's.

branches: 1.38.2; 1.38.10;
Avoid crashing when we do not explictly pass down a CPU when creating
a kthread(9) in the LSIDL (KTHREAD_IDLE) state.

Typo fix.

Rename slightly misleading KTHREAD_JOINABLE to KTHREAD_MUSTJOIN.

Retire varargs.h support. Move machine/stdarg.h logic into MI
sys/stdarg.h and expect compiler to provide proper builtins, defaulting
to the GCC interface. lint still has a special fallback.
Reduce abuse of _BSD_VA_LIST_ by defining __va_list by default and
derive va_list as required by standards.

Use a real panic as safe guard

Re-implement kthread_join(9), so that it actually works (hi haad@).

Hold the current lwp's mutex, instead of the lwp we want to wait for
in kthread_join(). This fixes panics with DIAGNOSTIC and possibly some
rare race conditons.

Add support for cpu-specific uarea allocation routines. Allows different
allocation for user and system lwps. MIPS will use this to map uareas of
system lwp used direct-mapped addresses (to reduce the overhead of
switching to kernel threads). ibm4xx could use to map uareas via direct
mapped addresses and avoid the problem of having the kernel stack not in
the TLB.

branches: 1.30.2; 1.30.4;
increment p_nrlwps in lwp_create rather than letting callers do so
as it's always decremented by lwp_exit. this fixes error recovery of
eg. aio_procinit.

Fix bogus KASSERT, typo and KNFize my previous patch.

Add support for kthread_join in our kernel thread implementation. This is used
by zfs but I think that it can be generaly usefull. Thread need to be created
with KTHREAD_JOINABLE flag and can be joined only once.
When joinable thread was created it will not e automatically reaped from system
and kthread_join must be called on it to reap it.

Ok by ad@.

branches: 1.27.2; 1.27.4;
Remove uarea swap-out functionality:

- Addresses the issue described in PR/38828.
- Some simplification in threading and sleepq subsystems.
- Eliminates pmap_collect() and, as a side note, allows pmap optimisations.
- Eliminates XS_CTL_DATA_ONSTACK in scsipi code.
- Avoids few scans on LWP list and thus potentially long holds of proc_lock.
- Cuts ~1.5k lines of code. Reduces amd64 kernel size by ~4k.
- Removes __SWAP_BROKEN cases.

Tested on x86, mips, acorn32 (thanks <mpumford>) and partly tested on
acorn26 (thanks to <bjh21>).

Discussed on <tech-kern>, reviewed by <ad>.

Fix a tyop in the previous commit

- Allow creating timeshard kthreads. To be used to fix the RAIDframe
parity rewrite issue.
- Create kthreads in the SCHED_RR class by default, not SCHED_FIFO.

branches: 1.24.8; 1.24.10;
Remove clause 3 and 4 from TNF licenses

branches: 1.23.2;
Merge proc::p_mutex and proc::p_smutex into a single adaptive mutex, since
we no longer need to guard against access from hardware interrupt handlers.

Additionally, if cloning a process with CLONE_SIGHAND, arrange to have the
child process share the parent's lock so that signal state may be kept in
sync. Partially addresses PR kern/37437.

branches: 1.22.2;
Move the LW_BOUND flag into the thread-private flag word. It can be tested
by other threads/CPUs but that is only done when the LWP is known to be in a
quiescent state (for example, on a run queue).

branches: 1.21.6;
Make schedstate_percpu::spc_lwplock an exernally allocated item. Remove
the hacks in sparc/cpu.c to reinitialize it. This should be in its own
cache line but that's another change.

Merge vmlocking2 to head.

branches: 1.19.2; 1.19.6;
Merge scheduler changes from the vmlocking branch. All discussed on
tech-kern:

- Invert priority space so that zero is the lowest priority. Rearrange
number and type of priority levels into bands. Add new bands like
'kernel real time'.
- Ignore the priority level passed to tsleep. Compute priority for
sleep dynamically.
- For SCHED_4BSD, make priority adjustment per-LWP, not per-process.

branches: 1.18.6; 1.18.8; 1.18.12; 1.18.14;
kthread_exit: l->l_name could be NULL.

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

branches: 1.16.6; 1.16.8;
Merge newlock2 to head.

branches: 1.15.18; 1.15.48;
Merge the nathanw_sa branch.

change uvm_uarea_alloc() to indicate whether the returned uarea is already
backed by physical pages (ie. because it reused a previously-freed one),
so that we can skip a bunch of useless work in that case.
this fixes the underlying problem behind PR 18543, and also speeds up fork()
quite a bit (eg. 7% on my pc, 1% on my ultra2) when we get a cache hit.

SIMPLEQ rototill:
- implement SIMPLEQ_REMOVE(head, elm, type, field). whilst it's O(n),
this mirrors the functionality of SLIST_REMOVE() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE()
- remove the unnecessary elm arg from SIMPLEQ_REMOVE_HEAD().
this mirrors the functionality of SLIST_REMOVE_HEAD() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE_HEAD()
- remove notes about SIMPLEQ not supporting arbitrary element removal
- use SIMPLEQ_FOREACH() instead of home-grown for loops
- use SIMPLEQ_EMPTY() appropriately
- use SIMPLEQ_*() instead of accessing sqh_first,sqh_last,sqe_next directly
- reorder manual page; be consistent about how the types are listed
- other minor cleanups

branches: 1.12.8;
add RCSIDs

branches: 1.11.2; 1.11.4; 1.11.8;
ANSI'ify.

Rather than starting init and creating kthreads by forking and then
doing a cpu_set_kpc(), just pass the entry point and argument all
the way down the fork path starting with fork1(). In order to
avoid special-casing the normal fork in every cpu_fork(), MI code
passes down child_return() and the child process pointer explicitly.

This fixes a race condition on multiprocessor systems; a CPU could
grab the newly created processes (which has been placed on a run queue)
before cpu_set_kpc() would be performed.

branches: 1.9.2;
__predict_false() fork1() failing.

branches: 1.8.2;
Make the kthread API a bit more friendly to loadable kernel modules.

Allow the caller to specify a stack for the child process. If NULL,
the child inherits the stack pointer from the parent (traditional
behavior). Like the signal stack, the stack area is secified as
a low address and a size; machine-dependent code accounts for stack
direction.

This is required for clone(2).

Allow an alternate exit signal (i.e. not SIGCHLD) to be delivered to the
parent, specified at fork time. Specify a new flag to wait4(2), WALTSIG,
to wait for processes which use an alternate exit signal.

This is required for clone(2).

Share everything with proc0.

minor nits -- replace vsprintf with vsnprinf

branches: 1.3.4;
#ifdef __STDC__ -> #if __STDC__

Implement a way to queue kernel threads for creation after init,
pagedaemon, reaper, etc. Caller provides a callback function and
argument which will be called to create the threads.

Move fork_kthread() to a new file, kern_kthread.c, and rename it to
kthread_create(). Implement kthread_exit() (causes a thrad to exit).
Set P_NOCLDWAIT on kernel threads, which will cause any of their children
to be reparented to init(8) (which is already prepared to wait out orphaned
processes).

kthread(9): Fix nested kthread_join.

No reason for one kthread_join to interfere with another, or to cause
non-cyclic dependencies to get stuck.

Uses struct lwp::l_private for this purpose, which for user threads
stores the tls pointer. I don't think anything in kthread(9) uses
l_private -- generally kernel threads will use lwp specificdata. But
maybe this should use a new member instead, or a union member with an
existing pointer for the purpose.

kthread(9): Attribute uarea to caller for kmsan.

New functions kthread_fpu_enter/exit.

The MI definitions don't do anything but maintain a flag, but MD code
can define kthread_fpu_enter/exit_md to actually enable/disable the
FPU. (These are almost pcu_load/discard on systems that use pcu(9),
except they apply to all PCUs.)

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2020/06/20/msg026524.html

The proposed kthread flag KTHREAD_FPU is not included because I
couldn't find any particular need for it that would not be covered by
just calling kthread_fpu_enter/exit in the kthread function.

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

branches: 1.44.2;
Minor scheduler cleanup:

- Adapt to cpu_need_resched() changes. Avoid lost & duplicate IPIs and ASTs.
sched_resched_cpu() and sched_resched_lwp() contain the logic for this.
- Changes for LSIDL to make the locking scheme match the intended design.
- Reduce lock contention and false sharing further.
- Numerous small bugfixes, including some corrections for SCHED_FIFO/RT.
- Use setrunnable() in more places, and merge cut & pasted code.

branches: 1.43.4;
KASSERT() that kthread_join()'s target is expecting to be joined.

As discussed on IRC.

- Propagate the signal mask from the ucontext_t to the newly created thread
as specified by _lwp_create(2)
- Reset the signal stack for threads created with _lwp_create(2)

branches: 1.41.2; 1.41.4;
Remove the unused and all-but-unusable kthread_destroy(). Also makes
it more obvious that the opposite of kthread_create() is kthread_exit().

Don't check if constant-sized KM_SLEEP allocations succeeded.

branches: 1.39.2; 1.39.16;
Add a __HAVE_CPU_UAREA_IDLELWP hook so that the MD code can allocate
special UAREAs for idle lwp's.

branches: 1.38.2; 1.38.10;
Avoid crashing when we do not explictly pass down a CPU when creating
a kthread(9) in the LSIDL (KTHREAD_IDLE) state.

Typo fix.

Rename slightly misleading KTHREAD_JOINABLE to KTHREAD_MUSTJOIN.

Retire varargs.h support. Move machine/stdarg.h logic into MI
sys/stdarg.h and expect compiler to provide proper builtins, defaulting
to the GCC interface. lint still has a special fallback.
Reduce abuse of _BSD_VA_LIST_ by defining __va_list by default and
derive va_list as required by standards.

Use a real panic as safe guard

Re-implement kthread_join(9), so that it actually works (hi haad@).

Hold the current lwp's mutex, instead of the lwp we want to wait for
in kthread_join(). This fixes panics with DIAGNOSTIC and possibly some
rare race conditons.

Add support for cpu-specific uarea allocation routines. Allows different
allocation for user and system lwps. MIPS will use this to map uareas of
system lwp used direct-mapped addresses (to reduce the overhead of
switching to kernel threads). ibm4xx could use to map uareas via direct
mapped addresses and avoid the problem of having the kernel stack not in
the TLB.

branches: 1.30.2; 1.30.4;
increment p_nrlwps in lwp_create rather than letting callers do so
as it's always decremented by lwp_exit. this fixes error recovery of
eg. aio_procinit.

Fix bogus KASSERT, typo and KNFize my previous patch.

Add support for kthread_join in our kernel thread implementation. This is used
by zfs but I think that it can be generaly usefull. Thread need to be created
with KTHREAD_JOINABLE flag and can be joined only once.
When joinable thread was created it will not e automatically reaped from system
and kthread_join must be called on it to reap it.

Ok by ad@.

branches: 1.27.2; 1.27.4;
Remove uarea swap-out functionality:

- Addresses the issue described in PR/38828.
- Some simplification in threading and sleepq subsystems.
- Eliminates pmap_collect() and, as a side note, allows pmap optimisations.
- Eliminates XS_CTL_DATA_ONSTACK in scsipi code.
- Avoids few scans on LWP list and thus potentially long holds of proc_lock.
- Cuts ~1.5k lines of code. Reduces amd64 kernel size by ~4k.
- Removes __SWAP_BROKEN cases.

Tested on x86, mips, acorn32 (thanks <mpumford>) and partly tested on
acorn26 (thanks to <bjh21>).

Discussed on <tech-kern>, reviewed by <ad>.

Fix a tyop in the previous commit

- Allow creating timeshard kthreads. To be used to fix the RAIDframe
parity rewrite issue.
- Create kthreads in the SCHED_RR class by default, not SCHED_FIFO.

branches: 1.24.8; 1.24.10;
Remove clause 3 and 4 from TNF licenses

branches: 1.23.2;
Merge proc::p_mutex and proc::p_smutex into a single adaptive mutex, since
we no longer need to guard against access from hardware interrupt handlers.

Additionally, if cloning a process with CLONE_SIGHAND, arrange to have the
child process share the parent's lock so that signal state may be kept in
sync. Partially addresses PR kern/37437.

branches: 1.22.2;
Move the LW_BOUND flag into the thread-private flag word. It can be tested
by other threads/CPUs but that is only done when the LWP is known to be in a
quiescent state (for example, on a run queue).

branches: 1.21.6;
Make schedstate_percpu::spc_lwplock an exernally allocated item. Remove
the hacks in sparc/cpu.c to reinitialize it. This should be in its own
cache line but that's another change.

Merge vmlocking2 to head.

branches: 1.19.2; 1.19.6;
Merge scheduler changes from the vmlocking branch. All discussed on
tech-kern:

- Invert priority space so that zero is the lowest priority. Rearrange
number and type of priority levels into bands. Add new bands like
'kernel real time'.
- Ignore the priority level passed to tsleep. Compute priority for
sleep dynamically.
- For SCHED_4BSD, make priority adjustment per-LWP, not per-process.

branches: 1.18.6; 1.18.8; 1.18.12; 1.18.14;
kthread_exit: l->l_name could be NULL.

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

branches: 1.16.6; 1.16.8;
Merge newlock2 to head.

branches: 1.15.18; 1.15.48;
Merge the nathanw_sa branch.

change uvm_uarea_alloc() to indicate whether the returned uarea is already
backed by physical pages (ie. because it reused a previously-freed one),
so that we can skip a bunch of useless work in that case.
this fixes the underlying problem behind PR 18543, and also speeds up fork()
quite a bit (eg. 7% on my pc, 1% on my ultra2) when we get a cache hit.

SIMPLEQ rototill:
- implement SIMPLEQ_REMOVE(head, elm, type, field). whilst it's O(n),
this mirrors the functionality of SLIST_REMOVE() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE()
- remove the unnecessary elm arg from SIMPLEQ_REMOVE_HEAD().
this mirrors the functionality of SLIST_REMOVE_HEAD() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE_HEAD()
- remove notes about SIMPLEQ not supporting arbitrary element removal
- use SIMPLEQ_FOREACH() instead of home-grown for loops
- use SIMPLEQ_EMPTY() appropriately
- use SIMPLEQ_*() instead of accessing sqh_first,sqh_last,sqe_next directly
- reorder manual page; be consistent about how the types are listed
- other minor cleanups

branches: 1.12.8;
add RCSIDs

branches: 1.11.2; 1.11.4; 1.11.8;
ANSI'ify.

Rather than starting init and creating kthreads by forking and then
doing a cpu_set_kpc(), just pass the entry point and argument all
the way down the fork path starting with fork1(). In order to
avoid special-casing the normal fork in every cpu_fork(), MI code
passes down child_return() and the child process pointer explicitly.

This fixes a race condition on multiprocessor systems; a CPU could
grab the newly created processes (which has been placed on a run queue)
before cpu_set_kpc() would be performed.

branches: 1.9.2;
__predict_false() fork1() failing.

branches: 1.8.2;
Make the kthread API a bit more friendly to loadable kernel modules.

Allow the caller to specify a stack for the child process. If NULL,
the child inherits the stack pointer from the parent (traditional
behavior). Like the signal stack, the stack area is secified as
a low address and a size; machine-dependent code accounts for stack
direction.

This is required for clone(2).

Allow an alternate exit signal (i.e. not SIGCHLD) to be delivered to the
parent, specified at fork time. Specify a new flag to wait4(2), WALTSIG,
to wait for processes which use an alternate exit signal.

This is required for clone(2).

Share everything with proc0.

minor nits -- replace vsprintf with vsnprinf

branches: 1.3.4;
#ifdef __STDC__ -> #if __STDC__

Implement a way to queue kernel threads for creation after init,
pagedaemon, reaper, etc. Caller provides a callback function and
argument which will be called to create the threads.

Move fork_kthread() to a new file, kern_kthread.c, and rename it to
kthread_create(). Implement kthread_exit() (causes a thrad to exit).
Set P_NOCLDWAIT on kernel threads, which will cause any of their children
to be reparented to init(8) (which is already prepared to wait out orphaned
processes).

kthread(9): Attribute uarea to caller for kmsan.

New functions kthread_fpu_enter/exit.

The MI definitions don't do anything but maintain a flag, but MD code
can define kthread_fpu_enter/exit_md to actually enable/disable the
FPU. (These are almost pcu_load/discard on systems that use pcu(9),
except they apply to all PCUs.)

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2020/06/20/msg026524.html

The proposed kthread flag KTHREAD_FPU is not included because I
couldn't find any particular need for it that would not be covered by
just calling kthread_fpu_enter/exit in the kthread function.

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

branches: 1.44.2;
Minor scheduler cleanup:

- Adapt to cpu_need_resched() changes. Avoid lost & duplicate IPIs and ASTs.
sched_resched_cpu() and sched_resched_lwp() contain the logic for this.
- Changes for LSIDL to make the locking scheme match the intended design.
- Reduce lock contention and false sharing further.
- Numerous small bugfixes, including some corrections for SCHED_FIFO/RT.
- Use setrunnable() in more places, and merge cut & pasted code.

branches: 1.43.4;
KASSERT() that kthread_join()'s target is expecting to be joined.

As discussed on IRC.

- Propagate the signal mask from the ucontext_t to the newly created thread
as specified by _lwp_create(2)
- Reset the signal stack for threads created with _lwp_create(2)

branches: 1.41.2; 1.41.4;
Remove the unused and all-but-unusable kthread_destroy(). Also makes
it more obvious that the opposite of kthread_create() is kthread_exit().

Don't check if constant-sized KM_SLEEP allocations succeeded.

branches: 1.39.2; 1.39.16;
Add a __HAVE_CPU_UAREA_IDLELWP hook so that the MD code can allocate
special UAREAs for idle lwp's.

branches: 1.38.2; 1.38.10;
Avoid crashing when we do not explictly pass down a CPU when creating
a kthread(9) in the LSIDL (KTHREAD_IDLE) state.

Typo fix.

Rename slightly misleading KTHREAD_JOINABLE to KTHREAD_MUSTJOIN.

Retire varargs.h support. Move machine/stdarg.h logic into MI
sys/stdarg.h and expect compiler to provide proper builtins, defaulting
to the GCC interface. lint still has a special fallback.
Reduce abuse of _BSD_VA_LIST_ by defining __va_list by default and
derive va_list as required by standards.

Use a real panic as safe guard

Re-implement kthread_join(9), so that it actually works (hi haad@).

Hold the current lwp's mutex, instead of the lwp we want to wait for
in kthread_join(). This fixes panics with DIAGNOSTIC and possibly some
rare race conditons.

Add support for cpu-specific uarea allocation routines. Allows different
allocation for user and system lwps. MIPS will use this to map uareas of
system lwp used direct-mapped addresses (to reduce the overhead of
switching to kernel threads). ibm4xx could use to map uareas via direct
mapped addresses and avoid the problem of having the kernel stack not in
the TLB.

branches: 1.30.2; 1.30.4;
increment p_nrlwps in lwp_create rather than letting callers do so
as it's always decremented by lwp_exit. this fixes error recovery of
eg. aio_procinit.

Fix bogus KASSERT, typo and KNFize my previous patch.

Add support for kthread_join in our kernel thread implementation. This is used
by zfs but I think that it can be generaly usefull. Thread need to be created
with KTHREAD_JOINABLE flag and can be joined only once.
When joinable thread was created it will not e automatically reaped from system
and kthread_join must be called on it to reap it.

Ok by ad@.

branches: 1.27.2; 1.27.4;
Remove uarea swap-out functionality:

- Addresses the issue described in PR/38828.
- Some simplification in threading and sleepq subsystems.
- Eliminates pmap_collect() and, as a side note, allows pmap optimisations.
- Eliminates XS_CTL_DATA_ONSTACK in scsipi code.
- Avoids few scans on LWP list and thus potentially long holds of proc_lock.
- Cuts ~1.5k lines of code. Reduces amd64 kernel size by ~4k.
- Removes __SWAP_BROKEN cases.

Tested on x86, mips, acorn32 (thanks <mpumford>) and partly tested on
acorn26 (thanks to <bjh21>).

Discussed on <tech-kern>, reviewed by <ad>.

Fix a tyop in the previous commit

- Allow creating timeshard kthreads. To be used to fix the RAIDframe
parity rewrite issue.
- Create kthreads in the SCHED_RR class by default, not SCHED_FIFO.

branches: 1.24.8; 1.24.10;
Remove clause 3 and 4 from TNF licenses

branches: 1.23.2;
Merge proc::p_mutex and proc::p_smutex into a single adaptive mutex, since
we no longer need to guard against access from hardware interrupt handlers.

Additionally, if cloning a process with CLONE_SIGHAND, arrange to have the
child process share the parent's lock so that signal state may be kept in
sync. Partially addresses PR kern/37437.

branches: 1.22.2;
Move the LW_BOUND flag into the thread-private flag word. It can be tested
by other threads/CPUs but that is only done when the LWP is known to be in a
quiescent state (for example, on a run queue).

branches: 1.21.6;
Make schedstate_percpu::spc_lwplock an exernally allocated item. Remove
the hacks in sparc/cpu.c to reinitialize it. This should be in its own
cache line but that's another change.

Merge vmlocking2 to head.

branches: 1.19.2; 1.19.6;
Merge scheduler changes from the vmlocking branch. All discussed on
tech-kern:

- Invert priority space so that zero is the lowest priority. Rearrange
number and type of priority levels into bands. Add new bands like
'kernel real time'.
- Ignore the priority level passed to tsleep. Compute priority for
sleep dynamically.
- For SCHED_4BSD, make priority adjustment per-LWP, not per-process.

branches: 1.18.6; 1.18.8; 1.18.12; 1.18.14;
kthread_exit: l->l_name could be NULL.

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

branches: 1.16.6; 1.16.8;
Merge newlock2 to head.

branches: 1.15.18; 1.15.48;
Merge the nathanw_sa branch.

change uvm_uarea_alloc() to indicate whether the returned uarea is already
backed by physical pages (ie. because it reused a previously-freed one),
so that we can skip a bunch of useless work in that case.
this fixes the underlying problem behind PR 18543, and also speeds up fork()
quite a bit (eg. 7% on my pc, 1% on my ultra2) when we get a cache hit.

SIMPLEQ rototill:
- implement SIMPLEQ_REMOVE(head, elm, type, field). whilst it's O(n),
this mirrors the functionality of SLIST_REMOVE() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE()
- remove the unnecessary elm arg from SIMPLEQ_REMOVE_HEAD().
this mirrors the functionality of SLIST_REMOVE_HEAD() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE_HEAD()
- remove notes about SIMPLEQ not supporting arbitrary element removal
- use SIMPLEQ_FOREACH() instead of home-grown for loops
- use SIMPLEQ_EMPTY() appropriately
- use SIMPLEQ_*() instead of accessing sqh_first,sqh_last,sqe_next directly
- reorder manual page; be consistent about how the types are listed
- other minor cleanups

branches: 1.12.8;
add RCSIDs

branches: 1.11.2; 1.11.4; 1.11.8;
ANSI'ify.

Rather than starting init and creating kthreads by forking and then
doing a cpu_set_kpc(), just pass the entry point and argument all
the way down the fork path starting with fork1(). In order to
avoid special-casing the normal fork in every cpu_fork(), MI code
passes down child_return() and the child process pointer explicitly.

This fixes a race condition on multiprocessor systems; a CPU could
grab the newly created processes (which has been placed on a run queue)
before cpu_set_kpc() would be performed.

branches: 1.9.2;
__predict_false() fork1() failing.

branches: 1.8.2;
Make the kthread API a bit more friendly to loadable kernel modules.

Allow the caller to specify a stack for the child process. If NULL,
the child inherits the stack pointer from the parent (traditional
behavior). Like the signal stack, the stack area is secified as
a low address and a size; machine-dependent code accounts for stack
direction.

This is required for clone(2).

Allow an alternate exit signal (i.e. not SIGCHLD) to be delivered to the
parent, specified at fork time. Specify a new flag to wait4(2), WALTSIG,
to wait for processes which use an alternate exit signal.

This is required for clone(2).

Share everything with proc0.

minor nits -- replace vsprintf with vsnprinf

branches: 1.3.4;
#ifdef __STDC__ -> #if __STDC__

Implement a way to queue kernel threads for creation after init,
pagedaemon, reaper, etc. Caller provides a callback function and
argument which will be called to create the threads.

Move fork_kthread() to a new file, kern_kthread.c, and rename it to
kthread_create(). Implement kthread_exit() (causes a thrad to exit).
Set P_NOCLDWAIT on kernel threads, which will cause any of their children
to be reparented to init(8) (which is already prepared to wait out orphaned
processes).

New functions kthread_fpu_enter/exit.

The MI definitions don't do anything but maintain a flag, but MD code
can define kthread_fpu_enter/exit_md to actually enable/disable the
FPU. (These are almost pcu_load/discard on systems that use pcu(9),
except they apply to all PCUs.)

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2020/06/20/msg026524.html

The proposed kthread flag KTHREAD_FPU is not included because I
couldn't find any particular need for it that would not be covered by
just calling kthread_fpu_enter/exit in the kthread function.

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

branches: 1.44.2;
Minor scheduler cleanup:

- Adapt to cpu_need_resched() changes. Avoid lost & duplicate IPIs and ASTs.
sched_resched_cpu() and sched_resched_lwp() contain the logic for this.
- Changes for LSIDL to make the locking scheme match the intended design.
- Reduce lock contention and false sharing further.
- Numerous small bugfixes, including some corrections for SCHED_FIFO/RT.
- Use setrunnable() in more places, and merge cut & pasted code.

branches: 1.43.4;
KASSERT() that kthread_join()'s target is expecting to be joined.

As discussed on IRC.

- Propagate the signal mask from the ucontext_t to the newly created thread
as specified by _lwp_create(2)
- Reset the signal stack for threads created with _lwp_create(2)

branches: 1.41.2; 1.41.4;
Remove the unused and all-but-unusable kthread_destroy(). Also makes
it more obvious that the opposite of kthread_create() is kthread_exit().

Don't check if constant-sized KM_SLEEP allocations succeeded.

branches: 1.39.2; 1.39.16;
Add a __HAVE_CPU_UAREA_IDLELWP hook so that the MD code can allocate
special UAREAs for idle lwp's.

branches: 1.38.2; 1.38.10;
Avoid crashing when we do not explictly pass down a CPU when creating
a kthread(9) in the LSIDL (KTHREAD_IDLE) state.

Typo fix.

Rename slightly misleading KTHREAD_JOINABLE to KTHREAD_MUSTJOIN.

Retire varargs.h support. Move machine/stdarg.h logic into MI
sys/stdarg.h and expect compiler to provide proper builtins, defaulting
to the GCC interface. lint still has a special fallback.
Reduce abuse of _BSD_VA_LIST_ by defining __va_list by default and
derive va_list as required by standards.

Use a real panic as safe guard

Re-implement kthread_join(9), so that it actually works (hi haad@).

Hold the current lwp's mutex, instead of the lwp we want to wait for
in kthread_join(). This fixes panics with DIAGNOSTIC and possibly some
rare race conditons.

Add support for cpu-specific uarea allocation routines. Allows different
allocation for user and system lwps. MIPS will use this to map uareas of
system lwp used direct-mapped addresses (to reduce the overhead of
switching to kernel threads). ibm4xx could use to map uareas via direct
mapped addresses and avoid the problem of having the kernel stack not in
the TLB.

branches: 1.30.2; 1.30.4;
increment p_nrlwps in lwp_create rather than letting callers do so
as it's always decremented by lwp_exit. this fixes error recovery of
eg. aio_procinit.

Fix bogus KASSERT, typo and KNFize my previous patch.

Add support for kthread_join in our kernel thread implementation. This is used
by zfs but I think that it can be generaly usefull. Thread need to be created
with KTHREAD_JOINABLE flag and can be joined only once.
When joinable thread was created it will not e automatically reaped from system
and kthread_join must be called on it to reap it.

Ok by ad@.

branches: 1.27.2; 1.27.4;
Remove uarea swap-out functionality:

- Addresses the issue described in PR/38828.
- Some simplification in threading and sleepq subsystems.
- Eliminates pmap_collect() and, as a side note, allows pmap optimisations.
- Eliminates XS_CTL_DATA_ONSTACK in scsipi code.
- Avoids few scans on LWP list and thus potentially long holds of proc_lock.
- Cuts ~1.5k lines of code. Reduces amd64 kernel size by ~4k.
- Removes __SWAP_BROKEN cases.

Tested on x86, mips, acorn32 (thanks <mpumford>) and partly tested on
acorn26 (thanks to <bjh21>).

Discussed on <tech-kern>, reviewed by <ad>.

Fix a tyop in the previous commit

- Allow creating timeshard kthreads. To be used to fix the RAIDframe
parity rewrite issue.
- Create kthreads in the SCHED_RR class by default, not SCHED_FIFO.

branches: 1.24.8; 1.24.10;
Remove clause 3 and 4 from TNF licenses

branches: 1.23.2;
Merge proc::p_mutex and proc::p_smutex into a single adaptive mutex, since
we no longer need to guard against access from hardware interrupt handlers.

Additionally, if cloning a process with CLONE_SIGHAND, arrange to have the
child process share the parent's lock so that signal state may be kept in
sync. Partially addresses PR kern/37437.

branches: 1.22.2;
Move the LW_BOUND flag into the thread-private flag word. It can be tested
by other threads/CPUs but that is only done when the LWP is known to be in a
quiescent state (for example, on a run queue).

branches: 1.21.6;
Make schedstate_percpu::spc_lwplock an exernally allocated item. Remove
the hacks in sparc/cpu.c to reinitialize it. This should be in its own
cache line but that's another change.

Merge vmlocking2 to head.

branches: 1.19.2; 1.19.6;
Merge scheduler changes from the vmlocking branch. All discussed on
tech-kern:

- Invert priority space so that zero is the lowest priority. Rearrange
number and type of priority levels into bands. Add new bands like
'kernel real time'.
- Ignore the priority level passed to tsleep. Compute priority for
sleep dynamically.
- For SCHED_4BSD, make priority adjustment per-LWP, not per-process.

branches: 1.18.6; 1.18.8; 1.18.12; 1.18.14;
kthread_exit: l->l_name could be NULL.

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

branches: 1.16.6; 1.16.8;
Merge newlock2 to head.

branches: 1.15.18; 1.15.48;
Merge the nathanw_sa branch.

change uvm_uarea_alloc() to indicate whether the returned uarea is already
backed by physical pages (ie. because it reused a previously-freed one),
so that we can skip a bunch of useless work in that case.
this fixes the underlying problem behind PR 18543, and also speeds up fork()
quite a bit (eg. 7% on my pc, 1% on my ultra2) when we get a cache hit.

SIMPLEQ rototill:
- implement SIMPLEQ_REMOVE(head, elm, type, field). whilst it's O(n),
this mirrors the functionality of SLIST_REMOVE() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE()
- remove the unnecessary elm arg from SIMPLEQ_REMOVE_HEAD().
this mirrors the functionality of SLIST_REMOVE_HEAD() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE_HEAD()
- remove notes about SIMPLEQ not supporting arbitrary element removal
- use SIMPLEQ_FOREACH() instead of home-grown for loops
- use SIMPLEQ_EMPTY() appropriately
- use SIMPLEQ_*() instead of accessing sqh_first,sqh_last,sqe_next directly
- reorder manual page; be consistent about how the types are listed
- other minor cleanups

branches: 1.12.8;
add RCSIDs

branches: 1.11.2; 1.11.4; 1.11.8;
ANSI'ify.

Rather than starting init and creating kthreads by forking and then
doing a cpu_set_kpc(), just pass the entry point and argument all
the way down the fork path starting with fork1(). In order to
avoid special-casing the normal fork in every cpu_fork(), MI code
passes down child_return() and the child process pointer explicitly.

This fixes a race condition on multiprocessor systems; a CPU could
grab the newly created processes (which has been placed on a run queue)
before cpu_set_kpc() would be performed.

branches: 1.9.2;
__predict_false() fork1() failing.

branches: 1.8.2;
Make the kthread API a bit more friendly to loadable kernel modules.

Allow the caller to specify a stack for the child process. If NULL,
the child inherits the stack pointer from the parent (traditional
behavior). Like the signal stack, the stack area is secified as
a low address and a size; machine-dependent code accounts for stack
direction.

This is required for clone(2).

Allow an alternate exit signal (i.e. not SIGCHLD) to be delivered to the
parent, specified at fork time. Specify a new flag to wait4(2), WALTSIG,
to wait for processes which use an alternate exit signal.

This is required for clone(2).

Share everything with proc0.

minor nits -- replace vsprintf with vsnprinf

branches: 1.3.4;
#ifdef __STDC__ -> #if __STDC__

Implement a way to queue kernel threads for creation after init,
pagedaemon, reaper, etc. Caller provides a callback function and
argument which will be called to create the threads.

Move fork_kthread() to a new file, kern_kthread.c, and rename it to
kthread_create(). Implement kthread_exit() (causes a thrad to exit).
Set P_NOCLDWAIT on kernel threads, which will cause any of their children
to be reparented to init(8) (which is already prepared to wait out orphaned
processes).

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

Minor scheduler cleanup:

- Adapt to cpu_need_resched() changes. Avoid lost & duplicate IPIs and ASTs.
sched_resched_cpu() and sched_resched_lwp() contain the logic for this.
- Changes for LSIDL to make the locking scheme match the intended design.
- Reduce lock contention and false sharing further.
- Numerous small bugfixes, including some corrections for SCHED_FIFO/RT.
- Use setrunnable() in more places, and merge cut & pasted code.

KASSERT() that kthread_join()'s target is expecting to be joined.

As discussed on IRC.

- Propagate the signal mask from the ucontext_t to the newly created thread
as specified by _lwp_create(2)
- Reset the signal stack for threads created with _lwp_create(2)

branches: 1.41.2; 1.41.4;
Remove the unused and all-but-unusable kthread_destroy(). Also makes
it more obvious that the opposite of kthread_create() is kthread_exit().

Don't check if constant-sized KM_SLEEP allocations succeeded.

branches: 1.39.2; 1.39.16;
Add a __HAVE_CPU_UAREA_IDLELWP hook so that the MD code can allocate
special UAREAs for idle lwp's.

branches: 1.38.2; 1.38.10;
Avoid crashing when we do not explictly pass down a CPU when creating
a kthread(9) in the LSIDL (KTHREAD_IDLE) state.

Typo fix.

Rename slightly misleading KTHREAD_JOINABLE to KTHREAD_MUSTJOIN.

Retire varargs.h support. Move machine/stdarg.h logic into MI
sys/stdarg.h and expect compiler to provide proper builtins, defaulting
to the GCC interface. lint still has a special fallback.
Reduce abuse of _BSD_VA_LIST_ by defining __va_list by default and
derive va_list as required by standards.

Use a real panic as safe guard

Re-implement kthread_join(9), so that it actually works (hi haad@).

Hold the current lwp's mutex, instead of the lwp we want to wait for
in kthread_join(). This fixes panics with DIAGNOSTIC and possibly some
rare race conditons.

Add support for cpu-specific uarea allocation routines. Allows different
allocation for user and system lwps. MIPS will use this to map uareas of
system lwp used direct-mapped addresses (to reduce the overhead of
switching to kernel threads). ibm4xx could use to map uareas via direct
mapped addresses and avoid the problem of having the kernel stack not in
the TLB.

branches: 1.30.2; 1.30.4;
increment p_nrlwps in lwp_create rather than letting callers do so
as it's always decremented by lwp_exit. this fixes error recovery of
eg. aio_procinit.

Fix bogus KASSERT, typo and KNFize my previous patch.

Add support for kthread_join in our kernel thread implementation. This is used
by zfs but I think that it can be generaly usefull. Thread need to be created
with KTHREAD_JOINABLE flag and can be joined only once.
When joinable thread was created it will not e automatically reaped from system
and kthread_join must be called on it to reap it.

Ok by ad@.

branches: 1.27.2; 1.27.4;
Remove uarea swap-out functionality:

- Addresses the issue described in PR/38828.
- Some simplification in threading and sleepq subsystems.
- Eliminates pmap_collect() and, as a side note, allows pmap optimisations.
- Eliminates XS_CTL_DATA_ONSTACK in scsipi code.
- Avoids few scans on LWP list and thus potentially long holds of proc_lock.
- Cuts ~1.5k lines of code. Reduces amd64 kernel size by ~4k.
- Removes __SWAP_BROKEN cases.

Tested on x86, mips, acorn32 (thanks <mpumford>) and partly tested on
acorn26 (thanks to <bjh21>).

Discussed on <tech-kern>, reviewed by <ad>.

Fix a tyop in the previous commit

- Allow creating timeshard kthreads. To be used to fix the RAIDframe
parity rewrite issue.
- Create kthreads in the SCHED_RR class by default, not SCHED_FIFO.

branches: 1.24.8; 1.24.10;
Remove clause 3 and 4 from TNF licenses

branches: 1.23.2;
Merge proc::p_mutex and proc::p_smutex into a single adaptive mutex, since
we no longer need to guard against access from hardware interrupt handlers.

Additionally, if cloning a process with CLONE_SIGHAND, arrange to have the
child process share the parent's lock so that signal state may be kept in
sync. Partially addresses PR kern/37437.

branches: 1.22.2;
Move the LW_BOUND flag into the thread-private flag word. It can be tested
by other threads/CPUs but that is only done when the LWP is known to be in a
quiescent state (for example, on a run queue).

branches: 1.21.6;
Make schedstate_percpu::spc_lwplock an exernally allocated item. Remove
the hacks in sparc/cpu.c to reinitialize it. This should be in its own
cache line but that's another change.

Merge vmlocking2 to head.

branches: 1.19.2; 1.19.6;
Merge scheduler changes from the vmlocking branch. All discussed on
tech-kern:

- Invert priority space so that zero is the lowest priority. Rearrange
number and type of priority levels into bands. Add new bands like
'kernel real time'.
- Ignore the priority level passed to tsleep. Compute priority for
sleep dynamically.
- For SCHED_4BSD, make priority adjustment per-LWP, not per-process.

branches: 1.18.6; 1.18.8; 1.18.12; 1.18.14;
kthread_exit: l->l_name could be NULL.

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

branches: 1.16.6; 1.16.8;
Merge newlock2 to head.

branches: 1.15.18; 1.15.48;
Merge the nathanw_sa branch.

change uvm_uarea_alloc() to indicate whether the returned uarea is already
backed by physical pages (ie. because it reused a previously-freed one),
so that we can skip a bunch of useless work in that case.
this fixes the underlying problem behind PR 18543, and also speeds up fork()
quite a bit (eg. 7% on my pc, 1% on my ultra2) when we get a cache hit.

SIMPLEQ rototill:
- implement SIMPLEQ_REMOVE(head, elm, type, field). whilst it's O(n),
this mirrors the functionality of SLIST_REMOVE() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE()
- remove the unnecessary elm arg from SIMPLEQ_REMOVE_HEAD().
this mirrors the functionality of SLIST_REMOVE_HEAD() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE_HEAD()
- remove notes about SIMPLEQ not supporting arbitrary element removal
- use SIMPLEQ_FOREACH() instead of home-grown for loops
- use SIMPLEQ_EMPTY() appropriately
- use SIMPLEQ_*() instead of accessing sqh_first,sqh_last,sqe_next directly
- reorder manual page; be consistent about how the types are listed
- other minor cleanups

branches: 1.12.8;
add RCSIDs

branches: 1.11.2; 1.11.4; 1.11.8;
ANSI'ify.

Rather than starting init and creating kthreads by forking and then
doing a cpu_set_kpc(), just pass the entry point and argument all
the way down the fork path starting with fork1(). In order to
avoid special-casing the normal fork in every cpu_fork(), MI code
passes down child_return() and the child process pointer explicitly.

This fixes a race condition on multiprocessor systems; a CPU could
grab the newly created processes (which has been placed on a run queue)
before cpu_set_kpc() would be performed.

branches: 1.9.2;
__predict_false() fork1() failing.

branches: 1.8.2;
Make the kthread API a bit more friendly to loadable kernel modules.

Allow the caller to specify a stack for the child process. If NULL,
the child inherits the stack pointer from the parent (traditional
behavior). Like the signal stack, the stack area is secified as
a low address and a size; machine-dependent code accounts for stack
direction.

This is required for clone(2).

Allow an alternate exit signal (i.e. not SIGCHLD) to be delivered to the
parent, specified at fork time. Specify a new flag to wait4(2), WALTSIG,
to wait for processes which use an alternate exit signal.

This is required for clone(2).

Share everything with proc0.

minor nits -- replace vsprintf with vsnprinf

branches: 1.3.4;
#ifdef __STDC__ -> #if __STDC__

Implement a way to queue kernel threads for creation after init,
pagedaemon, reaper, etc. Caller provides a callback function and
argument which will be called to create the threads.

Move fork_kthread() to a new file, kern_kthread.c, and rename it to
kthread_create(). Implement kthread_exit() (causes a thrad to exit).
Set P_NOCLDWAIT on kernel threads, which will cause any of their children
to be reparented to init(8) (which is already prepared to wait out orphaned
processes).

Minor scheduler cleanup:

- Adapt to cpu_need_resched() changes. Avoid lost & duplicate IPIs and ASTs.
sched_resched_cpu() and sched_resched_lwp() contain the logic for this.
- Changes for LSIDL to make the locking scheme match the intended design.
- Reduce lock contention and false sharing further.
- Numerous small bugfixes, including some corrections for SCHED_FIFO/RT.
- Use setrunnable() in more places, and merge cut & pasted code.

KASSERT() that kthread_join()'s target is expecting to be joined.

As discussed on IRC.

- Propagate the signal mask from the ucontext_t to the newly created thread
as specified by _lwp_create(2)
- Reset the signal stack for threads created with _lwp_create(2)

branches: 1.41.2; 1.41.4;
Remove the unused and all-but-unusable kthread_destroy(). Also makes
it more obvious that the opposite of kthread_create() is kthread_exit().

Don't check if constant-sized KM_SLEEP allocations succeeded.

branches: 1.39.2; 1.39.16;
Add a __HAVE_CPU_UAREA_IDLELWP hook so that the MD code can allocate
special UAREAs for idle lwp's.

branches: 1.38.2; 1.38.10;
Avoid crashing when we do not explictly pass down a CPU when creating
a kthread(9) in the LSIDL (KTHREAD_IDLE) state.

Typo fix.

Rename slightly misleading KTHREAD_JOINABLE to KTHREAD_MUSTJOIN.

Retire varargs.h support. Move machine/stdarg.h logic into MI
sys/stdarg.h and expect compiler to provide proper builtins, defaulting
to the GCC interface. lint still has a special fallback.
Reduce abuse of _BSD_VA_LIST_ by defining __va_list by default and
derive va_list as required by standards.

Use a real panic as safe guard

Re-implement kthread_join(9), so that it actually works (hi haad@).

Hold the current lwp's mutex, instead of the lwp we want to wait for
in kthread_join(). This fixes panics with DIAGNOSTIC and possibly some
rare race conditons.

Add support for cpu-specific uarea allocation routines. Allows different
allocation for user and system lwps. MIPS will use this to map uareas of
system lwp used direct-mapped addresses (to reduce the overhead of
switching to kernel threads). ibm4xx could use to map uareas via direct
mapped addresses and avoid the problem of having the kernel stack not in
the TLB.

branches: 1.30.2; 1.30.4;
increment p_nrlwps in lwp_create rather than letting callers do so
as it's always decremented by lwp_exit. this fixes error recovery of
eg. aio_procinit.

Fix bogus KASSERT, typo and KNFize my previous patch.

Add support for kthread_join in our kernel thread implementation. This is used
by zfs but I think that it can be generaly usefull. Thread need to be created
with KTHREAD_JOINABLE flag and can be joined only once.
When joinable thread was created it will not e automatically reaped from system
and kthread_join must be called on it to reap it.

Ok by ad@.

branches: 1.27.2; 1.27.4;
Remove uarea swap-out functionality:

- Addresses the issue described in PR/38828.
- Some simplification in threading and sleepq subsystems.
- Eliminates pmap_collect() and, as a side note, allows pmap optimisations.
- Eliminates XS_CTL_DATA_ONSTACK in scsipi code.
- Avoids few scans on LWP list and thus potentially long holds of proc_lock.
- Cuts ~1.5k lines of code. Reduces amd64 kernel size by ~4k.
- Removes __SWAP_BROKEN cases.

Tested on x86, mips, acorn32 (thanks <mpumford>) and partly tested on
acorn26 (thanks to <bjh21>).

Discussed on <tech-kern>, reviewed by <ad>.

Fix a tyop in the previous commit

- Allow creating timeshard kthreads. To be used to fix the RAIDframe
parity rewrite issue.
- Create kthreads in the SCHED_RR class by default, not SCHED_FIFO.

branches: 1.24.8; 1.24.10;
Remove clause 3 and 4 from TNF licenses

branches: 1.23.2;
Merge proc::p_mutex and proc::p_smutex into a single adaptive mutex, since
we no longer need to guard against access from hardware interrupt handlers.

Additionally, if cloning a process with CLONE_SIGHAND, arrange to have the
child process share the parent's lock so that signal state may be kept in
sync. Partially addresses PR kern/37437.

branches: 1.22.2;
Move the LW_BOUND flag into the thread-private flag word. It can be tested
by other threads/CPUs but that is only done when the LWP is known to be in a
quiescent state (for example, on a run queue).

branches: 1.21.6;
Make schedstate_percpu::spc_lwplock an exernally allocated item. Remove
the hacks in sparc/cpu.c to reinitialize it. This should be in its own
cache line but that's another change.

Merge vmlocking2 to head.

branches: 1.19.2; 1.19.6;
Merge scheduler changes from the vmlocking branch. All discussed on
tech-kern:

- Invert priority space so that zero is the lowest priority. Rearrange
number and type of priority levels into bands. Add new bands like
'kernel real time'.
- Ignore the priority level passed to tsleep. Compute priority for
sleep dynamically.
- For SCHED_4BSD, make priority adjustment per-LWP, not per-process.

branches: 1.18.6; 1.18.8; 1.18.12; 1.18.14;
kthread_exit: l->l_name could be NULL.

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

branches: 1.16.6; 1.16.8;
Merge newlock2 to head.

branches: 1.15.18; 1.15.48;
Merge the nathanw_sa branch.

change uvm_uarea_alloc() to indicate whether the returned uarea is already
backed by physical pages (ie. because it reused a previously-freed one),
so that we can skip a bunch of useless work in that case.
this fixes the underlying problem behind PR 18543, and also speeds up fork()
quite a bit (eg. 7% on my pc, 1% on my ultra2) when we get a cache hit.

SIMPLEQ rototill:
- implement SIMPLEQ_REMOVE(head, elm, type, field). whilst it's O(n),
this mirrors the functionality of SLIST_REMOVE() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE()
- remove the unnecessary elm arg from SIMPLEQ_REMOVE_HEAD().
this mirrors the functionality of SLIST_REMOVE_HEAD() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE_HEAD()
- remove notes about SIMPLEQ not supporting arbitrary element removal
- use SIMPLEQ_FOREACH() instead of home-grown for loops
- use SIMPLEQ_EMPTY() appropriately
- use SIMPLEQ_*() instead of accessing sqh_first,sqh_last,sqe_next directly
- reorder manual page; be consistent about how the types are listed
- other minor cleanups

branches: 1.12.8;
add RCSIDs

branches: 1.11.2; 1.11.4; 1.11.8;
ANSI'ify.

Rather than starting init and creating kthreads by forking and then
doing a cpu_set_kpc(), just pass the entry point and argument all
the way down the fork path starting with fork1(). In order to
avoid special-casing the normal fork in every cpu_fork(), MI code
passes down child_return() and the child process pointer explicitly.

This fixes a race condition on multiprocessor systems; a CPU could
grab the newly created processes (which has been placed on a run queue)
before cpu_set_kpc() would be performed.

branches: 1.9.2;
__predict_false() fork1() failing.

branches: 1.8.2;
Make the kthread API a bit more friendly to loadable kernel modules.

Allow the caller to specify a stack for the child process. If NULL,
the child inherits the stack pointer from the parent (traditional
behavior). Like the signal stack, the stack area is secified as
a low address and a size; machine-dependent code accounts for stack
direction.

This is required for clone(2).

Allow an alternate exit signal (i.e. not SIGCHLD) to be delivered to the
parent, specified at fork time. Specify a new flag to wait4(2), WALTSIG,
to wait for processes which use an alternate exit signal.

This is required for clone(2).

Share everything with proc0.

minor nits -- replace vsprintf with vsnprinf

branches: 1.3.4;
#ifdef __STDC__ -> #if __STDC__

Implement a way to queue kernel threads for creation after init,
pagedaemon, reaper, etc. Caller provides a callback function and
argument which will be called to create the threads.

Move fork_kthread() to a new file, kern_kthread.c, and rename it to
kthread_create(). Implement kthread_exit() (causes a thrad to exit).
Set P_NOCLDWAIT on kernel threads, which will cause any of their children
to be reparented to init(8) (which is already prepared to wait out orphaned
processes).

KASSERT() that kthread_join()'s target is expecting to be joined.

As discussed on IRC.

- Propagate the signal mask from the ucontext_t to the newly created thread
as specified by _lwp_create(2)
- Reset the signal stack for threads created with _lwp_create(2)

branches: 1.41.2; 1.41.4;
Remove the unused and all-but-unusable kthread_destroy(). Also makes
it more obvious that the opposite of kthread_create() is kthread_exit().

Don't check if constant-sized KM_SLEEP allocations succeeded.

branches: 1.39.2; 1.39.16;
Add a __HAVE_CPU_UAREA_IDLELWP hook so that the MD code can allocate
special UAREAs for idle lwp's.

branches: 1.38.2; 1.38.10;
Avoid crashing when we do not explictly pass down a CPU when creating
a kthread(9) in the LSIDL (KTHREAD_IDLE) state.

Typo fix.

Rename slightly misleading KTHREAD_JOINABLE to KTHREAD_MUSTJOIN.

Retire varargs.h support. Move machine/stdarg.h logic into MI
sys/stdarg.h and expect compiler to provide proper builtins, defaulting
to the GCC interface. lint still has a special fallback.
Reduce abuse of _BSD_VA_LIST_ by defining __va_list by default and
derive va_list as required by standards.

Use a real panic as safe guard

Re-implement kthread_join(9), so that it actually works (hi haad@).

Hold the current lwp's mutex, instead of the lwp we want to wait for
in kthread_join(). This fixes panics with DIAGNOSTIC and possibly some
rare race conditons.

Add support for cpu-specific uarea allocation routines. Allows different
allocation for user and system lwps. MIPS will use this to map uareas of
system lwp used direct-mapped addresses (to reduce the overhead of
switching to kernel threads). ibm4xx could use to map uareas via direct
mapped addresses and avoid the problem of having the kernel stack not in
the TLB.

branches: 1.30.2; 1.30.4;
increment p_nrlwps in lwp_create rather than letting callers do so
as it's always decremented by lwp_exit. this fixes error recovery of
eg. aio_procinit.

Fix bogus KASSERT, typo and KNFize my previous patch.

Add support for kthread_join in our kernel thread implementation. This is used
by zfs but I think that it can be generaly usefull. Thread need to be created
with KTHREAD_JOINABLE flag and can be joined only once.
When joinable thread was created it will not e automatically reaped from system
and kthread_join must be called on it to reap it.

Ok by ad@.

branches: 1.27.2; 1.27.4;
Remove uarea swap-out functionality:

- Addresses the issue described in PR/38828.
- Some simplification in threading and sleepq subsystems.
- Eliminates pmap_collect() and, as a side note, allows pmap optimisations.
- Eliminates XS_CTL_DATA_ONSTACK in scsipi code.
- Avoids few scans on LWP list and thus potentially long holds of proc_lock.
- Cuts ~1.5k lines of code. Reduces amd64 kernel size by ~4k.
- Removes __SWAP_BROKEN cases.

Tested on x86, mips, acorn32 (thanks <mpumford>) and partly tested on
acorn26 (thanks to <bjh21>).

Discussed on <tech-kern>, reviewed by <ad>.

Fix a tyop in the previous commit

- Allow creating timeshard kthreads. To be used to fix the RAIDframe
parity rewrite issue.
- Create kthreads in the SCHED_RR class by default, not SCHED_FIFO.

branches: 1.24.8; 1.24.10;
Remove clause 3 and 4 from TNF licenses

branches: 1.23.2;
Merge proc::p_mutex and proc::p_smutex into a single adaptive mutex, since
we no longer need to guard against access from hardware interrupt handlers.

Additionally, if cloning a process with CLONE_SIGHAND, arrange to have the
child process share the parent's lock so that signal state may be kept in
sync. Partially addresses PR kern/37437.

branches: 1.22.2;
Move the LW_BOUND flag into the thread-private flag word. It can be tested
by other threads/CPUs but that is only done when the LWP is known to be in a
quiescent state (for example, on a run queue).

branches: 1.21.6;
Make schedstate_percpu::spc_lwplock an exernally allocated item. Remove
the hacks in sparc/cpu.c to reinitialize it. This should be in its own
cache line but that's another change.

Merge vmlocking2 to head.

branches: 1.19.2; 1.19.6;
Merge scheduler changes from the vmlocking branch. All discussed on
tech-kern:

- Invert priority space so that zero is the lowest priority. Rearrange
number and type of priority levels into bands. Add new bands like
'kernel real time'.
- Ignore the priority level passed to tsleep. Compute priority for
sleep dynamically.
- For SCHED_4BSD, make priority adjustment per-LWP, not per-process.

branches: 1.18.6; 1.18.8; 1.18.12; 1.18.14;
kthread_exit: l->l_name could be NULL.

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

branches: 1.16.6; 1.16.8;
Merge newlock2 to head.

branches: 1.15.18; 1.15.48;
Merge the nathanw_sa branch.

change uvm_uarea_alloc() to indicate whether the returned uarea is already
backed by physical pages (ie. because it reused a previously-freed one),
so that we can skip a bunch of useless work in that case.
this fixes the underlying problem behind PR 18543, and also speeds up fork()
quite a bit (eg. 7% on my pc, 1% on my ultra2) when we get a cache hit.

SIMPLEQ rototill:
- implement SIMPLEQ_REMOVE(head, elm, type, field). whilst it's O(n),
this mirrors the functionality of SLIST_REMOVE() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE()
- remove the unnecessary elm arg from SIMPLEQ_REMOVE_HEAD().
this mirrors the functionality of SLIST_REMOVE_HEAD() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE_HEAD()
- remove notes about SIMPLEQ not supporting arbitrary element removal
- use SIMPLEQ_FOREACH() instead of home-grown for loops
- use SIMPLEQ_EMPTY() appropriately
- use SIMPLEQ_*() instead of accessing sqh_first,sqh_last,sqe_next directly
- reorder manual page; be consistent about how the types are listed
- other minor cleanups

branches: 1.12.8;
add RCSIDs

branches: 1.11.2; 1.11.4; 1.11.8;
ANSI'ify.

Rather than starting init and creating kthreads by forking and then
doing a cpu_set_kpc(), just pass the entry point and argument all
the way down the fork path starting with fork1(). In order to
avoid special-casing the normal fork in every cpu_fork(), MI code
passes down child_return() and the child process pointer explicitly.

This fixes a race condition on multiprocessor systems; a CPU could
grab the newly created processes (which has been placed on a run queue)
before cpu_set_kpc() would be performed.

branches: 1.9.2;
__predict_false() fork1() failing.

branches: 1.8.2;
Make the kthread API a bit more friendly to loadable kernel modules.

Allow the caller to specify a stack for the child process. If NULL,
the child inherits the stack pointer from the parent (traditional
behavior). Like the signal stack, the stack area is secified as
a low address and a size; machine-dependent code accounts for stack
direction.

This is required for clone(2).

Allow an alternate exit signal (i.e. not SIGCHLD) to be delivered to the
parent, specified at fork time. Specify a new flag to wait4(2), WALTSIG,
to wait for processes which use an alternate exit signal.

This is required for clone(2).

Share everything with proc0.

minor nits -- replace vsprintf with vsnprinf

branches: 1.3.4;
#ifdef __STDC__ -> #if __STDC__

Implement a way to queue kernel threads for creation after init,
pagedaemon, reaper, etc. Caller provides a callback function and
argument which will be called to create the threads.

Move fork_kthread() to a new file, kern_kthread.c, and rename it to
kthread_create(). Implement kthread_exit() (causes a thrad to exit).
Set P_NOCLDWAIT on kernel threads, which will cause any of their children
to be reparented to init(8) (which is already prepared to wait out orphaned
processes).

- Propagate the signal mask from the ucontext_t to the newly created thread
as specified by _lwp_create(2)
- Reset the signal stack for threads created with _lwp_create(2)

branches: 1.41.2; 1.41.4;
Remove the unused and all-but-unusable kthread_destroy(). Also makes
it more obvious that the opposite of kthread_create() is kthread_exit().

Don't check if constant-sized KM_SLEEP allocations succeeded.

branches: 1.39.16;
Add a __HAVE_CPU_UAREA_IDLELWP hook so that the MD code can allocate
special UAREAs for idle lwp's.

branches: 1.38.2; 1.38.10;
Avoid crashing when we do not explictly pass down a CPU when creating
a kthread(9) in the LSIDL (KTHREAD_IDLE) state.

Typo fix.

Rename slightly misleading KTHREAD_JOINABLE to KTHREAD_MUSTJOIN.

Retire varargs.h support. Move machine/stdarg.h logic into MI
sys/stdarg.h and expect compiler to provide proper builtins, defaulting
to the GCC interface. lint still has a special fallback.
Reduce abuse of _BSD_VA_LIST_ by defining __va_list by default and
derive va_list as required by standards.

Use a real panic as safe guard

Re-implement kthread_join(9), so that it actually works (hi haad@).

Hold the current lwp's mutex, instead of the lwp we want to wait for
in kthread_join(). This fixes panics with DIAGNOSTIC and possibly some
rare race conditons.

Add support for cpu-specific uarea allocation routines. Allows different
allocation for user and system lwps. MIPS will use this to map uareas of
system lwp used direct-mapped addresses (to reduce the overhead of
switching to kernel threads). ibm4xx could use to map uareas via direct
mapped addresses and avoid the problem of having the kernel stack not in
the TLB.

branches: 1.30.2; 1.30.4;
increment p_nrlwps in lwp_create rather than letting callers do so
as it's always decremented by lwp_exit. this fixes error recovery of
eg. aio_procinit.

Fix bogus KASSERT, typo and KNFize my previous patch.

Add support for kthread_join in our kernel thread implementation. This is used
by zfs but I think that it can be generaly usefull. Thread need to be created
with KTHREAD_JOINABLE flag and can be joined only once.
When joinable thread was created it will not e automatically reaped from system
and kthread_join must be called on it to reap it.

Ok by ad@.

branches: 1.27.2; 1.27.4;
Remove uarea swap-out functionality:

- Addresses the issue described in PR/38828.
- Some simplification in threading and sleepq subsystems.
- Eliminates pmap_collect() and, as a side note, allows pmap optimisations.
- Eliminates XS_CTL_DATA_ONSTACK in scsipi code.
- Avoids few scans on LWP list and thus potentially long holds of proc_lock.
- Cuts ~1.5k lines of code. Reduces amd64 kernel size by ~4k.
- Removes __SWAP_BROKEN cases.

Tested on x86, mips, acorn32 (thanks <mpumford>) and partly tested on
acorn26 (thanks to <bjh21>).

Discussed on <tech-kern>, reviewed by <ad>.

Fix a tyop in the previous commit

- Allow creating timeshard kthreads. To be used to fix the RAIDframe
parity rewrite issue.
- Create kthreads in the SCHED_RR class by default, not SCHED_FIFO.

branches: 1.24.8; 1.24.10;
Remove clause 3 and 4 from TNF licenses

branches: 1.23.2;
Merge proc::p_mutex and proc::p_smutex into a single adaptive mutex, since
we no longer need to guard against access from hardware interrupt handlers.

Additionally, if cloning a process with CLONE_SIGHAND, arrange to have the
child process share the parent's lock so that signal state may be kept in
sync. Partially addresses PR kern/37437.

branches: 1.22.2;
Move the LW_BOUND flag into the thread-private flag word. It can be tested
by other threads/CPUs but that is only done when the LWP is known to be in a
quiescent state (for example, on a run queue).

branches: 1.21.6;
Make schedstate_percpu::spc_lwplock an exernally allocated item. Remove
the hacks in sparc/cpu.c to reinitialize it. This should be in its own
cache line but that's another change.

Merge vmlocking2 to head.

branches: 1.19.2; 1.19.6;
Merge scheduler changes from the vmlocking branch. All discussed on
tech-kern:

- Invert priority space so that zero is the lowest priority. Rearrange
number and type of priority levels into bands. Add new bands like
'kernel real time'.
- Ignore the priority level passed to tsleep. Compute priority for
sleep dynamically.
- For SCHED_4BSD, make priority adjustment per-LWP, not per-process.

branches: 1.18.6; 1.18.8; 1.18.12; 1.18.14;
kthread_exit: l->l_name could be NULL.

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

branches: 1.16.6; 1.16.8;
Merge newlock2 to head.

branches: 1.15.18; 1.15.48;
Merge the nathanw_sa branch.

change uvm_uarea_alloc() to indicate whether the returned uarea is already
backed by physical pages (ie. because it reused a previously-freed one),
so that we can skip a bunch of useless work in that case.
this fixes the underlying problem behind PR 18543, and also speeds up fork()
quite a bit (eg. 7% on my pc, 1% on my ultra2) when we get a cache hit.

SIMPLEQ rototill:
- implement SIMPLEQ_REMOVE(head, elm, type, field). whilst it's O(n),
this mirrors the functionality of SLIST_REMOVE() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE()
- remove the unnecessary elm arg from SIMPLEQ_REMOVE_HEAD().
this mirrors the functionality of SLIST_REMOVE_HEAD() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE_HEAD()
- remove notes about SIMPLEQ not supporting arbitrary element removal
- use SIMPLEQ_FOREACH() instead of home-grown for loops
- use SIMPLEQ_EMPTY() appropriately
- use SIMPLEQ_*() instead of accessing sqh_first,sqh_last,sqe_next directly
- reorder manual page; be consistent about how the types are listed
- other minor cleanups

branches: 1.12.8;
add RCSIDs

branches: 1.11.2; 1.11.4; 1.11.8;
ANSI'ify.

Rather than starting init and creating kthreads by forking and then
doing a cpu_set_kpc(), just pass the entry point and argument all
the way down the fork path starting with fork1(). In order to
avoid special-casing the normal fork in every cpu_fork(), MI code
passes down child_return() and the child process pointer explicitly.

This fixes a race condition on multiprocessor systems; a CPU could
grab the newly created processes (which has been placed on a run queue)
before cpu_set_kpc() would be performed.

branches: 1.9.2;
__predict_false() fork1() failing.

branches: 1.8.2;
Make the kthread API a bit more friendly to loadable kernel modules.

Allow the caller to specify a stack for the child process. If NULL,
the child inherits the stack pointer from the parent (traditional
behavior). Like the signal stack, the stack area is secified as
a low address and a size; machine-dependent code accounts for stack
direction.

This is required for clone(2).

Allow an alternate exit signal (i.e. not SIGCHLD) to be delivered to the
parent, specified at fork time. Specify a new flag to wait4(2), WALTSIG,
to wait for processes which use an alternate exit signal.

This is required for clone(2).

Share everything with proc0.

minor nits -- replace vsprintf with vsnprinf

branches: 1.3.4;
#ifdef __STDC__ -> #if __STDC__

Implement a way to queue kernel threads for creation after init,
pagedaemon, reaper, etc. Caller provides a callback function and
argument which will be called to create the threads.

Move fork_kthread() to a new file, kern_kthread.c, and rename it to
kthread_create(). Implement kthread_exit() (causes a thrad to exit).
Set P_NOCLDWAIT on kernel threads, which will cause any of their children
to be reparented to init(8) (which is already prepared to wait out orphaned
processes).