rump: Sprinkle sys/syncobj.h here too.

Arrange to update cached LWP credentials in userret() rather than during
syscall/trap entry, eliminating a test+branch on every syscall/trap.

This wasn't possible in the 3.99.x timeframe when l->l_cred came about
because there wasn't a reliable/timely way to force an ONPROC LWP running on
a remote CPU into the kernel (which is just about the only new thing in
this scheme).

Eliminate l->l_biglocks. Originally I think it had a use but these days a
local variable will do.

Eliminate l->l_ncsw and l->l_nivcsw. From memory think they were added
before we had per-LWP struct rusage; the same is now tracked there.

rump: Nix membar_enter/exit after/before RUMPUSER_LWP_CREATE/DESTROY.

These make no sense and are obviously not needed for any of the
rumpuser implementations in-tree. (For single-threaded fiber and
pth_dummy, no barriers needed; for pth, RUMPUSER_LWP_CREATE/DESTROY
use pthread_mutex so don't need any barriers.)

It _might_ make sense to do membar_release/acquire around just
RUMPUSER_LWP_DESTROY, perhaps if there's some reference-counting
business involved. But if a rumpuser implementation really needs
that it can do it itself.

rumpkern/lwproc.c: Nix trailing whitespace.

rump: don't touch p_nlwps without holding p_lock

There was a race condition on p_nlwps. Heavy thread switching could
cause a kernel panic like:
panic: kernel diagnostic assertion "LIST_EMPTY(&p->p_lwps)" failed:
file "(hidden)/src/lib/librump/../../sys/rump/librump/rumpkern/lwproc.c", line 177

Fix a lock order reversal that caused hangs.

Move proc_lock into the data segment. It was dynamically allocated because
at the time we had mutex_obj_alloc() but not __cacheline_aligned.

- Replace pid_table_lock with a lockless lookup covered by pserialize, with
the "writer" side being pid_table expansion. The basic idea is that when
doing an LWP lookup there is usually already a lock held (p->p_lock), or a
spin mutex that needs to be taken (l->l_mutex), and either can be used to
get the found LWP stable and confidently determine that all is correct.

- For user processes LSLARVAL implies the same thing as LSIDL ("not visible
by ID"), and lookup by ID in proc0 doesn't really happen. In-tree the new
state should be understood by top(1), the tty subsystem and so on, and
would attract the attention of 3rd party kernel grovellers in time, so
remove it and just rely on LSIDL.

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

lwp0.l_lid needs to be 0.

Adapt to LWP ID allocation changes.

- Only increment nprocs when we're creating a new process, not just
when allocating a PID.
- Per above, proc_free_pid() no longer decrements nprocs. It's now done
in proc_free() right after proc_free_pid().
- Ensure nprocs is accessed using atomics everywhere.

branches: 1.44.4;
- Move the LW_RUNNING flag back into l_pflag: updating l_flag without lock
in softint_dispatch() is risky. May help with the "softint screwup"
panic.

- Correct the memory barriers around zombies switching into oblivion.

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

branches: 1.42.4;
Implement an aggressive psref leak detector

It is yet another psref leak detector that enables to tell where a leak occurs
while a simpler version that is already committed just tells an occurrence of a
leak.

Investigating of psref leaks is hard because once a leak occurs a percpu list of
psref that tracks references can be corrupted. A reference to a tracking object
is memorized in the list via an intermediate object (struct psref) that is
normally allocated on a stack of a thread. Thus, the intermediate object can be
overwritten on a leak resulting in corruption of the list.

The tracker makes a shadow entry to an intermediate object and stores some hints
into it (currently it's a caller address of psref_acquire). We can detect a
leak by checking the entries on certain points where any references should be
released such as the return point of syscalls and the end of each softint
handler.

The feature is expensive and enabled only if the kernel is built with
PSREF_DEBUG.

Proposed on tech-kern

Rumpkernel has its own thread deallocation. Add missing fstrans_lwp_dtor()
to lwproc_freelwp().

PR bin/50350: rump/rumpkern/t_sp/stress_{long,short} fail on Core 2 Quad

branches: 1.40.18;
Add lwp_find() - verbatim copy from the hard kernel.

Split p_xstat (composite wait(2) status code, or signal number depending
on context) into:
1. p_xexit: exit code
2. p_xsig: signal number
3. p_sflag & WCOREFLAG bit to indicated that the process core-dumped.

Fix the documentation of the flag bits in <sys/proc.h>

Allocate struct cpu_info dynamically. Saves quite a lot of BSS in the
common case and reduces rump kernel memory requirements by 10% or more
in really tiny deployments.

Put the kernelside rump kernel headers into <rump-sys> instead of
sprinkling them around the faction directories. Avoids having
to add a CPPFLAGS (or several) to pretty much every component
Makefile.

Leave compat headers around in the old locations.

The commit changes some autogenerated files, but I'll fix the
generators shortly and regen.

put lwp/proc stuff into the same source module (emul.c -> lwproc.c)

Give remote clients struct pmap storage. Although the pmap is unused,
that way we can sure that the pmap macro framework doesn't access all
the wrong places.

Don't share file descriptors between proc0 and local clients.

Use a different vmspace for rump kernel proc0 and local clients.
While the rump kernel and local clients are by definition in the same
host vmspace, there are subtle differences in how in-kernel code works
in case accessing the kernel vmspace or a user process vmspace.

Problem discovered by riastradh's "read(fd, NULL, 1)" test.

Account for lwps so that rump_sys_setuid() doesn't hit a KASSERT when
it tries to reaccount a switching procs lwps.

from Mato Lucina

branches: 1.31.4;
Init function pointers to nullop() so that callers don't need a dance

Reparent children of a dying lwproc. Fixes wpa_supplicant -B (and
most likely a bunch of other things).

implement lwp_create() and a few related interfaces.
main raison d'impalerment: the aio driver

branches: 1.28.2;
remove unnecesary verbosity: s/RUMP_CURLWP_MODEL/RUMP_CURLWP/

Allow compile-time optimizations to curlwp. This can have a pretty
staggering impact on performance. When running sendto() in a loop,
the improvement is 200k more calls per second with an inlined __thread
curlwp as opposed to the default. In other words, it shaves off hundreds
of CPU cycles per call (~20%). Even just eliminating the x86_curlwp()
call in favor of an inline gives an improvement of 60k calls per second.

Translate return values for emulations, e.g. Linux. For ports without
__HAVE_MINIMAL_EMUL, we simply look up the values from p->p_emul->e_errno.
For ports which cannot afford to keep an extra pointer per emul structure
around, we hope there is __HAVE_SYSCALL_INTERN support and thread the
errno values through p_emuldata. Notably, we cannot alter the syscall
method like most ports do with syscall_intern, since they do it via
p_mdproc, so MI code is not possible there.

stop ktrace at process exit

Enable holding implicit threads over explicit scheduling points (i.e.
a kernel driver calling rump_lwproc_switch()). Also, correctly handle
curcpu()->ci_curlwp and biglock there.

branches: 1.23.2;
Add RUMPUSER_LWP_CLEAR instead of overloading RUMPUSER_LWP_SET.
This simplifies some alternative hypervisor implementations.

Inform the rump kernel hypervisor of valid thread contexts so
that the implementation can allocate and release storage for them
in an optimal fashion, if necessary.

One more mutex which is used as a spin mutex and therefore needs
to be tagged as one, lest the mighty assert smites thee.

Support multiple syscall emuls.

Rump kernels with only local clients have it easy, since they have to
support at most two syscall vectors (some calls such as mount/unmount
can be made only through the "native" NetBSD vector). Remote clients
are a whole different tin of green beans. This change addresses local
clients only.

kill some -Wunused-but-set-variable warnings

branches: 1.18.4; 1.18.14;
More lim_free() fallout

Nuke all threads belonging to a process calling exec before allowing
the exec handshake to return.

In addition to being The Right Thing To Do, fixes some nasty
conditions for CLOEXEC fd's (or at least does so in theory, I
couldn't create any problems although I tried).

Fill in a functional struct lwp (especially l_mutex) before exposing
it on p_lwps.

branches: 1.15.2;
Explicitly zerofill some fields which are not within the boundaries
of p_start/endzero for whatever reasons.

Obviously, this fixes remote physio when running with MALLOC_OPTIONS J.
Otherwise PS_WEXIT is set in struct proc, proc_vmspace_getref() fails,
and copyout_proc() with it.

Reported by njoly

copy parent's p_comm when forking (if it exists)

Don't NULL out l_cpu, l_stat is the new rumpkernel way too.

Some lwp-walkers expect the correct value for l_stat, so use a flag
in l_flag instead of l_stat for the purpose of flagging lwps in a
dying proc.

fill in a default name for p_comm

branches: 1.10.2; 1.10.4;
Introduce RUMP_LOCALPROC_P() macro and use it.

When the client and the rump kernel are the same host process, all
threads blocking in the kernel automatically exit when the process
exists. However, for the sysproxy case this does not hold.
Typically it's ~harmless, but e.g. in the case of socket binding
following by poll it gets annoying.

Introduce sysproxy procexit, which wakes up all threads blocking
on a condition when a process's communication socket is closed.
The code is a little different from the regular kernel simply
because in a rump kernel l_mutex is not available at all times
(this is because scheduling happens on every kernel entry and exit,
and that path must be kept lockless for any reasonable performance).
Instead, use gating which makes sure all threads are either out of
the cv code or suspended in a well-known state. Then, wake up the
threads and tell them to get the hell out of our galaxy.

Support LOCKDEBUG. To use it, compile sys/rump with RUMP_LOCKDEBUG=yes.

requested by martin (sparc64 gdb cannot reliably produce a stack trace)

There is a use case where preserving the parent's fd table is
relevant, so to accommodate that change rump_lwproc_newproc() to
rump_lwproc_rfork(). The new interface has the rfork() fd table
semantics. The equivalent of rump_lwproc_newproc() is
rump_lwproc_rfork(RUMP_RFCFDG).

Support physio for remote processes.
==> add support for remote vmspace vmapbuf/vunmapbuf
==> add proper support for copyin/out_vmspace
==> add support for remote vmspace uvm_io
==> add support for non-curproc rumpuser_sp_copyin/out
==> store remote context in vm_map->pmap instead of
pthread_specificdata

In short, makes read/write of most (all?) block devices work from
a remote rump client via rump syscalls.

cleanup some old garbage

Add a new signal model RUMP_SIGMODEL_RECORD which records all
signals which are posted to a process.

Attach implicit threads to initproc instead of proc0. This way
applications which alter, by purpose or by accident, the uid in an
implicit thread are don't affect kernel threads.

from discussion with njoly

branches: 1.2.2; 1.2.4;
proper panic for trying to release implicit lwp

Implement rump_lwproc: the new lwp/proc management routines for
rump. These move the management of the pid/lwpid space from the
application into the kernel, make code more robust, and make it
possible to attach multiple lwp's to non-proc0 processes.

Arrange to update cached LWP credentials in userret() rather than during
syscall/trap entry, eliminating a test+branch on every syscall/trap.

This wasn't possible in the 3.99.x timeframe when l->l_cred came about
because there wasn't a reliable/timely way to force an ONPROC LWP running on
a remote CPU into the kernel (which is just about the only new thing in
this scheme).

Eliminate l->l_biglocks. Originally I think it had a use but these days a
local variable will do.

Eliminate l->l_ncsw and l->l_nivcsw. From memory think they were added
before we had per-LWP struct rusage; the same is now tracked there.

rump: Nix membar_enter/exit after/before RUMPUSER_LWP_CREATE/DESTROY.

These make no sense and are obviously not needed for any of the
rumpuser implementations in-tree. (For single-threaded fiber and
pth_dummy, no barriers needed; for pth, RUMPUSER_LWP_CREATE/DESTROY
use pthread_mutex so don't need any barriers.)

It _might_ make sense to do membar_release/acquire around just
RUMPUSER_LWP_DESTROY, perhaps if there's some reference-counting
business involved. But if a rumpuser implementation really needs
that it can do it itself.

rumpkern/lwproc.c: Nix trailing whitespace.

rump: don't touch p_nlwps without holding p_lock

There was a race condition on p_nlwps. Heavy thread switching could
cause a kernel panic like:
panic: kernel diagnostic assertion "LIST_EMPTY(&p->p_lwps)" failed:
file "(hidden)/src/lib/librump/../../sys/rump/librump/rumpkern/lwproc.c", line 177

Fix a lock order reversal that caused hangs.

Move proc_lock into the data segment. It was dynamically allocated because
at the time we had mutex_obj_alloc() but not __cacheline_aligned.

- Replace pid_table_lock with a lockless lookup covered by pserialize, with
the "writer" side being pid_table expansion. The basic idea is that when
doing an LWP lookup there is usually already a lock held (p->p_lock), or a
spin mutex that needs to be taken (l->l_mutex), and either can be used to
get the found LWP stable and confidently determine that all is correct.

- For user processes LSLARVAL implies the same thing as LSIDL ("not visible
by ID"), and lookup by ID in proc0 doesn't really happen. In-tree the new
state should be understood by top(1), the tty subsystem and so on, and
would attract the attention of 3rd party kernel grovellers in time, so
remove it and just rely on LSIDL.

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

lwp0.l_lid needs to be 0.

Adapt to LWP ID allocation changes.

- Only increment nprocs when we're creating a new process, not just
when allocating a PID.
- Per above, proc_free_pid() no longer decrements nprocs. It's now done
in proc_free() right after proc_free_pid().
- Ensure nprocs is accessed using atomics everywhere.

branches: 1.44.4;
- Move the LW_RUNNING flag back into l_pflag: updating l_flag without lock
in softint_dispatch() is risky. May help with the "softint screwup"
panic.

- Correct the memory barriers around zombies switching into oblivion.

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

branches: 1.42.4;
Implement an aggressive psref leak detector

It is yet another psref leak detector that enables to tell where a leak occurs
while a simpler version that is already committed just tells an occurrence of a
leak.

Investigating of psref leaks is hard because once a leak occurs a percpu list of
psref that tracks references can be corrupted. A reference to a tracking object
is memorized in the list via an intermediate object (struct psref) that is
normally allocated on a stack of a thread. Thus, the intermediate object can be
overwritten on a leak resulting in corruption of the list.

The tracker makes a shadow entry to an intermediate object and stores some hints
into it (currently it's a caller address of psref_acquire). We can detect a
leak by checking the entries on certain points where any references should be
released such as the return point of syscalls and the end of each softint
handler.

The feature is expensive and enabled only if the kernel is built with
PSREF_DEBUG.

Proposed on tech-kern

Rumpkernel has its own thread deallocation. Add missing fstrans_lwp_dtor()
to lwproc_freelwp().

PR bin/50350: rump/rumpkern/t_sp/stress_{long,short} fail on Core 2 Quad

branches: 1.40.18;
Add lwp_find() - verbatim copy from the hard kernel.

Split p_xstat (composite wait(2) status code, or signal number depending
on context) into:
1. p_xexit: exit code
2. p_xsig: signal number
3. p_sflag & WCOREFLAG bit to indicated that the process core-dumped.

Fix the documentation of the flag bits in <sys/proc.h>

Allocate struct cpu_info dynamically. Saves quite a lot of BSS in the
common case and reduces rump kernel memory requirements by 10% or more
in really tiny deployments.

Put the kernelside rump kernel headers into <rump-sys> instead of
sprinkling them around the faction directories. Avoids having
to add a CPPFLAGS (or several) to pretty much every component
Makefile.

Leave compat headers around in the old locations.

The commit changes some autogenerated files, but I'll fix the
generators shortly and regen.

put lwp/proc stuff into the same source module (emul.c -> lwproc.c)

Give remote clients struct pmap storage. Although the pmap is unused,
that way we can sure that the pmap macro framework doesn't access all
the wrong places.

Don't share file descriptors between proc0 and local clients.

Use a different vmspace for rump kernel proc0 and local clients.
While the rump kernel and local clients are by definition in the same
host vmspace, there are subtle differences in how in-kernel code works
in case accessing the kernel vmspace or a user process vmspace.

Problem discovered by riastradh's "read(fd, NULL, 1)" test.

Account for lwps so that rump_sys_setuid() doesn't hit a KASSERT when
it tries to reaccount a switching procs lwps.

from Mato Lucina

branches: 1.31.4;
Init function pointers to nullop() so that callers don't need a dance

Reparent children of a dying lwproc. Fixes wpa_supplicant -B (and
most likely a bunch of other things).

implement lwp_create() and a few related interfaces.
main raison d'impalerment: the aio driver

branches: 1.28.2;
remove unnecesary verbosity: s/RUMP_CURLWP_MODEL/RUMP_CURLWP/

Allow compile-time optimizations to curlwp. This can have a pretty
staggering impact on performance. When running sendto() in a loop,
the improvement is 200k more calls per second with an inlined __thread
curlwp as opposed to the default. In other words, it shaves off hundreds
of CPU cycles per call (~20%). Even just eliminating the x86_curlwp()
call in favor of an inline gives an improvement of 60k calls per second.

Translate return values for emulations, e.g. Linux. For ports without
__HAVE_MINIMAL_EMUL, we simply look up the values from p->p_emul->e_errno.
For ports which cannot afford to keep an extra pointer per emul structure
around, we hope there is __HAVE_SYSCALL_INTERN support and thread the
errno values through p_emuldata. Notably, we cannot alter the syscall
method like most ports do with syscall_intern, since they do it via
p_mdproc, so MI code is not possible there.

stop ktrace at process exit

Enable holding implicit threads over explicit scheduling points (i.e.
a kernel driver calling rump_lwproc_switch()). Also, correctly handle
curcpu()->ci_curlwp and biglock there.

branches: 1.23.2;
Add RUMPUSER_LWP_CLEAR instead of overloading RUMPUSER_LWP_SET.
This simplifies some alternative hypervisor implementations.

Inform the rump kernel hypervisor of valid thread contexts so
that the implementation can allocate and release storage for them
in an optimal fashion, if necessary.

One more mutex which is used as a spin mutex and therefore needs
to be tagged as one, lest the mighty assert smites thee.

Support multiple syscall emuls.

Rump kernels with only local clients have it easy, since they have to
support at most two syscall vectors (some calls such as mount/unmount
can be made only through the "native" NetBSD vector). Remote clients
are a whole different tin of green beans. This change addresses local
clients only.

kill some -Wunused-but-set-variable warnings

branches: 1.18.4; 1.18.14;
More lim_free() fallout

Nuke all threads belonging to a process calling exec before allowing
the exec handshake to return.

In addition to being The Right Thing To Do, fixes some nasty
conditions for CLOEXEC fd's (or at least does so in theory, I
couldn't create any problems although I tried).

Fill in a functional struct lwp (especially l_mutex) before exposing
it on p_lwps.

branches: 1.15.2;
Explicitly zerofill some fields which are not within the boundaries
of p_start/endzero for whatever reasons.

Obviously, this fixes remote physio when running with MALLOC_OPTIONS J.
Otherwise PS_WEXIT is set in struct proc, proc_vmspace_getref() fails,
and copyout_proc() with it.

Reported by njoly

copy parent's p_comm when forking (if it exists)

Don't NULL out l_cpu, l_stat is the new rumpkernel way too.

Some lwp-walkers expect the correct value for l_stat, so use a flag
in l_flag instead of l_stat for the purpose of flagging lwps in a
dying proc.

fill in a default name for p_comm

branches: 1.10.2; 1.10.4;
Introduce RUMP_LOCALPROC_P() macro and use it.

When the client and the rump kernel are the same host process, all
threads blocking in the kernel automatically exit when the process
exists. However, for the sysproxy case this does not hold.
Typically it's ~harmless, but e.g. in the case of socket binding
following by poll it gets annoying.

Introduce sysproxy procexit, which wakes up all threads blocking
on a condition when a process's communication socket is closed.
The code is a little different from the regular kernel simply
because in a rump kernel l_mutex is not available at all times
(this is because scheduling happens on every kernel entry and exit,
and that path must be kept lockless for any reasonable performance).
Instead, use gating which makes sure all threads are either out of
the cv code or suspended in a well-known state. Then, wake up the
threads and tell them to get the hell out of our galaxy.

Support LOCKDEBUG. To use it, compile sys/rump with RUMP_LOCKDEBUG=yes.

requested by martin (sparc64 gdb cannot reliably produce a stack trace)

There is a use case where preserving the parent's fd table is
relevant, so to accommodate that change rump_lwproc_newproc() to
rump_lwproc_rfork(). The new interface has the rfork() fd table
semantics. The equivalent of rump_lwproc_newproc() is
rump_lwproc_rfork(RUMP_RFCFDG).

Support physio for remote processes.
==> add support for remote vmspace vmapbuf/vunmapbuf
==> add proper support for copyin/out_vmspace
==> add support for remote vmspace uvm_io
==> add support for non-curproc rumpuser_sp_copyin/out
==> store remote context in vm_map->pmap instead of
pthread_specificdata

In short, makes read/write of most (all?) block devices work from
a remote rump client via rump syscalls.

cleanup some old garbage

Add a new signal model RUMP_SIGMODEL_RECORD which records all
signals which are posted to a process.

Attach implicit threads to initproc instead of proc0. This way
applications which alter, by purpose or by accident, the uid in an
implicit thread are don't affect kernel threads.

from discussion with njoly

branches: 1.2.2; 1.2.4;
proper panic for trying to release implicit lwp

Implement rump_lwproc: the new lwp/proc management routines for
rump. These move the management of the pid/lwpid space from the
application into the kernel, make code more robust, and make it
possible to attach multiple lwp's to non-proc0 processes.

rump: Nix membar_enter/exit after/before RUMPUSER_LWP_CREATE/DESTROY.

These make no sense and are obviously not needed for any of the
rumpuser implementations in-tree. (For single-threaded fiber and
pth_dummy, no barriers needed; for pth, RUMPUSER_LWP_CREATE/DESTROY
use pthread_mutex so don't need any barriers.)

It _might_ make sense to do membar_release/acquire around just
RUMPUSER_LWP_DESTROY, perhaps if there's some reference-counting
business involved. But if a rumpuser implementation really needs
that it can do it itself.

rumpkern/lwproc.c: Nix trailing whitespace.

rump: don't touch p_nlwps without holding p_lock

There was a race condition on p_nlwps. Heavy thread switching could
cause a kernel panic like:
panic: kernel diagnostic assertion "LIST_EMPTY(&p->p_lwps)" failed:
file "(hidden)/src/lib/librump/../../sys/rump/librump/rumpkern/lwproc.c", line 177

Fix a lock order reversal that caused hangs.

Move proc_lock into the data segment. It was dynamically allocated because
at the time we had mutex_obj_alloc() but not __cacheline_aligned.

- Replace pid_table_lock with a lockless lookup covered by pserialize, with
the "writer" side being pid_table expansion. The basic idea is that when
doing an LWP lookup there is usually already a lock held (p->p_lock), or a
spin mutex that needs to be taken (l->l_mutex), and either can be used to
get the found LWP stable and confidently determine that all is correct.

- For user processes LSLARVAL implies the same thing as LSIDL ("not visible
by ID"), and lookup by ID in proc0 doesn't really happen. In-tree the new
state should be understood by top(1), the tty subsystem and so on, and
would attract the attention of 3rd party kernel grovellers in time, so
remove it and just rely on LSIDL.

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

lwp0.l_lid needs to be 0.

Adapt to LWP ID allocation changes.

- Only increment nprocs when we're creating a new process, not just
when allocating a PID.
- Per above, proc_free_pid() no longer decrements nprocs. It's now done
in proc_free() right after proc_free_pid().
- Ensure nprocs is accessed using atomics everywhere.

branches: 1.44.4;
- Move the LW_RUNNING flag back into l_pflag: updating l_flag without lock
in softint_dispatch() is risky. May help with the "softint screwup"
panic.

- Correct the memory barriers around zombies switching into oblivion.

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

branches: 1.42.4;
Implement an aggressive psref leak detector

It is yet another psref leak detector that enables to tell where a leak occurs
while a simpler version that is already committed just tells an occurrence of a
leak.

Investigating of psref leaks is hard because once a leak occurs a percpu list of
psref that tracks references can be corrupted. A reference to a tracking object
is memorized in the list via an intermediate object (struct psref) that is
normally allocated on a stack of a thread. Thus, the intermediate object can be
overwritten on a leak resulting in corruption of the list.

The tracker makes a shadow entry to an intermediate object and stores some hints
into it (currently it's a caller address of psref_acquire). We can detect a
leak by checking the entries on certain points where any references should be
released such as the return point of syscalls and the end of each softint
handler.

The feature is expensive and enabled only if the kernel is built with
PSREF_DEBUG.

Proposed on tech-kern

Rumpkernel has its own thread deallocation. Add missing fstrans_lwp_dtor()
to lwproc_freelwp().

PR bin/50350: rump/rumpkern/t_sp/stress_{long,short} fail on Core 2 Quad

branches: 1.40.18;
Add lwp_find() - verbatim copy from the hard kernel.

Split p_xstat (composite wait(2) status code, or signal number depending
on context) into:
1. p_xexit: exit code
2. p_xsig: signal number
3. p_sflag & WCOREFLAG bit to indicated that the process core-dumped.

Fix the documentation of the flag bits in <sys/proc.h>

Allocate struct cpu_info dynamically. Saves quite a lot of BSS in the
common case and reduces rump kernel memory requirements by 10% or more
in really tiny deployments.

Put the kernelside rump kernel headers into <rump-sys> instead of
sprinkling them around the faction directories. Avoids having
to add a CPPFLAGS (or several) to pretty much every component
Makefile.

Leave compat headers around in the old locations.

The commit changes some autogenerated files, but I'll fix the
generators shortly and regen.

put lwp/proc stuff into the same source module (emul.c -> lwproc.c)

Give remote clients struct pmap storage. Although the pmap is unused,
that way we can sure that the pmap macro framework doesn't access all
the wrong places.

Don't share file descriptors between proc0 and local clients.

Use a different vmspace for rump kernel proc0 and local clients.
While the rump kernel and local clients are by definition in the same
host vmspace, there are subtle differences in how in-kernel code works
in case accessing the kernel vmspace or a user process vmspace.

Problem discovered by riastradh's "read(fd, NULL, 1)" test.

Account for lwps so that rump_sys_setuid() doesn't hit a KASSERT when
it tries to reaccount a switching procs lwps.

from Mato Lucina

branches: 1.31.4;
Init function pointers to nullop() so that callers don't need a dance

Reparent children of a dying lwproc. Fixes wpa_supplicant -B (and
most likely a bunch of other things).

implement lwp_create() and a few related interfaces.
main raison d'impalerment: the aio driver

branches: 1.28.2;
remove unnecesary verbosity: s/RUMP_CURLWP_MODEL/RUMP_CURLWP/

Allow compile-time optimizations to curlwp. This can have a pretty
staggering impact on performance. When running sendto() in a loop,
the improvement is 200k more calls per second with an inlined __thread
curlwp as opposed to the default. In other words, it shaves off hundreds
of CPU cycles per call (~20%). Even just eliminating the x86_curlwp()
call in favor of an inline gives an improvement of 60k calls per second.

Translate return values for emulations, e.g. Linux. For ports without
__HAVE_MINIMAL_EMUL, we simply look up the values from p->p_emul->e_errno.
For ports which cannot afford to keep an extra pointer per emul structure
around, we hope there is __HAVE_SYSCALL_INTERN support and thread the
errno values through p_emuldata. Notably, we cannot alter the syscall
method like most ports do with syscall_intern, since they do it via
p_mdproc, so MI code is not possible there.

stop ktrace at process exit

Enable holding implicit threads over explicit scheduling points (i.e.
a kernel driver calling rump_lwproc_switch()). Also, correctly handle
curcpu()->ci_curlwp and biglock there.

branches: 1.23.2;
Add RUMPUSER_LWP_CLEAR instead of overloading RUMPUSER_LWP_SET.
This simplifies some alternative hypervisor implementations.

Inform the rump kernel hypervisor of valid thread contexts so
that the implementation can allocate and release storage for them
in an optimal fashion, if necessary.

One more mutex which is used as a spin mutex and therefore needs
to be tagged as one, lest the mighty assert smites thee.

Support multiple syscall emuls.

Rump kernels with only local clients have it easy, since they have to
support at most two syscall vectors (some calls such as mount/unmount
can be made only through the "native" NetBSD vector). Remote clients
are a whole different tin of green beans. This change addresses local
clients only.

kill some -Wunused-but-set-variable warnings

branches: 1.18.4; 1.18.14;
More lim_free() fallout

Nuke all threads belonging to a process calling exec before allowing
the exec handshake to return.

In addition to being The Right Thing To Do, fixes some nasty
conditions for CLOEXEC fd's (or at least does so in theory, I
couldn't create any problems although I tried).

Fill in a functional struct lwp (especially l_mutex) before exposing
it on p_lwps.

branches: 1.15.2;
Explicitly zerofill some fields which are not within the boundaries
of p_start/endzero for whatever reasons.

Obviously, this fixes remote physio when running with MALLOC_OPTIONS J.
Otherwise PS_WEXIT is set in struct proc, proc_vmspace_getref() fails,
and copyout_proc() with it.

Reported by njoly

copy parent's p_comm when forking (if it exists)

Don't NULL out l_cpu, l_stat is the new rumpkernel way too.

Some lwp-walkers expect the correct value for l_stat, so use a flag
in l_flag instead of l_stat for the purpose of flagging lwps in a
dying proc.

fill in a default name for p_comm

branches: 1.10.2; 1.10.4;
Introduce RUMP_LOCALPROC_P() macro and use it.

When the client and the rump kernel are the same host process, all
threads blocking in the kernel automatically exit when the process
exists. However, for the sysproxy case this does not hold.
Typically it's ~harmless, but e.g. in the case of socket binding
following by poll it gets annoying.

Introduce sysproxy procexit, which wakes up all threads blocking
on a condition when a process's communication socket is closed.
The code is a little different from the regular kernel simply
because in a rump kernel l_mutex is not available at all times
(this is because scheduling happens on every kernel entry and exit,
and that path must be kept lockless for any reasonable performance).
Instead, use gating which makes sure all threads are either out of
the cv code or suspended in a well-known state. Then, wake up the
threads and tell them to get the hell out of our galaxy.

Support LOCKDEBUG. To use it, compile sys/rump with RUMP_LOCKDEBUG=yes.

requested by martin (sparc64 gdb cannot reliably produce a stack trace)

There is a use case where preserving the parent's fd table is
relevant, so to accommodate that change rump_lwproc_newproc() to
rump_lwproc_rfork(). The new interface has the rfork() fd table
semantics. The equivalent of rump_lwproc_newproc() is
rump_lwproc_rfork(RUMP_RFCFDG).

Support physio for remote processes.
==> add support for remote vmspace vmapbuf/vunmapbuf
==> add proper support for copyin/out_vmspace
==> add support for remote vmspace uvm_io
==> add support for non-curproc rumpuser_sp_copyin/out
==> store remote context in vm_map->pmap instead of
pthread_specificdata

In short, makes read/write of most (all?) block devices work from
a remote rump client via rump syscalls.

cleanup some old garbage

Add a new signal model RUMP_SIGMODEL_RECORD which records all
signals which are posted to a process.

Attach implicit threads to initproc instead of proc0. This way
applications which alter, by purpose or by accident, the uid in an
implicit thread are don't affect kernel threads.

from discussion with njoly

branches: 1.2.2; 1.2.4;
proper panic for trying to release implicit lwp

Implement rump_lwproc: the new lwp/proc management routines for
rump. These move the management of the pid/lwpid space from the
application into the kernel, make code more robust, and make it
possible to attach multiple lwp's to non-proc0 processes.

rump: don't touch p_nlwps without holding p_lock

There was a race condition on p_nlwps. Heavy thread switching could
cause a kernel panic like:
panic: kernel diagnostic assertion "LIST_EMPTY(&p->p_lwps)" failed:
file "(hidden)/src/lib/librump/../../sys/rump/librump/rumpkern/lwproc.c", line 177

Fix a lock order reversal that caused hangs.

Move proc_lock into the data segment. It was dynamically allocated because
at the time we had mutex_obj_alloc() but not __cacheline_aligned.

- Replace pid_table_lock with a lockless lookup covered by pserialize, with
the "writer" side being pid_table expansion. The basic idea is that when
doing an LWP lookup there is usually already a lock held (p->p_lock), or a
spin mutex that needs to be taken (l->l_mutex), and either can be used to
get the found LWP stable and confidently determine that all is correct.

- For user processes LSLARVAL implies the same thing as LSIDL ("not visible
by ID"), and lookup by ID in proc0 doesn't really happen. In-tree the new
state should be understood by top(1), the tty subsystem and so on, and
would attract the attention of 3rd party kernel grovellers in time, so
remove it and just rely on LSIDL.

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

lwp0.l_lid needs to be 0.

Adapt to LWP ID allocation changes.

- Only increment nprocs when we're creating a new process, not just
when allocating a PID.
- Per above, proc_free_pid() no longer decrements nprocs. It's now done
in proc_free() right after proc_free_pid().
- Ensure nprocs is accessed using atomics everywhere.

branches: 1.44.4;
- Move the LW_RUNNING flag back into l_pflag: updating l_flag without lock
in softint_dispatch() is risky. May help with the "softint screwup"
panic.

- Correct the memory barriers around zombies switching into oblivion.

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

branches: 1.42.4;
Implement an aggressive psref leak detector

It is yet another psref leak detector that enables to tell where a leak occurs
while a simpler version that is already committed just tells an occurrence of a
leak.

Investigating of psref leaks is hard because once a leak occurs a percpu list of
psref that tracks references can be corrupted. A reference to a tracking object
is memorized in the list via an intermediate object (struct psref) that is
normally allocated on a stack of a thread. Thus, the intermediate object can be
overwritten on a leak resulting in corruption of the list.

The tracker makes a shadow entry to an intermediate object and stores some hints
into it (currently it's a caller address of psref_acquire). We can detect a
leak by checking the entries on certain points where any references should be
released such as the return point of syscalls and the end of each softint
handler.

The feature is expensive and enabled only if the kernel is built with
PSREF_DEBUG.

Proposed on tech-kern

Rumpkernel has its own thread deallocation. Add missing fstrans_lwp_dtor()
to lwproc_freelwp().

PR bin/50350: rump/rumpkern/t_sp/stress_{long,short} fail on Core 2 Quad

branches: 1.40.18;
Add lwp_find() - verbatim copy from the hard kernel.

Split p_xstat (composite wait(2) status code, or signal number depending
on context) into:
1. p_xexit: exit code
2. p_xsig: signal number
3. p_sflag & WCOREFLAG bit to indicated that the process core-dumped.

Fix the documentation of the flag bits in <sys/proc.h>

Allocate struct cpu_info dynamically. Saves quite a lot of BSS in the
common case and reduces rump kernel memory requirements by 10% or more
in really tiny deployments.

Put the kernelside rump kernel headers into <rump-sys> instead of
sprinkling them around the faction directories. Avoids having
to add a CPPFLAGS (or several) to pretty much every component
Makefile.

Leave compat headers around in the old locations.

The commit changes some autogenerated files, but I'll fix the
generators shortly and regen.

put lwp/proc stuff into the same source module (emul.c -> lwproc.c)

Give remote clients struct pmap storage. Although the pmap is unused,
that way we can sure that the pmap macro framework doesn't access all
the wrong places.

Don't share file descriptors between proc0 and local clients.

Use a different vmspace for rump kernel proc0 and local clients.
While the rump kernel and local clients are by definition in the same
host vmspace, there are subtle differences in how in-kernel code works
in case accessing the kernel vmspace or a user process vmspace.

Problem discovered by riastradh's "read(fd, NULL, 1)" test.

Account for lwps so that rump_sys_setuid() doesn't hit a KASSERT when
it tries to reaccount a switching procs lwps.

from Mato Lucina

branches: 1.31.4;
Init function pointers to nullop() so that callers don't need a dance

Reparent children of a dying lwproc. Fixes wpa_supplicant -B (and
most likely a bunch of other things).

implement lwp_create() and a few related interfaces.
main raison d'impalerment: the aio driver

branches: 1.28.2;
remove unnecesary verbosity: s/RUMP_CURLWP_MODEL/RUMP_CURLWP/

Allow compile-time optimizations to curlwp. This can have a pretty
staggering impact on performance. When running sendto() in a loop,
the improvement is 200k more calls per second with an inlined __thread
curlwp as opposed to the default. In other words, it shaves off hundreds
of CPU cycles per call (~20%). Even just eliminating the x86_curlwp()
call in favor of an inline gives an improvement of 60k calls per second.

Translate return values for emulations, e.g. Linux. For ports without
__HAVE_MINIMAL_EMUL, we simply look up the values from p->p_emul->e_errno.
For ports which cannot afford to keep an extra pointer per emul structure
around, we hope there is __HAVE_SYSCALL_INTERN support and thread the
errno values through p_emuldata. Notably, we cannot alter the syscall
method like most ports do with syscall_intern, since they do it via
p_mdproc, so MI code is not possible there.

stop ktrace at process exit

Enable holding implicit threads over explicit scheduling points (i.e.
a kernel driver calling rump_lwproc_switch()). Also, correctly handle
curcpu()->ci_curlwp and biglock there.

branches: 1.23.2;
Add RUMPUSER_LWP_CLEAR instead of overloading RUMPUSER_LWP_SET.
This simplifies some alternative hypervisor implementations.

Inform the rump kernel hypervisor of valid thread contexts so
that the implementation can allocate and release storage for them
in an optimal fashion, if necessary.

One more mutex which is used as a spin mutex and therefore needs
to be tagged as one, lest the mighty assert smites thee.

Support multiple syscall emuls.

Rump kernels with only local clients have it easy, since they have to
support at most two syscall vectors (some calls such as mount/unmount
can be made only through the "native" NetBSD vector). Remote clients
are a whole different tin of green beans. This change addresses local
clients only.

kill some -Wunused-but-set-variable warnings

branches: 1.18.4; 1.18.14;
More lim_free() fallout

Nuke all threads belonging to a process calling exec before allowing
the exec handshake to return.

In addition to being The Right Thing To Do, fixes some nasty
conditions for CLOEXEC fd's (or at least does so in theory, I
couldn't create any problems although I tried).

Fill in a functional struct lwp (especially l_mutex) before exposing
it on p_lwps.

branches: 1.15.2;
Explicitly zerofill some fields which are not within the boundaries
of p_start/endzero for whatever reasons.

Obviously, this fixes remote physio when running with MALLOC_OPTIONS J.
Otherwise PS_WEXIT is set in struct proc, proc_vmspace_getref() fails,
and copyout_proc() with it.

Reported by njoly

copy parent's p_comm when forking (if it exists)

Don't NULL out l_cpu, l_stat is the new rumpkernel way too.

Some lwp-walkers expect the correct value for l_stat, so use a flag
in l_flag instead of l_stat for the purpose of flagging lwps in a
dying proc.

fill in a default name for p_comm

branches: 1.10.2; 1.10.4;
Introduce RUMP_LOCALPROC_P() macro and use it.

When the client and the rump kernel are the same host process, all
threads blocking in the kernel automatically exit when the process
exists. However, for the sysproxy case this does not hold.
Typically it's ~harmless, but e.g. in the case of socket binding
following by poll it gets annoying.

Introduce sysproxy procexit, which wakes up all threads blocking
on a condition when a process's communication socket is closed.
The code is a little different from the regular kernel simply
because in a rump kernel l_mutex is not available at all times
(this is because scheduling happens on every kernel entry and exit,
and that path must be kept lockless for any reasonable performance).
Instead, use gating which makes sure all threads are either out of
the cv code or suspended in a well-known state. Then, wake up the
threads and tell them to get the hell out of our galaxy.

Support LOCKDEBUG. To use it, compile sys/rump with RUMP_LOCKDEBUG=yes.

requested by martin (sparc64 gdb cannot reliably produce a stack trace)

There is a use case where preserving the parent's fd table is
relevant, so to accommodate that change rump_lwproc_newproc() to
rump_lwproc_rfork(). The new interface has the rfork() fd table
semantics. The equivalent of rump_lwproc_newproc() is
rump_lwproc_rfork(RUMP_RFCFDG).

Support physio for remote processes.
==> add support for remote vmspace vmapbuf/vunmapbuf
==> add proper support for copyin/out_vmspace
==> add support for remote vmspace uvm_io
==> add support for non-curproc rumpuser_sp_copyin/out
==> store remote context in vm_map->pmap instead of
pthread_specificdata

In short, makes read/write of most (all?) block devices work from
a remote rump client via rump syscalls.

cleanup some old garbage

Add a new signal model RUMP_SIGMODEL_RECORD which records all
signals which are posted to a process.

Attach implicit threads to initproc instead of proc0. This way
applications which alter, by purpose or by accident, the uid in an
implicit thread are don't affect kernel threads.

from discussion with njoly

branches: 1.2.2; 1.2.4;
proper panic for trying to release implicit lwp

Implement rump_lwproc: the new lwp/proc management routines for
rump. These move the management of the pid/lwpid space from the
application into the kernel, make code more robust, and make it
possible to attach multiple lwp's to non-proc0 processes.

Fix a lock order reversal that caused hangs.

Move proc_lock into the data segment. It was dynamically allocated because
at the time we had mutex_obj_alloc() but not __cacheline_aligned.

- Replace pid_table_lock with a lockless lookup covered by pserialize, with
the "writer" side being pid_table expansion. The basic idea is that when
doing an LWP lookup there is usually already a lock held (p->p_lock), or a
spin mutex that needs to be taken (l->l_mutex), and either can be used to
get the found LWP stable and confidently determine that all is correct.

- For user processes LSLARVAL implies the same thing as LSIDL ("not visible
by ID"), and lookup by ID in proc0 doesn't really happen. In-tree the new
state should be understood by top(1), the tty subsystem and so on, and
would attract the attention of 3rd party kernel grovellers in time, so
remove it and just rely on LSIDL.

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

lwp0.l_lid needs to be 0.

Adapt to LWP ID allocation changes.

- Only increment nprocs when we're creating a new process, not just
when allocating a PID.
- Per above, proc_free_pid() no longer decrements nprocs. It's now done
in proc_free() right after proc_free_pid().
- Ensure nprocs is accessed using atomics everywhere.

branches: 1.44.4;
- Move the LW_RUNNING flag back into l_pflag: updating l_flag without lock
in softint_dispatch() is risky. May help with the "softint screwup"
panic.

- Correct the memory barriers around zombies switching into oblivion.

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

branches: 1.42.4;
Implement an aggressive psref leak detector

It is yet another psref leak detector that enables to tell where a leak occurs
while a simpler version that is already committed just tells an occurrence of a
leak.

Investigating of psref leaks is hard because once a leak occurs a percpu list of
psref that tracks references can be corrupted. A reference to a tracking object
is memorized in the list via an intermediate object (struct psref) that is
normally allocated on a stack of a thread. Thus, the intermediate object can be
overwritten on a leak resulting in corruption of the list.

The tracker makes a shadow entry to an intermediate object and stores some hints
into it (currently it's a caller address of psref_acquire). We can detect a
leak by checking the entries on certain points where any references should be
released such as the return point of syscalls and the end of each softint
handler.

The feature is expensive and enabled only if the kernel is built with
PSREF_DEBUG.

Proposed on tech-kern

Rumpkernel has its own thread deallocation. Add missing fstrans_lwp_dtor()
to lwproc_freelwp().

PR bin/50350: rump/rumpkern/t_sp/stress_{long,short} fail on Core 2 Quad

branches: 1.40.18;
Add lwp_find() - verbatim copy from the hard kernel.

Split p_xstat (composite wait(2) status code, or signal number depending
on context) into:
1. p_xexit: exit code
2. p_xsig: signal number
3. p_sflag & WCOREFLAG bit to indicated that the process core-dumped.

Fix the documentation of the flag bits in <sys/proc.h>

Allocate struct cpu_info dynamically. Saves quite a lot of BSS in the
common case and reduces rump kernel memory requirements by 10% or more
in really tiny deployments.

Put the kernelside rump kernel headers into <rump-sys> instead of
sprinkling them around the faction directories. Avoids having
to add a CPPFLAGS (or several) to pretty much every component
Makefile.

Leave compat headers around in the old locations.

The commit changes some autogenerated files, but I'll fix the
generators shortly and regen.

put lwp/proc stuff into the same source module (emul.c -> lwproc.c)

Give remote clients struct pmap storage. Although the pmap is unused,
that way we can sure that the pmap macro framework doesn't access all
the wrong places.

Don't share file descriptors between proc0 and local clients.

Use a different vmspace for rump kernel proc0 and local clients.
While the rump kernel and local clients are by definition in the same
host vmspace, there are subtle differences in how in-kernel code works
in case accessing the kernel vmspace or a user process vmspace.

Problem discovered by riastradh's "read(fd, NULL, 1)" test.

Account for lwps so that rump_sys_setuid() doesn't hit a KASSERT when
it tries to reaccount a switching procs lwps.

from Mato Lucina

branches: 1.31.4;
Init function pointers to nullop() so that callers don't need a dance

Reparent children of a dying lwproc. Fixes wpa_supplicant -B (and
most likely a bunch of other things).

implement lwp_create() and a few related interfaces.
main raison d'impalerment: the aio driver

branches: 1.28.2;
remove unnecesary verbosity: s/RUMP_CURLWP_MODEL/RUMP_CURLWP/

Allow compile-time optimizations to curlwp. This can have a pretty
staggering impact on performance. When running sendto() in a loop,
the improvement is 200k more calls per second with an inlined __thread
curlwp as opposed to the default. In other words, it shaves off hundreds
of CPU cycles per call (~20%). Even just eliminating the x86_curlwp()
call in favor of an inline gives an improvement of 60k calls per second.

Translate return values for emulations, e.g. Linux. For ports without
__HAVE_MINIMAL_EMUL, we simply look up the values from p->p_emul->e_errno.
For ports which cannot afford to keep an extra pointer per emul structure
around, we hope there is __HAVE_SYSCALL_INTERN support and thread the
errno values through p_emuldata. Notably, we cannot alter the syscall
method like most ports do with syscall_intern, since they do it via
p_mdproc, so MI code is not possible there.

stop ktrace at process exit

Enable holding implicit threads over explicit scheduling points (i.e.
a kernel driver calling rump_lwproc_switch()). Also, correctly handle
curcpu()->ci_curlwp and biglock there.

branches: 1.23.2;
Add RUMPUSER_LWP_CLEAR instead of overloading RUMPUSER_LWP_SET.
This simplifies some alternative hypervisor implementations.

Inform the rump kernel hypervisor of valid thread contexts so
that the implementation can allocate and release storage for them
in an optimal fashion, if necessary.

One more mutex which is used as a spin mutex and therefore needs
to be tagged as one, lest the mighty assert smites thee.

Support multiple syscall emuls.

Rump kernels with only local clients have it easy, since they have to
support at most two syscall vectors (some calls such as mount/unmount
can be made only through the "native" NetBSD vector). Remote clients
are a whole different tin of green beans. This change addresses local
clients only.

kill some -Wunused-but-set-variable warnings

branches: 1.18.4; 1.18.14;
More lim_free() fallout

Nuke all threads belonging to a process calling exec before allowing
the exec handshake to return.

In addition to being The Right Thing To Do, fixes some nasty
conditions for CLOEXEC fd's (or at least does so in theory, I
couldn't create any problems although I tried).

Fill in a functional struct lwp (especially l_mutex) before exposing
it on p_lwps.

branches: 1.15.2;
Explicitly zerofill some fields which are not within the boundaries
of p_start/endzero for whatever reasons.

Obviously, this fixes remote physio when running with MALLOC_OPTIONS J.
Otherwise PS_WEXIT is set in struct proc, proc_vmspace_getref() fails,
and copyout_proc() with it.

Reported by njoly

copy parent's p_comm when forking (if it exists)

Don't NULL out l_cpu, l_stat is the new rumpkernel way too.

Some lwp-walkers expect the correct value for l_stat, so use a flag
in l_flag instead of l_stat for the purpose of flagging lwps in a
dying proc.

fill in a default name for p_comm

branches: 1.10.2; 1.10.4;
Introduce RUMP_LOCALPROC_P() macro and use it.

When the client and the rump kernel are the same host process, all
threads blocking in the kernel automatically exit when the process
exists. However, for the sysproxy case this does not hold.
Typically it's ~harmless, but e.g. in the case of socket binding
following by poll it gets annoying.

Introduce sysproxy procexit, which wakes up all threads blocking
on a condition when a process's communication socket is closed.
The code is a little different from the regular kernel simply
because in a rump kernel l_mutex is not available at all times
(this is because scheduling happens on every kernel entry and exit,
and that path must be kept lockless for any reasonable performance).
Instead, use gating which makes sure all threads are either out of
the cv code or suspended in a well-known state. Then, wake up the
threads and tell them to get the hell out of our galaxy.

Support LOCKDEBUG. To use it, compile sys/rump with RUMP_LOCKDEBUG=yes.

requested by martin (sparc64 gdb cannot reliably produce a stack trace)

There is a use case where preserving the parent's fd table is
relevant, so to accommodate that change rump_lwproc_newproc() to
rump_lwproc_rfork(). The new interface has the rfork() fd table
semantics. The equivalent of rump_lwproc_newproc() is
rump_lwproc_rfork(RUMP_RFCFDG).

Support physio for remote processes.
==> add support for remote vmspace vmapbuf/vunmapbuf
==> add proper support for copyin/out_vmspace
==> add support for remote vmspace uvm_io
==> add support for non-curproc rumpuser_sp_copyin/out
==> store remote context in vm_map->pmap instead of
pthread_specificdata

In short, makes read/write of most (all?) block devices work from
a remote rump client via rump syscalls.

cleanup some old garbage

Add a new signal model RUMP_SIGMODEL_RECORD which records all
signals which are posted to a process.

Attach implicit threads to initproc instead of proc0. This way
applications which alter, by purpose or by accident, the uid in an
implicit thread are don't affect kernel threads.

from discussion with njoly

branches: 1.2.2; 1.2.4;
proper panic for trying to release implicit lwp

Implement rump_lwproc: the new lwp/proc management routines for
rump. These move the management of the pid/lwpid space from the
application into the kernel, make code more robust, and make it
possible to attach multiple lwp's to non-proc0 processes.

Move proc_lock into the data segment. It was dynamically allocated because
at the time we had mutex_obj_alloc() but not __cacheline_aligned.

- Replace pid_table_lock with a lockless lookup covered by pserialize, with
the "writer" side being pid_table expansion. The basic idea is that when
doing an LWP lookup there is usually already a lock held (p->p_lock), or a
spin mutex that needs to be taken (l->l_mutex), and either can be used to
get the found LWP stable and confidently determine that all is correct.

- For user processes LSLARVAL implies the same thing as LSIDL ("not visible
by ID"), and lookup by ID in proc0 doesn't really happen. In-tree the new
state should be understood by top(1), the tty subsystem and so on, and
would attract the attention of 3rd party kernel grovellers in time, so
remove it and just rely on LSIDL.

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

lwp0.l_lid needs to be 0.

Adapt to LWP ID allocation changes.

- Only increment nprocs when we're creating a new process, not just
when allocating a PID.
- Per above, proc_free_pid() no longer decrements nprocs. It's now done
in proc_free() right after proc_free_pid().
- Ensure nprocs is accessed using atomics everywhere.

branches: 1.44.4;
- Move the LW_RUNNING flag back into l_pflag: updating l_flag without lock
in softint_dispatch() is risky. May help with the "softint screwup"
panic.

- Correct the memory barriers around zombies switching into oblivion.

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

branches: 1.42.4;
Implement an aggressive psref leak detector

It is yet another psref leak detector that enables to tell where a leak occurs
while a simpler version that is already committed just tells an occurrence of a
leak.

Investigating of psref leaks is hard because once a leak occurs a percpu list of
psref that tracks references can be corrupted. A reference to a tracking object
is memorized in the list via an intermediate object (struct psref) that is
normally allocated on a stack of a thread. Thus, the intermediate object can be
overwritten on a leak resulting in corruption of the list.

The tracker makes a shadow entry to an intermediate object and stores some hints
into it (currently it's a caller address of psref_acquire). We can detect a
leak by checking the entries on certain points where any references should be
released such as the return point of syscalls and the end of each softint
handler.

The feature is expensive and enabled only if the kernel is built with
PSREF_DEBUG.

Proposed on tech-kern

Rumpkernel has its own thread deallocation. Add missing fstrans_lwp_dtor()
to lwproc_freelwp().

PR bin/50350: rump/rumpkern/t_sp/stress_{long,short} fail on Core 2 Quad

branches: 1.40.18;
Add lwp_find() - verbatim copy from the hard kernel.

Split p_xstat (composite wait(2) status code, or signal number depending
on context) into:
1. p_xexit: exit code
2. p_xsig: signal number
3. p_sflag & WCOREFLAG bit to indicated that the process core-dumped.

Fix the documentation of the flag bits in <sys/proc.h>

Allocate struct cpu_info dynamically. Saves quite a lot of BSS in the
common case and reduces rump kernel memory requirements by 10% or more
in really tiny deployments.

Put the kernelside rump kernel headers into <rump-sys> instead of
sprinkling them around the faction directories. Avoids having
to add a CPPFLAGS (or several) to pretty much every component
Makefile.

Leave compat headers around in the old locations.

The commit changes some autogenerated files, but I'll fix the
generators shortly and regen.

put lwp/proc stuff into the same source module (emul.c -> lwproc.c)

Give remote clients struct pmap storage. Although the pmap is unused,
that way we can sure that the pmap macro framework doesn't access all
the wrong places.

Don't share file descriptors between proc0 and local clients.

Use a different vmspace for rump kernel proc0 and local clients.
While the rump kernel and local clients are by definition in the same
host vmspace, there are subtle differences in how in-kernel code works
in case accessing the kernel vmspace or a user process vmspace.

Problem discovered by riastradh's "read(fd, NULL, 1)" test.

Account for lwps so that rump_sys_setuid() doesn't hit a KASSERT when
it tries to reaccount a switching procs lwps.

from Mato Lucina

branches: 1.31.4;
Init function pointers to nullop() so that callers don't need a dance

Reparent children of a dying lwproc. Fixes wpa_supplicant -B (and
most likely a bunch of other things).

implement lwp_create() and a few related interfaces.
main raison d'impalerment: the aio driver

branches: 1.28.2;
remove unnecesary verbosity: s/RUMP_CURLWP_MODEL/RUMP_CURLWP/

Allow compile-time optimizations to curlwp. This can have a pretty
staggering impact on performance. When running sendto() in a loop,
the improvement is 200k more calls per second with an inlined __thread
curlwp as opposed to the default. In other words, it shaves off hundreds
of CPU cycles per call (~20%). Even just eliminating the x86_curlwp()
call in favor of an inline gives an improvement of 60k calls per second.

Translate return values for emulations, e.g. Linux. For ports without
__HAVE_MINIMAL_EMUL, we simply look up the values from p->p_emul->e_errno.
For ports which cannot afford to keep an extra pointer per emul structure
around, we hope there is __HAVE_SYSCALL_INTERN support and thread the
errno values through p_emuldata. Notably, we cannot alter the syscall
method like most ports do with syscall_intern, since they do it via
p_mdproc, so MI code is not possible there.

stop ktrace at process exit

Enable holding implicit threads over explicit scheduling points (i.e.
a kernel driver calling rump_lwproc_switch()). Also, correctly handle
curcpu()->ci_curlwp and biglock there.

branches: 1.23.2;
Add RUMPUSER_LWP_CLEAR instead of overloading RUMPUSER_LWP_SET.
This simplifies some alternative hypervisor implementations.

Inform the rump kernel hypervisor of valid thread contexts so
that the implementation can allocate and release storage for them
in an optimal fashion, if necessary.

One more mutex which is used as a spin mutex and therefore needs
to be tagged as one, lest the mighty assert smites thee.

Support multiple syscall emuls.

Rump kernels with only local clients have it easy, since they have to
support at most two syscall vectors (some calls such as mount/unmount
can be made only through the "native" NetBSD vector). Remote clients
are a whole different tin of green beans. This change addresses local
clients only.

kill some -Wunused-but-set-variable warnings

branches: 1.18.4; 1.18.14;
More lim_free() fallout

Nuke all threads belonging to a process calling exec before allowing
the exec handshake to return.

In addition to being The Right Thing To Do, fixes some nasty
conditions for CLOEXEC fd's (or at least does so in theory, I
couldn't create any problems although I tried).

Fill in a functional struct lwp (especially l_mutex) before exposing
it on p_lwps.

branches: 1.15.2;
Explicitly zerofill some fields which are not within the boundaries
of p_start/endzero for whatever reasons.

Obviously, this fixes remote physio when running with MALLOC_OPTIONS J.
Otherwise PS_WEXIT is set in struct proc, proc_vmspace_getref() fails,
and copyout_proc() with it.

Reported by njoly

copy parent's p_comm when forking (if it exists)

Don't NULL out l_cpu, l_stat is the new rumpkernel way too.

Some lwp-walkers expect the correct value for l_stat, so use a flag
in l_flag instead of l_stat for the purpose of flagging lwps in a
dying proc.

fill in a default name for p_comm

branches: 1.10.2; 1.10.4;
Introduce RUMP_LOCALPROC_P() macro and use it.

When the client and the rump kernel are the same host process, all
threads blocking in the kernel automatically exit when the process
exists. However, for the sysproxy case this does not hold.
Typically it's ~harmless, but e.g. in the case of socket binding
following by poll it gets annoying.

Introduce sysproxy procexit, which wakes up all threads blocking
on a condition when a process's communication socket is closed.
The code is a little different from the regular kernel simply
because in a rump kernel l_mutex is not available at all times
(this is because scheduling happens on every kernel entry and exit,
and that path must be kept lockless for any reasonable performance).
Instead, use gating which makes sure all threads are either out of
the cv code or suspended in a well-known state. Then, wake up the
threads and tell them to get the hell out of our galaxy.

Support LOCKDEBUG. To use it, compile sys/rump with RUMP_LOCKDEBUG=yes.

requested by martin (sparc64 gdb cannot reliably produce a stack trace)

There is a use case where preserving the parent's fd table is
relevant, so to accommodate that change rump_lwproc_newproc() to
rump_lwproc_rfork(). The new interface has the rfork() fd table
semantics. The equivalent of rump_lwproc_newproc() is
rump_lwproc_rfork(RUMP_RFCFDG).

Support physio for remote processes.
==> add support for remote vmspace vmapbuf/vunmapbuf
==> add proper support for copyin/out_vmspace
==> add support for remote vmspace uvm_io
==> add support for non-curproc rumpuser_sp_copyin/out
==> store remote context in vm_map->pmap instead of
pthread_specificdata

In short, makes read/write of most (all?) block devices work from
a remote rump client via rump syscalls.

cleanup some old garbage

Add a new signal model RUMP_SIGMODEL_RECORD which records all
signals which are posted to a process.

Attach implicit threads to initproc instead of proc0. This way
applications which alter, by purpose or by accident, the uid in an
implicit thread are don't affect kernel threads.

from discussion with njoly

branches: 1.2.2; 1.2.4;
proper panic for trying to release implicit lwp

Implement rump_lwproc: the new lwp/proc management routines for
rump. These move the management of the pid/lwpid space from the
application into the kernel, make code more robust, and make it
possible to attach multiple lwp's to non-proc0 processes.

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

lwp0.l_lid needs to be 0.

Adapt to LWP ID allocation changes.

- Only increment nprocs when we're creating a new process, not just
when allocating a PID.
- Per above, proc_free_pid() no longer decrements nprocs. It's now done
in proc_free() right after proc_free_pid().
- Ensure nprocs is accessed using atomics everywhere.

branches: 1.44.4;
- Move the LW_RUNNING flag back into l_pflag: updating l_flag without lock
in softint_dispatch() is risky. May help with the "softint screwup"
panic.

- Correct the memory barriers around zombies switching into oblivion.

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

branches: 1.42.4;
Implement an aggressive psref leak detector

It is yet another psref leak detector that enables to tell where a leak occurs
while a simpler version that is already committed just tells an occurrence of a
leak.

Investigating of psref leaks is hard because once a leak occurs a percpu list of
psref that tracks references can be corrupted. A reference to a tracking object
is memorized in the list via an intermediate object (struct psref) that is
normally allocated on a stack of a thread. Thus, the intermediate object can be
overwritten on a leak resulting in corruption of the list.

The tracker makes a shadow entry to an intermediate object and stores some hints
into it (currently it's a caller address of psref_acquire). We can detect a
leak by checking the entries on certain points where any references should be
released such as the return point of syscalls and the end of each softint
handler.

The feature is expensive and enabled only if the kernel is built with
PSREF_DEBUG.

Proposed on tech-kern

Rumpkernel has its own thread deallocation. Add missing fstrans_lwp_dtor()
to lwproc_freelwp().

PR bin/50350: rump/rumpkern/t_sp/stress_{long,short} fail on Core 2 Quad

branches: 1.40.18;
Add lwp_find() - verbatim copy from the hard kernel.

Split p_xstat (composite wait(2) status code, or signal number depending
on context) into:
1. p_xexit: exit code
2. p_xsig: signal number
3. p_sflag & WCOREFLAG bit to indicated that the process core-dumped.

Fix the documentation of the flag bits in <sys/proc.h>

Allocate struct cpu_info dynamically. Saves quite a lot of BSS in the
common case and reduces rump kernel memory requirements by 10% or more
in really tiny deployments.

Put the kernelside rump kernel headers into <rump-sys> instead of
sprinkling them around the faction directories. Avoids having
to add a CPPFLAGS (or several) to pretty much every component
Makefile.

Leave compat headers around in the old locations.

The commit changes some autogenerated files, but I'll fix the
generators shortly and regen.

put lwp/proc stuff into the same source module (emul.c -> lwproc.c)

Give remote clients struct pmap storage. Although the pmap is unused,
that way we can sure that the pmap macro framework doesn't access all
the wrong places.

Don't share file descriptors between proc0 and local clients.

Use a different vmspace for rump kernel proc0 and local clients.
While the rump kernel and local clients are by definition in the same
host vmspace, there are subtle differences in how in-kernel code works
in case accessing the kernel vmspace or a user process vmspace.

Problem discovered by riastradh's "read(fd, NULL, 1)" test.

Account for lwps so that rump_sys_setuid() doesn't hit a KASSERT when
it tries to reaccount a switching procs lwps.

from Mato Lucina

branches: 1.31.4;
Init function pointers to nullop() so that callers don't need a dance

Reparent children of a dying lwproc. Fixes wpa_supplicant -B (and
most likely a bunch of other things).

implement lwp_create() and a few related interfaces.
main raison d'impalerment: the aio driver

branches: 1.28.2;
remove unnecesary verbosity: s/RUMP_CURLWP_MODEL/RUMP_CURLWP/

Allow compile-time optimizations to curlwp. This can have a pretty
staggering impact on performance. When running sendto() in a loop,
the improvement is 200k more calls per second with an inlined __thread
curlwp as opposed to the default. In other words, it shaves off hundreds
of CPU cycles per call (~20%). Even just eliminating the x86_curlwp()
call in favor of an inline gives an improvement of 60k calls per second.

Translate return values for emulations, e.g. Linux. For ports without
__HAVE_MINIMAL_EMUL, we simply look up the values from p->p_emul->e_errno.
For ports which cannot afford to keep an extra pointer per emul structure
around, we hope there is __HAVE_SYSCALL_INTERN support and thread the
errno values through p_emuldata. Notably, we cannot alter the syscall
method like most ports do with syscall_intern, since they do it via
p_mdproc, so MI code is not possible there.

stop ktrace at process exit

Enable holding implicit threads over explicit scheduling points (i.e.
a kernel driver calling rump_lwproc_switch()). Also, correctly handle
curcpu()->ci_curlwp and biglock there.

branches: 1.23.2;
Add RUMPUSER_LWP_CLEAR instead of overloading RUMPUSER_LWP_SET.
This simplifies some alternative hypervisor implementations.

Inform the rump kernel hypervisor of valid thread contexts so
that the implementation can allocate and release storage for them
in an optimal fashion, if necessary.

One more mutex which is used as a spin mutex and therefore needs
to be tagged as one, lest the mighty assert smites thee.

Support multiple syscall emuls.

Rump kernels with only local clients have it easy, since they have to
support at most two syscall vectors (some calls such as mount/unmount
can be made only through the "native" NetBSD vector). Remote clients
are a whole different tin of green beans. This change addresses local
clients only.

kill some -Wunused-but-set-variable warnings

branches: 1.18.4; 1.18.14;
More lim_free() fallout

Nuke all threads belonging to a process calling exec before allowing
the exec handshake to return.

In addition to being The Right Thing To Do, fixes some nasty
conditions for CLOEXEC fd's (or at least does so in theory, I
couldn't create any problems although I tried).

Fill in a functional struct lwp (especially l_mutex) before exposing
it on p_lwps.

branches: 1.15.2;
Explicitly zerofill some fields which are not within the boundaries
of p_start/endzero for whatever reasons.

Obviously, this fixes remote physio when running with MALLOC_OPTIONS J.
Otherwise PS_WEXIT is set in struct proc, proc_vmspace_getref() fails,
and copyout_proc() with it.

Reported by njoly

copy parent's p_comm when forking (if it exists)

Don't NULL out l_cpu, l_stat is the new rumpkernel way too.

Some lwp-walkers expect the correct value for l_stat, so use a flag
in l_flag instead of l_stat for the purpose of flagging lwps in a
dying proc.

fill in a default name for p_comm

branches: 1.10.2; 1.10.4;
Introduce RUMP_LOCALPROC_P() macro and use it.

When the client and the rump kernel are the same host process, all
threads blocking in the kernel automatically exit when the process
exists. However, for the sysproxy case this does not hold.
Typically it's ~harmless, but e.g. in the case of socket binding
following by poll it gets annoying.

Introduce sysproxy procexit, which wakes up all threads blocking
on a condition when a process's communication socket is closed.
The code is a little different from the regular kernel simply
because in a rump kernel l_mutex is not available at all times
(this is because scheduling happens on every kernel entry and exit,
and that path must be kept lockless for any reasonable performance).
Instead, use gating which makes sure all threads are either out of
the cv code or suspended in a well-known state. Then, wake up the
threads and tell them to get the hell out of our galaxy.

Support LOCKDEBUG. To use it, compile sys/rump with RUMP_LOCKDEBUG=yes.

requested by martin (sparc64 gdb cannot reliably produce a stack trace)

There is a use case where preserving the parent's fd table is
relevant, so to accommodate that change rump_lwproc_newproc() to
rump_lwproc_rfork(). The new interface has the rfork() fd table
semantics. The equivalent of rump_lwproc_newproc() is
rump_lwproc_rfork(RUMP_RFCFDG).

Support physio for remote processes.
==> add support for remote vmspace vmapbuf/vunmapbuf
==> add proper support for copyin/out_vmspace
==> add support for remote vmspace uvm_io
==> add support for non-curproc rumpuser_sp_copyin/out
==> store remote context in vm_map->pmap instead of
pthread_specificdata

In short, makes read/write of most (all?) block devices work from
a remote rump client via rump syscalls.

cleanup some old garbage

Add a new signal model RUMP_SIGMODEL_RECORD which records all
signals which are posted to a process.

Attach implicit threads to initproc instead of proc0. This way
applications which alter, by purpose or by accident, the uid in an
implicit thread are don't affect kernel threads.

from discussion with njoly

branches: 1.2.2; 1.2.4;
proper panic for trying to release implicit lwp

Implement rump_lwproc: the new lwp/proc management routines for
rump. These move the management of the pid/lwpid space from the
application into the kernel, make code more robust, and make it
possible to attach multiple lwp's to non-proc0 processes.

lwp0.l_lid needs to be 0.

Adapt to LWP ID allocation changes.

- Only increment nprocs when we're creating a new process, not just
when allocating a PID.
- Per above, proc_free_pid() no longer decrements nprocs. It's now done
in proc_free() right after proc_free_pid().
- Ensure nprocs is accessed using atomics everywhere.

branches: 1.44.4;
- Move the LW_RUNNING flag back into l_pflag: updating l_flag without lock
in softint_dispatch() is risky. May help with the "softint screwup"
panic.

- Correct the memory barriers around zombies switching into oblivion.

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

branches: 1.42.4;
Implement an aggressive psref leak detector

It is yet another psref leak detector that enables to tell where a leak occurs
while a simpler version that is already committed just tells an occurrence of a
leak.

Investigating of psref leaks is hard because once a leak occurs a percpu list of
psref that tracks references can be corrupted. A reference to a tracking object
is memorized in the list via an intermediate object (struct psref) that is
normally allocated on a stack of a thread. Thus, the intermediate object can be
overwritten on a leak resulting in corruption of the list.

The tracker makes a shadow entry to an intermediate object and stores some hints
into it (currently it's a caller address of psref_acquire). We can detect a
leak by checking the entries on certain points where any references should be
released such as the return point of syscalls and the end of each softint
handler.

The feature is expensive and enabled only if the kernel is built with
PSREF_DEBUG.

Proposed on tech-kern

Rumpkernel has its own thread deallocation. Add missing fstrans_lwp_dtor()
to lwproc_freelwp().

PR bin/50350: rump/rumpkern/t_sp/stress_{long,short} fail on Core 2 Quad

branches: 1.40.18;
Add lwp_find() - verbatim copy from the hard kernel.

Split p_xstat (composite wait(2) status code, or signal number depending
on context) into:
1. p_xexit: exit code
2. p_xsig: signal number
3. p_sflag & WCOREFLAG bit to indicated that the process core-dumped.

Fix the documentation of the flag bits in <sys/proc.h>

Allocate struct cpu_info dynamically. Saves quite a lot of BSS in the
common case and reduces rump kernel memory requirements by 10% or more
in really tiny deployments.

Put the kernelside rump kernel headers into <rump-sys> instead of
sprinkling them around the faction directories. Avoids having
to add a CPPFLAGS (or several) to pretty much every component
Makefile.

Leave compat headers around in the old locations.

The commit changes some autogenerated files, but I'll fix the
generators shortly and regen.

put lwp/proc stuff into the same source module (emul.c -> lwproc.c)

Give remote clients struct pmap storage. Although the pmap is unused,
that way we can sure that the pmap macro framework doesn't access all
the wrong places.

Don't share file descriptors between proc0 and local clients.

Use a different vmspace for rump kernel proc0 and local clients.
While the rump kernel and local clients are by definition in the same
host vmspace, there are subtle differences in how in-kernel code works
in case accessing the kernel vmspace or a user process vmspace.

Problem discovered by riastradh's "read(fd, NULL, 1)" test.

Account for lwps so that rump_sys_setuid() doesn't hit a KASSERT when
it tries to reaccount a switching procs lwps.

from Mato Lucina

branches: 1.31.4;
Init function pointers to nullop() so that callers don't need a dance

Reparent children of a dying lwproc. Fixes wpa_supplicant -B (and
most likely a bunch of other things).

implement lwp_create() and a few related interfaces.
main raison d'impalerment: the aio driver

branches: 1.28.2;
remove unnecesary verbosity: s/RUMP_CURLWP_MODEL/RUMP_CURLWP/

Allow compile-time optimizations to curlwp. This can have a pretty
staggering impact on performance. When running sendto() in a loop,
the improvement is 200k more calls per second with an inlined __thread
curlwp as opposed to the default. In other words, it shaves off hundreds
of CPU cycles per call (~20%). Even just eliminating the x86_curlwp()
call in favor of an inline gives an improvement of 60k calls per second.