remove maxmem extern, var removed from all archs long ago

Adjust KERNEL_ASSERT_UNLOCKED() to not assert during a panic.

KERNEL_ASSERT_UNLOCKED calls _kernel_lock_held() which returns true
if panicstr || db_active which triggers this assert. Workaround this by
checking them before.

This will alter the following Syzkaller reports:
Reported-by: syzbot+169110a0815838ab5940@syzkaller.appspotmail.com
Reported-by: syzbot+3c2eced405b9de6f79c2@syzkaller.appspotmail.com

OK mpi@

clockintr: move callback-specific API behaviors to "clockrequest" namespace

The API's behavior when invoked from a callback function is impossible
to document. Move the special behavior into a distinct namespace,
"clockrequest".

- Add a 'struct clockrequest'. Basically a stripped-down 'struct clockintr'
for exclusive use during clockintr_dispatch().
- In clockintr_queue, replace the "cq_shadow" clockintr with a "cq_request"
clockrequest. They serve the same purpose.
- CLST_SHADOW_PENDING -> CR_RESCHEDULE; different namespace, same meaning.
- CLST_IGNORE_SHADOW -> CLST_IGNORE_REQUEST; same meaning.
- Move shadow branch in clockintr_advance() to clockrequest_advance().
- clockintr_request_random() becomes clockrequest_advance_random().
- Delete dead shadow branches in clockintr_cancel(), clockintr_schedule().
- Callback functions now get a clockrequest pointer instead of a special
clockintr pointer: update all prototypes, callers.

No functional change intended.

kernel: expand fixed clock interrupt periods to 64-bit values

Technically, all the current fixed clock interrupt periods fit within
an unsigned 32-bit value. But 32-bit multiplication is an accident
waiting to happen. So, expand the fixed periods for hardclock,
statclock, profclock, and roundrobin to 64-bit values.

One exception: statclock_mask remains 32-bit because random(9) yields
32-bit values. Update the initclocks() comment to make it clear that
this is not an accident.

clockintr, statclock: eliminate clockintr_statclock() wrapper

- Move remaining statclock variables from kern_clockintr.c to
kern_clock.c. Move statclock variable initialization from
clockintr_init() into initclocks().

- Change statclock() prototype to make it a legal clockintr
callback function and establish the handle with statclock()
instead clockintr_statclock().

- Merge the contents of clockintr_statclock() into statclock().
statclock() can now reschedule itself and handles multiple
expirations transparently.

- Make statclock_avg visible from sys/systm.h so that clockintr_cpu_init()
can use it to advance the statclock across suspend/hibernate.

Thread: https://marc.info/?l=openbsd-tech&m=169428749720476&w=2

clockintr: replace CL_RNDSTAT with global variable statclock_is_randomized

In order to separate the statclock from the clock interrupt subsystem
we need to move all statclock state out into the broader kernel.

Start by replacing the CL_RNDSTAT flag with a new global variable,
"statclock_is_randomized", in kern_clock.c. Update all clockintr_init()
callers to set the boolean instead of passing the flag.

Thread: https://marc.info/?l=openbsd-tech&m=169428749720476&w=2

all platforms: separate cpu_initclocks() from cpu_startclock()

To give the primary CPU an opportunity to perform clock interrupt
preparation in a machine-independent manner we need to separate the
"initialization" parts of cpu_initclocks() from the "start the clock
interrupt" parts. Currently, cpu_initclocks() does everything all at
once, so there is no space for this MI setup.

Many platforms have more-or-less already done this separation by
implementing a separate routine named "cpu_startclock()". This patch
promotes cpu_startclock() from de facto standard to mandatory API.

- Prototype cpu_startclock() in sys/systm.h alongside cpu_initclocks().
The separation of responsibility between the two routines is a bit
fuzzy but the basic guidelines are as follows:

+ cpu_initclocks() must initialize hz, stathz, and profhz, and call
clockintr_init().

+ cpu_startclock() must call clockintr_cpu_init() and start the clock
interrupt cycle on the calling CPU.

These guidelines will shift in the future, but that's the way things
stand as of *this* commit.

- In initclocks(): first call cpu_initclocks(), then do MI setup, and
last call cpu_startclock().

- On platforms where cpu_startclock() already exists: don't call
cpu_startclock() from cpu_initclocks() anymore.

- On platforms where cpu_startclock() doesn't yet exist: implement it.
Usually this is as simple as dividing cpu_initclocks() in two.

Tested on amd64 (i8254, lapic), arm64, i386 (i8254, lapic), macppc,
mips64/octeon, and sparc64. Tested on arm/armv7 (agtimer(4)) by
phessler@ and jmatthew@. Tested on m88k/luna88k by aoyama@. Tested
on powerpc64 by gkoehler@ and mlarkin@. Tested on riscv64 by
jmatthew@.

Thread: https://marc.info/?l=openbsd-tech&m=169195251322149&w=2

hardclock(9): move setitimer(2) code into itimer_update()

- Move the setitimer(2) code responsible for updating the ITIMER_VIRTUAL
and ITIMER_PROF timers from hardclock(9) into a new clock interrupt
routine, itimer_update(). itimer_update() is periodic and runs at the
same frequency as the hardclock.

+ Revise itimerdecr() to run within itimer_mtx instead of entering
and leaving it.

- Each schedstate_percpu has its own itimer_update() handle, spc_itimer.
A new scheduler flag, SPCF_ITIMER, indicates whether spc_itimer was
started during the last mi_switch() and needs to be stopped during the
next mi_switch() or sched_exit().

- A new per-process flag, PS_ITIMER, indicates whether ITIMER_VIRTUAL
and/or ITIMER_PROF are running. Checking the flag is easier than
entering itimer_mtx to check process.ps_timer[]. The flag is set
and cleared in a new helper function, process_reset_itimer_flag().

- In setitimer(), call need_resched() when the state of ITIMER_VIRTUAL
or ITIMER_PROF is changed to force an mi_switch() and update
spc_itimer.

claudio@ notes that ITIMER_PROF could be implemented as a high-res
timer using the thread's execution time as a guide for when to
interrupt the process and assert SIGPROF. This would probably work
really well in single-threaded processes. ITIMER_VIRTUAL would be
more difficult to make high-res, though, as you need to exclude time
spent in the kernel.

Tested on powerpc64 by gkoehler@. With input from claudio@.

Thread: https://marc.info/?l=openbsd-tech&m=169038818517101&w=2

ok claudio@

struct sleep_state is no longer used, remove it.
Also remove the priority argument to sleep_finish() the code can use
the p_flag P_SINTR flag to know if the signal check is needed or not.
OK cheloha@ kettenis@ mpi@

First step at removing struct sleep_state.

Pass the timeout and sleep priority not only to sleep_setup() but also
to sleep_finish(). With that sls_timeout and sls_catch can be removed
from struct sleep_state.

The timeout is now setup first thing in sleep_finish() and no longer as
last thing in sleep_setup(). This should not cause a noticeable difference
since the code run between sleep_setup() and sleep_finish() is minimal.

OK kettenis@

On systems without xonly mmu hardware-enforcement, we can still mitigate
against classic BROP with a range-checking wrapper in front of copyin() and
copyinstr() which ensures the userland source doesn't overlap the main program
text, ld.so text, signal tramp text (it's mapping is hard to distinguish
so it comes along for the ride), or libc.so text. ld.so tells the kernel
libc.so text range with msyscall(2). The range checking for 2-4 elements is
done without locking (because all 4 ranges are immutable!) and is inexpensive.

write(sock, &open, 400) now fails with EFAULT. No programs have been
discovered which require reading their own text segments with a system call.

On a machine without mmu enforcement, a test program reports the following:
userland kernel
ld.so readable unreadable
mmap xz unreadable unreadable
mmap x readable readable
mmap nrx readable readable
mmap nwx readable readable
mmap xnwx readable readable
main readable unreadable
libc unmapped? readable unreadable
libc mapped readable unreadable

ok kettenis, additional help from miod

Remove copystr(9), unless used internally by copy{in,out}str.

Allow suspend with root on sdmmc(4).

ok deraadt@

Clean up the netlock macros. Merge NET_RLOCK_IN_SOFTNET and
NET_RLOCK_IN_IOCTL, which have the same implementation. The R and
W are hard to see, call the new macro NET_LOCK_SHARED. Rename the
opposite assertion from NET_ASSERT_WLOCKED to NET_ASSERT_LOCKED_EXCLUSIVE.
Update some outdated comments about net locking.
OK mpi@ mvs@

Add db_rint(), an MI interface to db_enter() copied from kdbrint() in vax code

If ddb.console is set and your serial console driver uses it, db_rint(),
lets you enter ddb(4) by typing the ESC D escape sequence. This is
useful for drivers like sfuart(4) where the hardware doesn't have a true
BREAK mechanism.

Suggested by miod@, ok kettenis@ miod@

Remove old poll/select wakeup mechanism.

Also remove unneeded seltrue() and selfalse().

OK mpi@ jsg@

We only have one syscall table: inline sysent/SYS_MAXSYSCALL and
SYS_syscall as the nosys() function into the MD syscall entry
routines and the SYSCALL_DEBUG support. Adjust alpha's syscall
check to match the other archs. Also, make sysent const to get it
into .rodata.

With that, 'struct emul' is unused: delete it and all its references

ok millert@

kernel: introduce per-CPU panic(9) message buffers

Add a 512-byte buffer (ci_panicbuf) to each cpu_info struct on each
platform for use by panic(9). The first panic on a given CPU writes
its message to this buffer. Subsequent panics on a given CPU print
the panic message to the console but do not modify the buffer. This
aids debugging in two cases:

- If 2+ CPUs panic simultaneously there is no risk of garbled messages
in the panic buffer.

- If a CPU panics and then the operator causes a second panic while
using ddb(4), the operator can still recall the first failure on
a particular CPU.

Misc. changes to support this bigger change:

- Set panicstr atomically to identify the first CPU to reach panic().

- Tweak db_show_panic_cmd() to print all panic messages across all
CPUs. Prefix the first panic with an asterisk ('*').

- Prefer db_printf() to printf() during a panic if we have it.
Apparently it disturbs less global state.

- On amd64, tweak fault() to write the local panic buffer. This needs
more work.

Prompted by bluhm@ and deraadt@. Mostly written by deraadt@.
Discussed with bluhm@, deraadt@ and kettenis@.

Borne from a discussion on tech@ about making panic(9) more MP-safe:

https://marc.info/?l=openbsd-tech&m=162086462316143&w=2

ok kettenis@, visa@, bluhm@, deraadt@

time to add NET_ASSERT_WLOCKED()

with moving towards NET_RLOCK...() we need NET_ASSERT_WLOCKED()
to check caller owns netlock exclusively.

OK @bluhm

Simplify sleep_setup API to two operations in preparation for splitting
the SCHED_LOCK().

Putting a thread on a sleep queue is reduce to the following:

sleep_setup();
/* check condition or release lock */
sleep_finish();

Previous version ok cheloha@, jmatthew@, ok claudio@

Remove obsolete vnode operation vector declarations.

OK bluhm@, claudio@, mpi@, semarie@

Make NET_LOCK() assertions conditional to DIAGNOSTIC

This saves about 2.5 KiB off amd64's RAMDISK after gzip compression.

OK deraadt@, mpi@, cheloha@

tsleep(9): add global "nowake" channel for threads avoiding wakeup(9)

It would be convenient if there were a channel a thread could sleep on
to indicate they do not want any wakeup(9) broadcasts. The easiest way
to do this is to add an "int nowake" to kern_synch.c and extern it in
sys/systm.h. You use it like this:

#include <sys/systm.h>

tsleep_nsec(&nowait, ...);

There is now no need to handroll a local dead channel, e.g.

int chan;

tsleep_nsec(&chan, ...);

which expands the stack. Local dead channels will be replaced with
&nowake in later patches.

One possible problem with this "one global channel" approach is sleep
queue congestion. If you have lots of threads sleeping on &nowake you
might slow down a wakeup(9) on a different channel that hashes into
the same queue. Unsure how much of problem this actually is, if at all.

NetBSD and FreeBSD have a "pause" interface in the kernel that chooses
a suitable channel automatically. To keep things simple and avoid
adding a new interface we will start with this global channel.

Discussed with mpi@, claudio@, kettenis@, and deraadt@.

Basically designed by kettenis@, who vetoed my other proposals.

Bugs caught by deraadt@, tb@, and patrick@.

Declare hw_{prod,serial,uuid,vendor,ver} in <sys/systm.h>.

OK deraadt@, mpi@

dev/rndvar.h no longer has statistical interfaces (removed during various
conversion steps). it only contains kernel prototypes for 4 interfaces,
all of which legitimately belong in sys/systm.h, which are already included
by all enqueue_randomness() users.

Document the various flavors of NET_LOCK() and rename the reader version.

Since our last concurrency mistake only ioctl(2) ans sysctl(2) code path
take the reader lock. This is mostly for documentation purpose as long as
the softnet thread is converted back to use a read lock.

dlg@ said that comments should be good enough.

ok sashan@

tsleep_nsec(9): add MAXTSLP macro, the maximum sleep duration

This macro will be useful for truncating durations below INFSLP
(UINT64_MAX) when converting from a timespec or timeval to a count
of nanoseconds before calling tsleep_nsec(9), msleep_nsec(9), or
rwsleep_nsec(9).

A relative timespec can hold many more nanoseconds than a uint64_t
can. TIMESPEC_TO_NSEC() and TIMEVAL_TO_NSEC() check for overflow,
returning UINT64_MAX if the conversion would overflow a uint64_t.

Thus, MAXTSLP will make it easy to avoid inadvertently passing INFSLP
to tsleep_nsec(9) et al. when the caller intended to set a timeout.

The code in such a case might look something like this:

uint64_t nsecs = MIN(TIMESPEC_TO_NSEC(&ts), MAXTSLP);

The macro may also be useful for rejecting intervals that are "too large",
e.g. for sockets with timeouts, if the timeout duration is to be stored
as a uint64_t in an object in the kernel. The code in such a case might
look something like this:

case SIOCTIMEOUT:
{
struct timeval *tv = (struct timeval *)data;
uint64_t nsecs;

if (tv->tv_sec < 0 || !timerisvalid(tv))
return EINVAL;

nsecs = TIMEVAL_TO_NSEC(tv);
if (nsecs > MAXTSLP)
return EOVERFLOW;

obj.timeout = nsecs;
break;
}

Idea suggested by visa@.

ok visa@

Move kernel locking inside the sleep machinery. This enables calling
rwsleep(9) with PCATCH and rw_enter(9) with RW_INTR without the kernel
lock. In addition, now tsleep(9) with PCATCH should be safe to use
without the kernel lock if the sleep is purely time-based.

Tested by anton@, cheloha@, chris@
OK anton@, cheloha@

softclock: move softintr registration/scheduling into timeout module

softclock() is scheduled from hardclock(9) because long ago callouts were
processed from hardclock(9) directly. The introduction of timeout(9) circa
2000 moved all callout processing into a dedicated module, but the softclock
scheduling stayed behind in hardclock(9).

We can move all the softclock() "stuff" into the timeout module to make
kern_clock.c a bit cleaner. Neither initclocks() nor hardclock(9) need
to "know" about softclock(). The initial softclock() softintr registration
can be done from timeout_proc_init() and softclock() can be scheduled
from timeout_hardclock_update().

ok visa@

Add tsleep_nsec(9), msleep_nsec(9), and rwsleep_nsec(9).

Equivalent to their unsuffixed counterparts except that (a) they take
a timeout in terms of nanoseconds, and (b) INFSLP, aka UINT64_MAX (not
zero) indicates that a timeout should not be set.

For now, zero nanoseconds is not a strictly valid invocation: we log a
warning on DIAGNOSTIC kernels if we see such a call. We still sleep
until the next tick in such a case, however. In the future this could
become some sort of poll... TBD.

To facilitate conversions to these interfaces: add inline conversion
functions to sys/time.h for turning your timeout into nanoseconds.

Also do a few easy conversions for warmup and to demonstrate how
further conversions should be done.

Lots of input from mpi@ and ratchov@. Additional input from tedu@,
deraadt@, mortimer@, millert@, and claudio@.

Partly inspired by FreeBSD r247787.

positive feedback from deraadt@, ok mpi@

Remove file name and line number output from witness(4)

Reduce code clutter by removing the file name and line number output
from witness(4). Typically it is easy enough to locate offending locks
using the stack traces that are shown in lock order conflict reports.
Tricky cases can be tracked using sysctl kern.witness.locktrace=1 .

This patch additionally removes the witness(4) wrapper for mutexes.
Now each mutex implementation has to invoke the WITNESS_*() macros
in order to utilize the checker.

Discussed with and OK dlg@, OK mpi@

Add sleep_finish_all(), which provides the common combo of sleep_finish(),
sleep_finish_timeout(), and sleep_finish_signal() with error preferencing,
and then use it in five places.

ok mpi@

Do not panic from ddb(4) when a lock requirement isn't fulfilled.

Extend the logic already present for panic() to any DDB-related
operation such that if ddb(4) is entered because of a fault or
other trap it is still possible to call 'boot reboot'.

While here stop printing splassert() messages as well, to not fill
the buffer.

ok visa@, deraadt@

Use a temporary chacha instance to fill large randomdata sections. Avoids
grabbing the rnglock repeatedly.

ok deraadt@ djm@

Show uvm_fault and trace when typing show panic on a page fault'd kernel

Currently there is only support for amd64, if this change settles
I will add support for the rest of the architectures.

OK kettenis@.

add code to provide simple wait condition handling.

this will be used to replace the bare sleep_state handling in a
bunch of places, starting with the barriers.

Do not call splassert_fail() if splassert_ctl is <= 0.

This matches splassert(9)s behavior and prevent noise when a CPU
panic(9) and set splassert_ctl to 0.

Found the hardway by sthen@

Introduce a reader version of the NET_LOCK().

This will be used to first allow read-only ioctl(2) to be executed while
the softnet taskq is running. Then it will allows us to execute multiple
softnet taskq in parallel.

Tested by Hrvoje Popovski, ok kettenis@, sashan@, visa@, tb@

Remove NET_LOCK()'s argument.

Tested by Hrvoje Popovski, ok bluhm@

Stop doing an splsoftnet()/splx() dance inside the NET_LOCK().

This will allow us to not carry a returned value when entering a critical
section.

ok bluhm@, visa@

clang has builtin_memmove. ok deraadt

Add copyin32(9) prototype.

Enable the NET_LOCK(), take 3.

Recursions are still marked as XXXSMP.

ok deraadt@, bluhm@

Rename Debugger() into db_enter().

Using a name with the 'db_' prefix makes it invisible from the dynamic
profiler.

ok deraadt@, kettenis@, visa@

Unifdef KGDB.

It doesn't compile und hasn't been working during the last decade.

ok kettenis@, deraadt@

Hook up mplock to witness(4) on amd64 and i386.

Revert the NET_LOCK() and bring back pf's contention lock for release.

For the moment the NET_LOCK() is always taken by threads running under
KERNEL_LOCK(). That means it doesn't buy us anything except a possible
deadlock that we did not spot. So make sure this doesn't happen, we'll
have plenty of time in the next release cycle to stress test it.

ok visa@

Wrap the NET_LOCK() into a per-socket solock() that does nothing for
unix domain sockets.

This should prevent the multiple deadlock related to unix domain sockets.

Inputs from millert@ and bluhm@, ok bluhm@

Enable the NET_LOCK(), take 2.

Recursions are currently known and marked a XXXSMP.

Please report any assert to bugs@

In preparation of compiling our kernels with -ffreestanding, explicitly map
a few performance-critical functions to compiler builtins. Since the
builtins supported by gcc3, gcc4 and clang are not the same, there are
(unfortunately) some compiler checks to make sure we only do the mapping
for builtins that are actually supported by the compiler.

ok jca@, tom@, guenther@

Change NET_LOCK()/NET_UNLOCK() to be simple wrappers around
splsoftnet()/splx() until the known issues are fixed.

In other words, stop using a rwlock since it creates a deadlock when
chrome is used.

Issue reported by Dimitris Papastamos and kettenis@

ok visa@

Introduce the NET_LOCK() a rwlock used to serialize accesses to the parts
of the network stack that are not yet ready to be executed in parallel or
where new sleeping points are not possible.

This first pass replace all the entry points leading to ip_output(). This
is done to not introduce new sleeping points when trying to acquire ART's
write lock, needed when a new L2 entry is created via the RT_RESOLVE.

Inputs from and ok bluhm@, ok dlg@

introduce hashfree() function to free hash tables, with sizes.
ok guenther

garbage collect dead prototype

ok kettenis@ mpi@

Introduce rwsleep(9), an equivalent to msleep(9) but for code protected
by a write lock.

ok guenther@, vgross@

Introduce Dynamic Profiling, a ddb(4) based & gprof compatible kernel
profiling framework.

Code patching is used to enable probes when entering functions. The
probes will call a mcount()-like function to match the behavior of a
GPROF kernel.

Currently only available on amd64 and guarded under DDBPROF. Support
for other archs will follow soon.

A new sysctl knob, ddb.console, need to be set to 1 in securelevel 0
to be able to use this feature.

Inputs and ok guenther@

Write the system time back to the RTC every 30 minutes.
This fixes the problem that long-running machines which were not
shut down properly would reboot with a badly offset system time.

hints and ok kettenis@

MPSAFE is never used, so get rid of it.

OK natano@ mpi@ guenther@

Backout the previous fix for the sendsyslog(2) with LOG_CONS solution.
Permanently holding /dev/console open in the kernel works only until
init(8) calls revoke(2). After that the console device vnode cannot
be used anymore. It still resulted in a hanging init(8) if it tried
to syslog(3) something. With the backout also dmesg -s works again.

If sendsyslog(2) is called with LOG_CONS before syslogd(8) has been
started and before init(8) has opened the console, the kernel could
crash as the console device has not been initialized. Open
/dev/console in the kernel before starting init(8) and keep it open.
This way sendsyslog(2) can be called early in the system.
OK beck@ deraadt@

Remove unused ``curpriority'' define.

Its description might be confusing, it was the pre-SMP parent of what
is now ``spc_curpriority'' which reflects the ``p_usrpri'' of curproc.

Remove now unused legacy uiomovei() function.

All its callers got reviewed and converted to
use uiomove() properly.

ok deraadt@

Replace mountroothook_establish(9) by config_mountroot(9) a narrower API
similar to config_defer(9).

ok mikeb@, deraadt@

Move hzto(9) to the attic; OK dlg

First step towards making uiomove() take a size_t size argument:
- rename uiomove() to uiomovei() and update all its users.
- introduce uiomove(), which is similar to uiomovei() but with a size_t.
- rewrite uiomovei() as an uiomove() wrapper.
ok kettenis@

retire shutdown hooks; ok deraadt, krw

Convert watchdog(4) devices to use autoconf(9) framework.

ok deraadt, tests on glxpcib and ok mpi

Add __attribute__((__bounded__)) to arc4random_buf().
ok deraadt@ tedu@

move arc4random prototype to systm.h. more appropriate for most code
to include that than rdnvar.h. ok deraadt dlg

remove LKM support

KERNEL_ASSERT_LOCKED(9): Assertion for kernel lock (Rev. 3)

This adds a new assertion macro, KERNEL_ASSERT_LOCKED(), to assert that
kernel_lock is held. In the long process of removing kernel_lock, there will
be a lot (hundreds or thousands) of use of this; virtually almost all functions
in !MP-safe subsystems should have this assertion. Thus this assertion should
have a short, good name.

Not only that "KERNEL_ASSERT_LOCKED" is consistent with other KERNEL_* and
SCHED_ASSERT_LOCKED() macros.

Input from dlg@ guenther@ kettenis@.

OK dlg@ guenther@

Replace all ovbcopy with memmove; swap the src and dst arguments too
ok otto

Add tstohz(9) as the timespec analog to tvtohz(9).

ok miod

shuffle around some poison code, prototypes, values...
allow some more pool debug code to be enabled if not compiled in
bump poison size back up to 64

rthreads are always enabled. remove the sysctl.
ok deraadt guenther kettenis matthew

separate memory poisoning code to a new file and make it usable kernel wide
ok deraadt

Add explicit __attribute__ ((__format__(__kprintf__)))) to the functions and
function pointer arguments which are {used as,} wrappers around the kernel
printf function.
No functional change.

Swap arguments to wdog_register() since it is nicer, and prepare
wdog_shutdown() for external usage.

Explicitly annotate setjmp() and longjmp() (and friends) as
__returns_twice and __dead instead of depending on GCC's special
handling of these function names.

With input from kettenis@ and guenther@
Fixes a warning from clang
ok matthew@

Move the common bits of syscall invocation and return handling into
an MI file, <sys/syscall_mi.h>, correcting inconsistencies and the
handling when copyin() of arguments fails.

Tested on i386, amd64, sparc64, and alpha (thanks naddy@)
Any issues with other platforms will be fixed in tree.

header name from millert@; ok miod@

Apply profiling to all threads instead of just the thread that called
profil() by moving P_PROFIL from proc->p_flag to process->ps_flags with
matching adjustment in fork1() and exit1()

ok matthew@

Switch back to bootduid, however remember to include sys/systm.h...

Clean up after P_BIGLOCK removal.
KERNEL_PROC_LOCK -> KERNEL_LOCK
KERNEL_PROC_UNLOCK -> KERNEL_UNLOCK

oga@ ok

Allow the root device to be identified via its disklabel UID.

ok deraadt@ marco@ krw@

add a new function, explicit_bzero, to be used for erasing "secret" stuff.
unlike normal bzero, we guarantee that the compiler will not optimize out
calls to this function for otherwise dead variables.
to be adjusted as needed when compilers and linkers get smarter.
ok deraadt miod

Add assertwaitok(9) to declare code paths that assume they can sleep.
Currently only checks that we're not in an interrupt context, but will
soon check that we're not holding any mutexes either.

Update malloc(9) and pool(9) to use assertwaitok(9) as appropriate.

"i like it" art@, oga@, marco@; "i see no harm" deraadt@; too trivial
for me to bother prying actual oks from people.

remove the powerhook code. All architectures now use the ca_activate tree
traversal code to suspend/resume
ok oga kettenis blambert

All PWR_{SUSPEND,RESUME} can now be replaced by DVACT_{SUSPEND,RESUME}

kill PWR_STANDBY (apm can use PWR_SUSPEND instead). While here, renumber
PWR_{SUSPEND,RESUME} so that they match the values of DAVCT_{SUSPEND,RESUME}
so that we can eventually (many more steps...) kill the powerhook garbage
and use the activate mechanism.
no objections

Change hzto(9) and tvtohz(9) arguments to const pointers.

ok krw@, "of course" tedu@

Devices which don't have read or write functionality should not return
enodev to poll, because this returns an errno of 19 in revents. Oops.
Use seltrue where needed, and use a new selfalse function for those which
don't know if the next op will be non-blocking
Mostly discussed with guenther and miod

Eliminate RTHREADS kernel option in favor of a sysctl. The actual status
(not done) hasn't changed, but now it's less work to test things.
ok art deraadt

remove proc.h include from uvm_map.h. This has far reaching effects, as
sysctl.h was reliant on this particular include, and many drivers included
sysctl.h unnecessarily. remove sysctl.h or add proc.h as needed.
ok deraadt

move some of proc.h's greatest hits to systm.h, speeding up compiles.
lots of build testing by deraadt, ok/feedback deraadt guenther kettenis

Get rid of __HAVE_GENERIC_SOFT_INTERRUPTS now that all our platforms support it.

ok jsing@, miod@

Count number of cpus found (potentially not attached) and store that
in sysctl hw.ncpufound; ok miod kettenis

queue the mountroot hooks to be run in the same order

merge vfs_opv_init into vfs_op_init and remove the former,
as they where called consecutively in vfs_init.

Add possibility to add flags to syscalls in syscalls.master to mark
syscalls as NOLOCK and MPSAFE. The flags have slightly different semantics:
NOLOCK - the syscall doesn't grab any locks whatsoever.
MPSAFE - the syscall deals with its own locking.

What this means in practice is that NOLOCK syscalls can always be done
without the biglock. The MPSAFE syscalls can be done without the biglock
on CPUs that don't handle interrupts that require biglock (to preserve
lock ordering).

deraadt@ ok

some architectures called setroot() from cpu_configure(), *way* before some
subsystems were enabled. others used a *md_diskconf -> diskconf() method to
make sure init_main could "do late setroot". Change all architectures to
have diskconf(), use it directly & late. tested by todd and myself on most
architectures, ok miod too

Don't use LK_CANRECURSE for the kernel lock, okay miod@ art@

typos; from bret lambert

use the underscore variants of _BYTE_ORDER which are always defined
even when various "strict" compiler options are used
ok deraadt millert

Remove unused _{ins,rem}que functions - they were not even implemented on
all architectures.

convert _FOO_SOURCE -> __FOO_VISIBLE in machine. OK deraadt@

bounds checking for copystr, copyin and copyinstr;
tested by moritz@ otto@ deraadt@, ok deraadt@

mountroothooks are called after the root filesystem is mounted.

We don't have vsprintf/sprintf in the kernel anymore, spotted
by form@pdp-11.org.ru.

ok millert@ deraadt@

boundary-check memcpy and friends. henning ok

debranch SMP, have fun

pull ncpus support from smp tree into main branch.
remove alpha specific definition of ncpus.
OK (and tested on alpha) deraadt@

unobfuscate systm.h: use va_list for vprintf.
_BSD_VA_LIST_ explained by millert@, okay drahn@

put an mi wrapper around stdarg.h/varargs.h. gcc3 moved stdarg/varargs macros
to built-ins, so eventually we will have one version of these files.
Special adjustments for the kernel to cope: machine/stdarg.h -> sys/stdarg.h
and machine/ansi.h needs to have a _BSD_VA_LIST_ for syslog* prototypes.
okay millert@, drahn@, miod@.

sprinkle some __kprintf__ attributes around functions which use format
strings in the kernel to make gcc aware of the extra modifiers
deraadt@ ok

remove caddr_t casts. it's just silly to cast something when the function
takes a void *. convert uiomove to take a void * as well. ok deraadt@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

Match vprintf prototype to userland and standards.

deraadt@ ok

add kern.watchdog sysctl and generic watchdog interface;
based on feedback and discussions with mickey, henric, fgsch and jakob.
ok art@, mickey@, jakob@, henric@

Remove fetch(9) and store(9) functions from the kernel, and replace the few
remaining instances of them with appropriate copy(9) usage.

ok art@, tested on all arches unless my memory is non-ECC

- Add a flags argument to dohooks.
The flag can be either HOOK_REMOVE or HOOK_REMOVE|HOOK_FREE.
o HOOK_REMOVE removes the hook from the list before executing it.
o HOOK_FREE frees the hook after that.

- Let dostartuphooks use HOOK_REMOVE|HOOK_FREE so we can reclaim the memory.

- Let doshutdownhooks use HOOK_REMOVE so that when some shutdown hook
panics (they do that all the #@$%! time these days) we don't loop
for ever. Don't HOOK_FREE, it doesn't matter and I don't want to add
another possible panic condition for shutdown hooks.

- Actually free the pointer we're throwing away in hook_disestablish (I wonder
how much memory this has leaked over the years).

Remove kernel support for NTP. ok deraadt@ and tholo@

Implement splassert() for sparc - a tool for finding problems related to
spl handling (already found 3 problems).

Man page in a few seconds.
deraadt@ ok.

remove ambiguity in version,ostype,osversion,osrelease and their constanity, they are and declarre 'em accordingly also removing private externies of those

Final __P removal plus some cosmetic fixups

First round of __P removal in sys

Add a tvtohz function. Like hzto, but doesn't subtract the current time.

Cleanup mountroot-related definitions.

branches: 1.44.2;
Let fork1, uvm_fork, and cpu_fork take a function/argument pair as argument,
instead of doing fork1, cpu_set_kpc. This lets us retire cpu_set_kpc and
avoid a multiprocessor race.

This commit breaks vax because it doesn't look like any other arch, someone
working on vax might want to look at this and try to adapt the code to be
more like the rest of the world.

Idea and uvm parts from NetBSD.

be and le varients of syscallarg; from netbsd

Remove the old timeout legacy code.

Startup hooks. Can be used for providing root/swap devices from device
systems which want configuration to finish late, like I2O. Implemented via
a general hooks mechanism which the shutdown hooks have been converted to
use as well. It even has manpages!

kill old vm

place extern cold here; per discussion w/ art@

Rename configure() to cpu_configure().
Move it from cpu_startup() to main().

branches: 1.37.2;
(sy_call_t): define a type for the functions in sysent
PR 1032

snprintf in kernel; assar@stacken.kth.se

This shouldn't have been committed with the previous commit, revert
(experimental code)

Merge dvdio.h and cdio.h, don't use typedefs, get rid of bitfields (no
good reason to use them, not packed structures anyway).

add APM powerhooks.
from NetBSD, Sat Jun 26 08:25:25 1999 UTC by augustss:

Add powerhooks, i.e., the ability to register a function that will be
called when the machine does a suspend or resume.
XXX Will go away when Jason's kevents come to life.

Fix rootdev handling, use disk checksums to find the device we were booted
from. Hopefully this will fix all the hangs/panics where the root device
was not found.

proto mem*() functions

fix some typos; kwesterback@home.com

add scdebug_{call,ret} to help SYSCALL_DEBUG compile.
remove nsysent extern declaration, since it's no longer defined anywhere,
and SYS_MAXSYSCALL is used everywhere instead.
niklas@ -- ok

zap the newhashinit hack.
Add an extra flag to hashinit telling if it should wait in malloc.
update all calls to hashinit.

uvm doesn't use nswdev or nswmap.
add prototype for kcopy (uvm)

Add newhashinit(), which is identical to hashinit() except it takes a flags
arg for passing to malloc() (hashinit always uses M_WAITOK which is not
always what you want). Everything that uses hashinit should really
get converted to newhashinit and then newhashinit can be renamed.

Generalize cpu_set_kpc to take any kind of arg; mostly from NetBSD

Updates for VFS Lite 2 + soft update.

add prototype for vprintf

back out vfs lite2 till after 2.2

VFS Lite2 Changes

Prototype hardpps() if PPS_SYNC option is present

protect from multiple includes (required by gpl_math_emulate)

Debugger() is needed by KGDB not just DDB

-Wcast-qual happiness

back out bitmask_snprintf()

Added bitmap_snprintf proto

export vfs_opv_init*

proto mountroot and friends

-Wall happiness, especially for sparc/stand

-Wall happiness (especially for sparc)

__assert added, impl from netbsd, however put elsewhere. use it instead
of private versions (one even using the userland header) in if_sn.c

-Wall, -Wstrict-prototypes and some KNF cleanup

make printf/addlog return 0, for compat to userland

-Wall & -Wstrict-prototype fixes

Add prototype for hardupdate() ifdef NTP.

proto more stuff

partial sync with netbsd 960418, more to come

Merge of NetBSD 960317

From NetBSD: Merge with NetBSD 960217

branches: 1.1.1;
Initial revision

Adjust KERNEL_ASSERT_UNLOCKED() to not assert during a panic.

KERNEL_ASSERT_UNLOCKED calls _kernel_lock_held() which returns true
if panicstr || db_active which triggers this assert. Workaround this by
checking them before.

This will alter the following Syzkaller reports:
Reported-by: syzbot+169110a0815838ab5940@syzkaller.appspotmail.com
Reported-by: syzbot+3c2eced405b9de6f79c2@syzkaller.appspotmail.com

OK mpi@

clockintr: move callback-specific API behaviors to "clockrequest" namespace

The API's behavior when invoked from a callback function is impossible
to document. Move the special behavior into a distinct namespace,
"clockrequest".

- Add a 'struct clockrequest'. Basically a stripped-down 'struct clockintr'
for exclusive use during clockintr_dispatch().
- In clockintr_queue, replace the "cq_shadow" clockintr with a "cq_request"
clockrequest. They serve the same purpose.
- CLST_SHADOW_PENDING -> CR_RESCHEDULE; different namespace, same meaning.
- CLST_IGNORE_SHADOW -> CLST_IGNORE_REQUEST; same meaning.
- Move shadow branch in clockintr_advance() to clockrequest_advance().
- clockintr_request_random() becomes clockrequest_advance_random().
- Delete dead shadow branches in clockintr_cancel(), clockintr_schedule().
- Callback functions now get a clockrequest pointer instead of a special
clockintr pointer: update all prototypes, callers.

No functional change intended.

kernel: expand fixed clock interrupt periods to 64-bit values

Technically, all the current fixed clock interrupt periods fit within
an unsigned 32-bit value. But 32-bit multiplication is an accident
waiting to happen. So, expand the fixed periods for hardclock,
statclock, profclock, and roundrobin to 64-bit values.

One exception: statclock_mask remains 32-bit because random(9) yields
32-bit values. Update the initclocks() comment to make it clear that
this is not an accident.

clockintr, statclock: eliminate clockintr_statclock() wrapper

- Move remaining statclock variables from kern_clockintr.c to
kern_clock.c. Move statclock variable initialization from
clockintr_init() into initclocks().

- Change statclock() prototype to make it a legal clockintr
callback function and establish the handle with statclock()
instead clockintr_statclock().

- Merge the contents of clockintr_statclock() into statclock().
statclock() can now reschedule itself and handles multiple
expirations transparently.

- Make statclock_avg visible from sys/systm.h so that clockintr_cpu_init()
can use it to advance the statclock across suspend/hibernate.

Thread: https://marc.info/?l=openbsd-tech&m=169428749720476&w=2

clockintr: replace CL_RNDSTAT with global variable statclock_is_randomized

In order to separate the statclock from the clock interrupt subsystem
we need to move all statclock state out into the broader kernel.

Start by replacing the CL_RNDSTAT flag with a new global variable,
"statclock_is_randomized", in kern_clock.c. Update all clockintr_init()
callers to set the boolean instead of passing the flag.

Thread: https://marc.info/?l=openbsd-tech&m=169428749720476&w=2

all platforms: separate cpu_initclocks() from cpu_startclock()

To give the primary CPU an opportunity to perform clock interrupt
preparation in a machine-independent manner we need to separate the
"initialization" parts of cpu_initclocks() from the "start the clock
interrupt" parts. Currently, cpu_initclocks() does everything all at
once, so there is no space for this MI setup.

Many platforms have more-or-less already done this separation by
implementing a separate routine named "cpu_startclock()". This patch
promotes cpu_startclock() from de facto standard to mandatory API.

- Prototype cpu_startclock() in sys/systm.h alongside cpu_initclocks().
The separation of responsibility between the two routines is a bit
fuzzy but the basic guidelines are as follows:

+ cpu_initclocks() must initialize hz, stathz, and profhz, and call
clockintr_init().

+ cpu_startclock() must call clockintr_cpu_init() and start the clock
interrupt cycle on the calling CPU.

These guidelines will shift in the future, but that's the way things
stand as of *this* commit.

- In initclocks(): first call cpu_initclocks(), then do MI setup, and
last call cpu_startclock().

- On platforms where cpu_startclock() already exists: don't call
cpu_startclock() from cpu_initclocks() anymore.

- On platforms where cpu_startclock() doesn't yet exist: implement it.
Usually this is as simple as dividing cpu_initclocks() in two.

Tested on amd64 (i8254, lapic), arm64, i386 (i8254, lapic), macppc,
mips64/octeon, and sparc64. Tested on arm/armv7 (agtimer(4)) by
phessler@ and jmatthew@. Tested on m88k/luna88k by aoyama@. Tested
on powerpc64 by gkoehler@ and mlarkin@. Tested on riscv64 by
jmatthew@.

Thread: https://marc.info/?l=openbsd-tech&m=169195251322149&w=2

hardclock(9): move setitimer(2) code into itimer_update()

- Move the setitimer(2) code responsible for updating the ITIMER_VIRTUAL
and ITIMER_PROF timers from hardclock(9) into a new clock interrupt
routine, itimer_update(). itimer_update() is periodic and runs at the
same frequency as the hardclock.

+ Revise itimerdecr() to run within itimer_mtx instead of entering
and leaving it.

- Each schedstate_percpu has its own itimer_update() handle, spc_itimer.
A new scheduler flag, SPCF_ITIMER, indicates whether spc_itimer was
started during the last mi_switch() and needs to be stopped during the
next mi_switch() or sched_exit().

- A new per-process flag, PS_ITIMER, indicates whether ITIMER_VIRTUAL
and/or ITIMER_PROF are running. Checking the flag is easier than
entering itimer_mtx to check process.ps_timer[]. The flag is set
and cleared in a new helper function, process_reset_itimer_flag().

- In setitimer(), call need_resched() when the state of ITIMER_VIRTUAL
or ITIMER_PROF is changed to force an mi_switch() and update
spc_itimer.

claudio@ notes that ITIMER_PROF could be implemented as a high-res
timer using the thread's execution time as a guide for when to
interrupt the process and assert SIGPROF. This would probably work
really well in single-threaded processes. ITIMER_VIRTUAL would be
more difficult to make high-res, though, as you need to exclude time
spent in the kernel.

Tested on powerpc64 by gkoehler@. With input from claudio@.

Thread: https://marc.info/?l=openbsd-tech&m=169038818517101&w=2

ok claudio@

struct sleep_state is no longer used, remove it.
Also remove the priority argument to sleep_finish() the code can use
the p_flag P_SINTR flag to know if the signal check is needed or not.
OK cheloha@ kettenis@ mpi@

First step at removing struct sleep_state.

Pass the timeout and sleep priority not only to sleep_setup() but also
to sleep_finish(). With that sls_timeout and sls_catch can be removed
from struct sleep_state.

The timeout is now setup first thing in sleep_finish() and no longer as
last thing in sleep_setup(). This should not cause a noticeable difference
since the code run between sleep_setup() and sleep_finish() is minimal.

OK kettenis@

On systems without xonly mmu hardware-enforcement, we can still mitigate
against classic BROP with a range-checking wrapper in front of copyin() and
copyinstr() which ensures the userland source doesn't overlap the main program
text, ld.so text, signal tramp text (it's mapping is hard to distinguish
so it comes along for the ride), or libc.so text. ld.so tells the kernel
libc.so text range with msyscall(2). The range checking for 2-4 elements is
done without locking (because all 4 ranges are immutable!) and is inexpensive.

write(sock, &open, 400) now fails with EFAULT. No programs have been
discovered which require reading their own text segments with a system call.

On a machine without mmu enforcement, a test program reports the following:
userland kernel
ld.so readable unreadable
mmap xz unreadable unreadable
mmap x readable readable
mmap nrx readable readable
mmap nwx readable readable
mmap xnwx readable readable
main readable unreadable
libc unmapped? readable unreadable
libc mapped readable unreadable

ok kettenis, additional help from miod

Remove copystr(9), unless used internally by copy{in,out}str.

Allow suspend with root on sdmmc(4).

ok deraadt@

Clean up the netlock macros. Merge NET_RLOCK_IN_SOFTNET and
NET_RLOCK_IN_IOCTL, which have the same implementation. The R and
W are hard to see, call the new macro NET_LOCK_SHARED. Rename the
opposite assertion from NET_ASSERT_WLOCKED to NET_ASSERT_LOCKED_EXCLUSIVE.
Update some outdated comments about net locking.
OK mpi@ mvs@

Add db_rint(), an MI interface to db_enter() copied from kdbrint() in vax code

If ddb.console is set and your serial console driver uses it, db_rint(),
lets you enter ddb(4) by typing the ESC D escape sequence. This is
useful for drivers like sfuart(4) where the hardware doesn't have a true
BREAK mechanism.

Suggested by miod@, ok kettenis@ miod@

Remove old poll/select wakeup mechanism.

Also remove unneeded seltrue() and selfalse().

OK mpi@ jsg@

We only have one syscall table: inline sysent/SYS_MAXSYSCALL and
SYS_syscall as the nosys() function into the MD syscall entry
routines and the SYSCALL_DEBUG support. Adjust alpha's syscall
check to match the other archs. Also, make sysent const to get it
into .rodata.

With that, 'struct emul' is unused: delete it and all its references

ok millert@

kernel: introduce per-CPU panic(9) message buffers

Add a 512-byte buffer (ci_panicbuf) to each cpu_info struct on each
platform for use by panic(9). The first panic on a given CPU writes
its message to this buffer. Subsequent panics on a given CPU print
the panic message to the console but do not modify the buffer. This
aids debugging in two cases:

- If 2+ CPUs panic simultaneously there is no risk of garbled messages
in the panic buffer.

- If a CPU panics and then the operator causes a second panic while
using ddb(4), the operator can still recall the first failure on
a particular CPU.

Misc. changes to support this bigger change:

- Set panicstr atomically to identify the first CPU to reach panic().

- Tweak db_show_panic_cmd() to print all panic messages across all
CPUs. Prefix the first panic with an asterisk ('*').

- Prefer db_printf() to printf() during a panic if we have it.
Apparently it disturbs less global state.

- On amd64, tweak fault() to write the local panic buffer. This needs
more work.

Prompted by bluhm@ and deraadt@. Mostly written by deraadt@.
Discussed with bluhm@, deraadt@ and kettenis@.

Borne from a discussion on tech@ about making panic(9) more MP-safe:

https://marc.info/?l=openbsd-tech&m=162086462316143&w=2

ok kettenis@, visa@, bluhm@, deraadt@

time to add NET_ASSERT_WLOCKED()

with moving towards NET_RLOCK...() we need NET_ASSERT_WLOCKED()
to check caller owns netlock exclusively.

OK @bluhm

Simplify sleep_setup API to two operations in preparation for splitting
the SCHED_LOCK().

Putting a thread on a sleep queue is reduce to the following:

sleep_setup();
/* check condition or release lock */
sleep_finish();

Previous version ok cheloha@, jmatthew@, ok claudio@

Remove obsolete vnode operation vector declarations.

OK bluhm@, claudio@, mpi@, semarie@

Make NET_LOCK() assertions conditional to DIAGNOSTIC

This saves about 2.5 KiB off amd64's RAMDISK after gzip compression.

OK deraadt@, mpi@, cheloha@

tsleep(9): add global "nowake" channel for threads avoiding wakeup(9)

It would be convenient if there were a channel a thread could sleep on
to indicate they do not want any wakeup(9) broadcasts. The easiest way
to do this is to add an "int nowake" to kern_synch.c and extern it in
sys/systm.h. You use it like this:

#include <sys/systm.h>

tsleep_nsec(&nowait, ...);

There is now no need to handroll a local dead channel, e.g.

int chan;

tsleep_nsec(&chan, ...);

which expands the stack. Local dead channels will be replaced with
&nowake in later patches.

One possible problem with this "one global channel" approach is sleep
queue congestion. If you have lots of threads sleeping on &nowake you
might slow down a wakeup(9) on a different channel that hashes into
the same queue. Unsure how much of problem this actually is, if at all.

NetBSD and FreeBSD have a "pause" interface in the kernel that chooses
a suitable channel automatically. To keep things simple and avoid
adding a new interface we will start with this global channel.

Discussed with mpi@, claudio@, kettenis@, and deraadt@.

Basically designed by kettenis@, who vetoed my other proposals.

Bugs caught by deraadt@, tb@, and patrick@.

Declare hw_{prod,serial,uuid,vendor,ver} in <sys/systm.h>.

OK deraadt@, mpi@

dev/rndvar.h no longer has statistical interfaces (removed during various
conversion steps). it only contains kernel prototypes for 4 interfaces,
all of which legitimately belong in sys/systm.h, which are already included
by all enqueue_randomness() users.

Document the various flavors of NET_LOCK() and rename the reader version.

Since our last concurrency mistake only ioctl(2) ans sysctl(2) code path
take the reader lock. This is mostly for documentation purpose as long as
the softnet thread is converted back to use a read lock.

dlg@ said that comments should be good enough.

ok sashan@

tsleep_nsec(9): add MAXTSLP macro, the maximum sleep duration

This macro will be useful for truncating durations below INFSLP
(UINT64_MAX) when converting from a timespec or timeval to a count
of nanoseconds before calling tsleep_nsec(9), msleep_nsec(9), or
rwsleep_nsec(9).

A relative timespec can hold many more nanoseconds than a uint64_t
can. TIMESPEC_TO_NSEC() and TIMEVAL_TO_NSEC() check for overflow,
returning UINT64_MAX if the conversion would overflow a uint64_t.

Thus, MAXTSLP will make it easy to avoid inadvertently passing INFSLP
to tsleep_nsec(9) et al. when the caller intended to set a timeout.

The code in such a case might look something like this:

uint64_t nsecs = MIN(TIMESPEC_TO_NSEC(&ts), MAXTSLP);

The macro may also be useful for rejecting intervals that are "too large",
e.g. for sockets with timeouts, if the timeout duration is to be stored
as a uint64_t in an object in the kernel. The code in such a case might
look something like this:

case SIOCTIMEOUT:
{
struct timeval *tv = (struct timeval *)data;
uint64_t nsecs;

if (tv->tv_sec < 0 || !timerisvalid(tv))
return EINVAL;

nsecs = TIMEVAL_TO_NSEC(tv);
if (nsecs > MAXTSLP)
return EOVERFLOW;

obj.timeout = nsecs;
break;
}

Idea suggested by visa@.

ok visa@

Move kernel locking inside the sleep machinery. This enables calling
rwsleep(9) with PCATCH and rw_enter(9) with RW_INTR without the kernel
lock. In addition, now tsleep(9) with PCATCH should be safe to use
without the kernel lock if the sleep is purely time-based.

Tested by anton@, cheloha@, chris@
OK anton@, cheloha@

softclock: move softintr registration/scheduling into timeout module

softclock() is scheduled from hardclock(9) because long ago callouts were
processed from hardclock(9) directly. The introduction of timeout(9) circa
2000 moved all callout processing into a dedicated module, but the softclock
scheduling stayed behind in hardclock(9).

We can move all the softclock() "stuff" into the timeout module to make
kern_clock.c a bit cleaner. Neither initclocks() nor hardclock(9) need
to "know" about softclock(). The initial softclock() softintr registration
can be done from timeout_proc_init() and softclock() can be scheduled
from timeout_hardclock_update().

ok visa@

Add tsleep_nsec(9), msleep_nsec(9), and rwsleep_nsec(9).

Equivalent to their unsuffixed counterparts except that (a) they take
a timeout in terms of nanoseconds, and (b) INFSLP, aka UINT64_MAX (not
zero) indicates that a timeout should not be set.

For now, zero nanoseconds is not a strictly valid invocation: we log a
warning on DIAGNOSTIC kernels if we see such a call. We still sleep
until the next tick in such a case, however. In the future this could
become some sort of poll... TBD.

To facilitate conversions to these interfaces: add inline conversion
functions to sys/time.h for turning your timeout into nanoseconds.

Also do a few easy conversions for warmup and to demonstrate how
further conversions should be done.

Lots of input from mpi@ and ratchov@. Additional input from tedu@,
deraadt@, mortimer@, millert@, and claudio@.

Partly inspired by FreeBSD r247787.

positive feedback from deraadt@, ok mpi@

Remove file name and line number output from witness(4)

Reduce code clutter by removing the file name and line number output
from witness(4). Typically it is easy enough to locate offending locks
using the stack traces that are shown in lock order conflict reports.
Tricky cases can be tracked using sysctl kern.witness.locktrace=1 .

This patch additionally removes the witness(4) wrapper for mutexes.
Now each mutex implementation has to invoke the WITNESS_*() macros
in order to utilize the checker.

Discussed with and OK dlg@, OK mpi@

Add sleep_finish_all(), which provides the common combo of sleep_finish(),
sleep_finish_timeout(), and sleep_finish_signal() with error preferencing,
and then use it in five places.

ok mpi@

Do not panic from ddb(4) when a lock requirement isn't fulfilled.

Extend the logic already present for panic() to any DDB-related
operation such that if ddb(4) is entered because of a fault or
other trap it is still possible to call 'boot reboot'.

While here stop printing splassert() messages as well, to not fill
the buffer.

ok visa@, deraadt@

Use a temporary chacha instance to fill large randomdata sections. Avoids
grabbing the rnglock repeatedly.

ok deraadt@ djm@

Show uvm_fault and trace when typing show panic on a page fault'd kernel

Currently there is only support for amd64, if this change settles
I will add support for the rest of the architectures.

OK kettenis@.

add code to provide simple wait condition handling.

this will be used to replace the bare sleep_state handling in a
bunch of places, starting with the barriers.

Do not call splassert_fail() if splassert_ctl is <= 0.

This matches splassert(9)s behavior and prevent noise when a CPU
panic(9) and set splassert_ctl to 0.

Found the hardway by sthen@

Introduce a reader version of the NET_LOCK().

This will be used to first allow read-only ioctl(2) to be executed while
the softnet taskq is running. Then it will allows us to execute multiple
softnet taskq in parallel.

Tested by Hrvoje Popovski, ok kettenis@, sashan@, visa@, tb@

Remove NET_LOCK()'s argument.

Tested by Hrvoje Popovski, ok bluhm@

Stop doing an splsoftnet()/splx() dance inside the NET_LOCK().

This will allow us to not carry a returned value when entering a critical
section.

ok bluhm@, visa@

clang has builtin_memmove. ok deraadt

Add copyin32(9) prototype.

Enable the NET_LOCK(), take 3.

Recursions are still marked as XXXSMP.

ok deraadt@, bluhm@

Rename Debugger() into db_enter().

Using a name with the 'db_' prefix makes it invisible from the dynamic
profiler.

ok deraadt@, kettenis@, visa@

Unifdef KGDB.

It doesn't compile und hasn't been working during the last decade.

ok kettenis@, deraadt@

Hook up mplock to witness(4) on amd64 and i386.

Revert the NET_LOCK() and bring back pf's contention lock for release.

For the moment the NET_LOCK() is always taken by threads running under
KERNEL_LOCK(). That means it doesn't buy us anything except a possible
deadlock that we did not spot. So make sure this doesn't happen, we'll
have plenty of time in the next release cycle to stress test it.

ok visa@

Wrap the NET_LOCK() into a per-socket solock() that does nothing for
unix domain sockets.

This should prevent the multiple deadlock related to unix domain sockets.

Inputs from millert@ and bluhm@, ok bluhm@

Enable the NET_LOCK(), take 2.

Recursions are currently known and marked a XXXSMP.

Please report any assert to bugs@

In preparation of compiling our kernels with -ffreestanding, explicitly map
a few performance-critical functions to compiler builtins. Since the
builtins supported by gcc3, gcc4 and clang are not the same, there are
(unfortunately) some compiler checks to make sure we only do the mapping
for builtins that are actually supported by the compiler.

ok jca@, tom@, guenther@

Change NET_LOCK()/NET_UNLOCK() to be simple wrappers around
splsoftnet()/splx() until the known issues are fixed.

In other words, stop using a rwlock since it creates a deadlock when
chrome is used.

Issue reported by Dimitris Papastamos and kettenis@

ok visa@

Introduce the NET_LOCK() a rwlock used to serialize accesses to the parts
of the network stack that are not yet ready to be executed in parallel or
where new sleeping points are not possible.

This first pass replace all the entry points leading to ip_output(). This
is done to not introduce new sleeping points when trying to acquire ART's
write lock, needed when a new L2 entry is created via the RT_RESOLVE.

Inputs from and ok bluhm@, ok dlg@

introduce hashfree() function to free hash tables, with sizes.
ok guenther

garbage collect dead prototype

ok kettenis@ mpi@

Introduce rwsleep(9), an equivalent to msleep(9) but for code protected
by a write lock.

ok guenther@, vgross@

Introduce Dynamic Profiling, a ddb(4) based & gprof compatible kernel
profiling framework.

Code patching is used to enable probes when entering functions. The
probes will call a mcount()-like function to match the behavior of a
GPROF kernel.

Currently only available on amd64 and guarded under DDBPROF. Support
for other archs will follow soon.

A new sysctl knob, ddb.console, need to be set to 1 in securelevel 0
to be able to use this feature.

Inputs and ok guenther@

Write the system time back to the RTC every 30 minutes.
This fixes the problem that long-running machines which were not
shut down properly would reboot with a badly offset system time.

hints and ok kettenis@

MPSAFE is never used, so get rid of it.

OK natano@ mpi@ guenther@

Backout the previous fix for the sendsyslog(2) with LOG_CONS solution.
Permanently holding /dev/console open in the kernel works only until
init(8) calls revoke(2). After that the console device vnode cannot
be used anymore. It still resulted in a hanging init(8) if it tried
to syslog(3) something. With the backout also dmesg -s works again.

If sendsyslog(2) is called with LOG_CONS before syslogd(8) has been
started and before init(8) has opened the console, the kernel could
crash as the console device has not been initialized. Open
/dev/console in the kernel before starting init(8) and keep it open.
This way sendsyslog(2) can be called early in the system.
OK beck@ deraadt@

Remove unused ``curpriority'' define.

Its description might be confusing, it was the pre-SMP parent of what
is now ``spc_curpriority'' which reflects the ``p_usrpri'' of curproc.

Remove now unused legacy uiomovei() function.

All its callers got reviewed and converted to
use uiomove() properly.

ok deraadt@

Replace mountroothook_establish(9) by config_mountroot(9) a narrower API
similar to config_defer(9).

ok mikeb@, deraadt@

Move hzto(9) to the attic; OK dlg

First step towards making uiomove() take a size_t size argument:
- rename uiomove() to uiomovei() and update all its users.
- introduce uiomove(), which is similar to uiomovei() but with a size_t.
- rewrite uiomovei() as an uiomove() wrapper.
ok kettenis@

retire shutdown hooks; ok deraadt, krw

Convert watchdog(4) devices to use autoconf(9) framework.

ok deraadt, tests on glxpcib and ok mpi

Add __attribute__((__bounded__)) to arc4random_buf().
ok deraadt@ tedu@

move arc4random prototype to systm.h. more appropriate for most code
to include that than rdnvar.h. ok deraadt dlg

remove LKM support

KERNEL_ASSERT_LOCKED(9): Assertion for kernel lock (Rev. 3)

This adds a new assertion macro, KERNEL_ASSERT_LOCKED(), to assert that
kernel_lock is held. In the long process of removing kernel_lock, there will
be a lot (hundreds or thousands) of use of this; virtually almost all functions
in !MP-safe subsystems should have this assertion. Thus this assertion should
have a short, good name.

Not only that "KERNEL_ASSERT_LOCKED" is consistent with other KERNEL_* and
SCHED_ASSERT_LOCKED() macros.

Input from dlg@ guenther@ kettenis@.

OK dlg@ guenther@

Replace all ovbcopy with memmove; swap the src and dst arguments too
ok otto

Add tstohz(9) as the timespec analog to tvtohz(9).

ok miod

shuffle around some poison code, prototypes, values...
allow some more pool debug code to be enabled if not compiled in
bump poison size back up to 64

rthreads are always enabled. remove the sysctl.
ok deraadt guenther kettenis matthew

separate memory poisoning code to a new file and make it usable kernel wide
ok deraadt

Add explicit __attribute__ ((__format__(__kprintf__)))) to the functions and
function pointer arguments which are {used as,} wrappers around the kernel
printf function.
No functional change.

Swap arguments to wdog_register() since it is nicer, and prepare
wdog_shutdown() for external usage.

Explicitly annotate setjmp() and longjmp() (and friends) as
__returns_twice and __dead instead of depending on GCC's special
handling of these function names.

With input from kettenis@ and guenther@
Fixes a warning from clang
ok matthew@

Move the common bits of syscall invocation and return handling into
an MI file, <sys/syscall_mi.h>, correcting inconsistencies and the
handling when copyin() of arguments fails.

Tested on i386, amd64, sparc64, and alpha (thanks naddy@)
Any issues with other platforms will be fixed in tree.

header name from millert@; ok miod@

Apply profiling to all threads instead of just the thread that called
profil() by moving P_PROFIL from proc->p_flag to process->ps_flags with
matching adjustment in fork1() and exit1()

ok matthew@

Switch back to bootduid, however remember to include sys/systm.h...

Clean up after P_BIGLOCK removal.
KERNEL_PROC_LOCK -> KERNEL_LOCK
KERNEL_PROC_UNLOCK -> KERNEL_UNLOCK

oga@ ok

Allow the root device to be identified via its disklabel UID.

ok deraadt@ marco@ krw@

add a new function, explicit_bzero, to be used for erasing "secret" stuff.
unlike normal bzero, we guarantee that the compiler will not optimize out
calls to this function for otherwise dead variables.
to be adjusted as needed when compilers and linkers get smarter.
ok deraadt miod

Add assertwaitok(9) to declare code paths that assume they can sleep.
Currently only checks that we're not in an interrupt context, but will
soon check that we're not holding any mutexes either.

Update malloc(9) and pool(9) to use assertwaitok(9) as appropriate.

"i like it" art@, oga@, marco@; "i see no harm" deraadt@; too trivial
for me to bother prying actual oks from people.

remove the powerhook code. All architectures now use the ca_activate tree
traversal code to suspend/resume
ok oga kettenis blambert

All PWR_{SUSPEND,RESUME} can now be replaced by DVACT_{SUSPEND,RESUME}

kill PWR_STANDBY (apm can use PWR_SUSPEND instead). While here, renumber
PWR_{SUSPEND,RESUME} so that they match the values of DAVCT_{SUSPEND,RESUME}
so that we can eventually (many more steps...) kill the powerhook garbage
and use the activate mechanism.
no objections

Change hzto(9) and tvtohz(9) arguments to const pointers.

ok krw@, "of course" tedu@

Devices which don't have read or write functionality should not return
enodev to poll, because this returns an errno of 19 in revents. Oops.
Use seltrue where needed, and use a new selfalse function for those which
don't know if the next op will be non-blocking
Mostly discussed with guenther and miod

Eliminate RTHREADS kernel option in favor of a sysctl. The actual status
(not done) hasn't changed, but now it's less work to test things.
ok art deraadt

remove proc.h include from uvm_map.h. This has far reaching effects, as
sysctl.h was reliant on this particular include, and many drivers included
sysctl.h unnecessarily. remove sysctl.h or add proc.h as needed.
ok deraadt

move some of proc.h's greatest hits to systm.h, speeding up compiles.
lots of build testing by deraadt, ok/feedback deraadt guenther kettenis

Get rid of __HAVE_GENERIC_SOFT_INTERRUPTS now that all our platforms support it.

ok jsing@, miod@

Count number of cpus found (potentially not attached) and store that
in sysctl hw.ncpufound; ok miod kettenis

queue the mountroot hooks to be run in the same order

merge vfs_opv_init into vfs_op_init and remove the former,
as they where called consecutively in vfs_init.

Add possibility to add flags to syscalls in syscalls.master to mark
syscalls as NOLOCK and MPSAFE. The flags have slightly different semantics:
NOLOCK - the syscall doesn't grab any locks whatsoever.
MPSAFE - the syscall deals with its own locking.

What this means in practice is that NOLOCK syscalls can always be done
without the biglock. The MPSAFE syscalls can be done without the biglock
on CPUs that don't handle interrupts that require biglock (to preserve
lock ordering).

deraadt@ ok

some architectures called setroot() from cpu_configure(), *way* before some
subsystems were enabled. others used a *md_diskconf -> diskconf() method to
make sure init_main could "do late setroot". Change all architectures to
have diskconf(), use it directly & late. tested by todd and myself on most
architectures, ok miod too

Don't use LK_CANRECURSE for the kernel lock, okay miod@ art@

typos; from bret lambert

use the underscore variants of _BYTE_ORDER which are always defined
even when various "strict" compiler options are used
ok deraadt millert

Remove unused _{ins,rem}que functions - they were not even implemented on
all architectures.

convert _FOO_SOURCE -> __FOO_VISIBLE in machine. OK deraadt@

bounds checking for copystr, copyin and copyinstr;
tested by moritz@ otto@ deraadt@, ok deraadt@

mountroothooks are called after the root filesystem is mounted.

We don't have vsprintf/sprintf in the kernel anymore, spotted
by form@pdp-11.org.ru.

ok millert@ deraadt@

boundary-check memcpy and friends. henning ok

debranch SMP, have fun

pull ncpus support from smp tree into main branch.
remove alpha specific definition of ncpus.
OK (and tested on alpha) deraadt@

unobfuscate systm.h: use va_list for vprintf.
_BSD_VA_LIST_ explained by millert@, okay drahn@

put an mi wrapper around stdarg.h/varargs.h. gcc3 moved stdarg/varargs macros
to built-ins, so eventually we will have one version of these files.
Special adjustments for the kernel to cope: machine/stdarg.h -> sys/stdarg.h
and machine/ansi.h needs to have a _BSD_VA_LIST_ for syslog* prototypes.
okay millert@, drahn@, miod@.

sprinkle some __kprintf__ attributes around functions which use format
strings in the kernel to make gcc aware of the extra modifiers
deraadt@ ok

remove caddr_t casts. it's just silly to cast something when the function
takes a void *. convert uiomove to take a void * as well. ok deraadt@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

Match vprintf prototype to userland and standards.

deraadt@ ok

add kern.watchdog sysctl and generic watchdog interface;
based on feedback and discussions with mickey, henric, fgsch and jakob.
ok art@, mickey@, jakob@, henric@

Remove fetch(9) and store(9) functions from the kernel, and replace the few
remaining instances of them with appropriate copy(9) usage.

ok art@, tested on all arches unless my memory is non-ECC

- Add a flags argument to dohooks.
The flag can be either HOOK_REMOVE or HOOK_REMOVE|HOOK_FREE.
o HOOK_REMOVE removes the hook from the list before executing it.
o HOOK_FREE frees the hook after that.

- Let dostartuphooks use HOOK_REMOVE|HOOK_FREE so we can reclaim the memory.

- Let doshutdownhooks use HOOK_REMOVE so that when some shutdown hook
panics (they do that all the #@$%! time these days) we don't loop
for ever. Don't HOOK_FREE, it doesn't matter and I don't want to add
another possible panic condition for shutdown hooks.

- Actually free the pointer we're throwing away in hook_disestablish (I wonder
how much memory this has leaked over the years).

Remove kernel support for NTP. ok deraadt@ and tholo@

Implement splassert() for sparc - a tool for finding problems related to
spl handling (already found 3 problems).

Man page in a few seconds.
deraadt@ ok.

remove ambiguity in version,ostype,osversion,osrelease and their constanity, they are and declarre 'em accordingly also removing private externies of those

Final __P removal plus some cosmetic fixups

First round of __P removal in sys

Add a tvtohz function. Like hzto, but doesn't subtract the current time.

Cleanup mountroot-related definitions.

branches: 1.44.2;
Let fork1, uvm_fork, and cpu_fork take a function/argument pair as argument,
instead of doing fork1, cpu_set_kpc. This lets us retire cpu_set_kpc and
avoid a multiprocessor race.

This commit breaks vax because it doesn't look like any other arch, someone
working on vax might want to look at this and try to adapt the code to be
more like the rest of the world.

Idea and uvm parts from NetBSD.

be and le varients of syscallarg; from netbsd

Remove the old timeout legacy code.

Startup hooks. Can be used for providing root/swap devices from device
systems which want configuration to finish late, like I2O. Implemented via
a general hooks mechanism which the shutdown hooks have been converted to
use as well. It even has manpages!

kill old vm

place extern cold here; per discussion w/ art@

Rename configure() to cpu_configure().
Move it from cpu_startup() to main().

branches: 1.37.2;
(sy_call_t): define a type for the functions in sysent
PR 1032

snprintf in kernel; assar@stacken.kth.se

This shouldn't have been committed with the previous commit, revert
(experimental code)

Merge dvdio.h and cdio.h, don't use typedefs, get rid of bitfields (no
good reason to use them, not packed structures anyway).

add APM powerhooks.
from NetBSD, Sat Jun 26 08:25:25 1999 UTC by augustss:

Add powerhooks, i.e., the ability to register a function that will be
called when the machine does a suspend or resume.
XXX Will go away when Jason's kevents come to life.

Fix rootdev handling, use disk checksums to find the device we were booted
from. Hopefully this will fix all the hangs/panics where the root device
was not found.

proto mem*() functions

fix some typos; kwesterback@home.com

add scdebug_{call,ret} to help SYSCALL_DEBUG compile.
remove nsysent extern declaration, since it's no longer defined anywhere,
and SYS_MAXSYSCALL is used everywhere instead.
niklas@ -- ok

zap the newhashinit hack.
Add an extra flag to hashinit telling if it should wait in malloc.
update all calls to hashinit.

uvm doesn't use nswdev or nswmap.
add prototype for kcopy (uvm)

Add newhashinit(), which is identical to hashinit() except it takes a flags
arg for passing to malloc() (hashinit always uses M_WAITOK which is not
always what you want). Everything that uses hashinit should really
get converted to newhashinit and then newhashinit can be renamed.

Generalize cpu_set_kpc to take any kind of arg; mostly from NetBSD

Updates for VFS Lite 2 + soft update.

add prototype for vprintf

back out vfs lite2 till after 2.2

VFS Lite2 Changes

Prototype hardpps() if PPS_SYNC option is present

protect from multiple includes (required by gpl_math_emulate)

Debugger() is needed by KGDB not just DDB

-Wcast-qual happiness

back out bitmask_snprintf()

Added bitmap_snprintf proto

export vfs_opv_init*

proto mountroot and friends

-Wall happiness, especially for sparc/stand

-Wall happiness (especially for sparc)

__assert added, impl from netbsd, however put elsewhere. use it instead
of private versions (one even using the userland header) in if_sn.c

-Wall, -Wstrict-prototypes and some KNF cleanup

make printf/addlog return 0, for compat to userland

-Wall & -Wstrict-prototype fixes

Add prototype for hardupdate() ifdef NTP.

proto more stuff

partial sync with netbsd 960418, more to come

Merge of NetBSD 960317

From NetBSD: Merge with NetBSD 960217

branches: 1.1.1;
Initial revision

clockintr: move callback-specific API behaviors to "clockrequest" namespace

The API's behavior when invoked from a callback function is impossible
to document. Move the special behavior into a distinct namespace,
"clockrequest".

- Add a 'struct clockrequest'. Basically a stripped-down 'struct clockintr'
for exclusive use during clockintr_dispatch().
- In clockintr_queue, replace the "cq_shadow" clockintr with a "cq_request"
clockrequest. They serve the same purpose.
- CLST_SHADOW_PENDING -> CR_RESCHEDULE; different namespace, same meaning.
- CLST_IGNORE_SHADOW -> CLST_IGNORE_REQUEST; same meaning.
- Move shadow branch in clockintr_advance() to clockrequest_advance().
- clockintr_request_random() becomes clockrequest_advance_random().
- Delete dead shadow branches in clockintr_cancel(), clockintr_schedule().
- Callback functions now get a clockrequest pointer instead of a special
clockintr pointer: update all prototypes, callers.

No functional change intended.

kernel: expand fixed clock interrupt periods to 64-bit values

Technically, all the current fixed clock interrupt periods fit within
an unsigned 32-bit value. But 32-bit multiplication is an accident
waiting to happen. So, expand the fixed periods for hardclock,
statclock, profclock, and roundrobin to 64-bit values.

One exception: statclock_mask remains 32-bit because random(9) yields
32-bit values. Update the initclocks() comment to make it clear that
this is not an accident.

clockintr, statclock: eliminate clockintr_statclock() wrapper

- Move remaining statclock variables from kern_clockintr.c to
kern_clock.c. Move statclock variable initialization from
clockintr_init() into initclocks().

- Change statclock() prototype to make it a legal clockintr
callback function and establish the handle with statclock()
instead clockintr_statclock().

- Merge the contents of clockintr_statclock() into statclock().
statclock() can now reschedule itself and handles multiple
expirations transparently.

- Make statclock_avg visible from sys/systm.h so that clockintr_cpu_init()
can use it to advance the statclock across suspend/hibernate.

Thread: https://marc.info/?l=openbsd-tech&m=169428749720476&w=2

clockintr: replace CL_RNDSTAT with global variable statclock_is_randomized

In order to separate the statclock from the clock interrupt subsystem
we need to move all statclock state out into the broader kernel.

Start by replacing the CL_RNDSTAT flag with a new global variable,
"statclock_is_randomized", in kern_clock.c. Update all clockintr_init()
callers to set the boolean instead of passing the flag.

Thread: https://marc.info/?l=openbsd-tech&m=169428749720476&w=2

all platforms: separate cpu_initclocks() from cpu_startclock()

To give the primary CPU an opportunity to perform clock interrupt
preparation in a machine-independent manner we need to separate the
"initialization" parts of cpu_initclocks() from the "start the clock
interrupt" parts. Currently, cpu_initclocks() does everything all at
once, so there is no space for this MI setup.

Many platforms have more-or-less already done this separation by
implementing a separate routine named "cpu_startclock()". This patch
promotes cpu_startclock() from de facto standard to mandatory API.

- Prototype cpu_startclock() in sys/systm.h alongside cpu_initclocks().
The separation of responsibility between the two routines is a bit
fuzzy but the basic guidelines are as follows:

+ cpu_initclocks() must initialize hz, stathz, and profhz, and call
clockintr_init().

+ cpu_startclock() must call clockintr_cpu_init() and start the clock
interrupt cycle on the calling CPU.

These guidelines will shift in the future, but that's the way things
stand as of *this* commit.

- In initclocks(): first call cpu_initclocks(), then do MI setup, and
last call cpu_startclock().

- On platforms where cpu_startclock() already exists: don't call
cpu_startclock() from cpu_initclocks() anymore.

- On platforms where cpu_startclock() doesn't yet exist: implement it.
Usually this is as simple as dividing cpu_initclocks() in two.

Tested on amd64 (i8254, lapic), arm64, i386 (i8254, lapic), macppc,
mips64/octeon, and sparc64. Tested on arm/armv7 (agtimer(4)) by
phessler@ and jmatthew@. Tested on m88k/luna88k by aoyama@. Tested
on powerpc64 by gkoehler@ and mlarkin@. Tested on riscv64 by
jmatthew@.

Thread: https://marc.info/?l=openbsd-tech&m=169195251322149&w=2

hardclock(9): move setitimer(2) code into itimer_update()

- Move the setitimer(2) code responsible for updating the ITIMER_VIRTUAL
and ITIMER_PROF timers from hardclock(9) into a new clock interrupt
routine, itimer_update(). itimer_update() is periodic and runs at the
same frequency as the hardclock.

+ Revise itimerdecr() to run within itimer_mtx instead of entering
and leaving it.

- Each schedstate_percpu has its own itimer_update() handle, spc_itimer.
A new scheduler flag, SPCF_ITIMER, indicates whether spc_itimer was
started during the last mi_switch() and needs to be stopped during the
next mi_switch() or sched_exit().

- A new per-process flag, PS_ITIMER, indicates whether ITIMER_VIRTUAL
and/or ITIMER_PROF are running. Checking the flag is easier than
entering itimer_mtx to check process.ps_timer[]. The flag is set
and cleared in a new helper function, process_reset_itimer_flag().

- In setitimer(), call need_resched() when the state of ITIMER_VIRTUAL
or ITIMER_PROF is changed to force an mi_switch() and update
spc_itimer.

claudio@ notes that ITIMER_PROF could be implemented as a high-res
timer using the thread's execution time as a guide for when to
interrupt the process and assert SIGPROF. This would probably work
really well in single-threaded processes. ITIMER_VIRTUAL would be
more difficult to make high-res, though, as you need to exclude time
spent in the kernel.

Tested on powerpc64 by gkoehler@. With input from claudio@.

Thread: https://marc.info/?l=openbsd-tech&m=169038818517101&w=2

ok claudio@

struct sleep_state is no longer used, remove it.
Also remove the priority argument to sleep_finish() the code can use
the p_flag P_SINTR flag to know if the signal check is needed or not.
OK cheloha@ kettenis@ mpi@

First step at removing struct sleep_state.

Pass the timeout and sleep priority not only to sleep_setup() but also
to sleep_finish(). With that sls_timeout and sls_catch can be removed
from struct sleep_state.

The timeout is now setup first thing in sleep_finish() and no longer as
last thing in sleep_setup(). This should not cause a noticeable difference
since the code run between sleep_setup() and sleep_finish() is minimal.

OK kettenis@

On systems without xonly mmu hardware-enforcement, we can still mitigate
against classic BROP with a range-checking wrapper in front of copyin() and
copyinstr() which ensures the userland source doesn't overlap the main program
text, ld.so text, signal tramp text (it's mapping is hard to distinguish
so it comes along for the ride), or libc.so text. ld.so tells the kernel
libc.so text range with msyscall(2). The range checking for 2-4 elements is
done without locking (because all 4 ranges are immutable!) and is inexpensive.

write(sock, &open, 400) now fails with EFAULT. No programs have been
discovered which require reading their own text segments with a system call.

On a machine without mmu enforcement, a test program reports the following:
userland kernel
ld.so readable unreadable
mmap xz unreadable unreadable
mmap x readable readable
mmap nrx readable readable
mmap nwx readable readable
mmap xnwx readable readable
main readable unreadable
libc unmapped? readable unreadable
libc mapped readable unreadable

ok kettenis, additional help from miod

Remove copystr(9), unless used internally by copy{in,out}str.

Allow suspend with root on sdmmc(4).

ok deraadt@

Clean up the netlock macros. Merge NET_RLOCK_IN_SOFTNET and
NET_RLOCK_IN_IOCTL, which have the same implementation. The R and
W are hard to see, call the new macro NET_LOCK_SHARED. Rename the
opposite assertion from NET_ASSERT_WLOCKED to NET_ASSERT_LOCKED_EXCLUSIVE.
Update some outdated comments about net locking.
OK mpi@ mvs@

Add db_rint(), an MI interface to db_enter() copied from kdbrint() in vax code

If ddb.console is set and your serial console driver uses it, db_rint(),
lets you enter ddb(4) by typing the ESC D escape sequence. This is
useful for drivers like sfuart(4) where the hardware doesn't have a true
BREAK mechanism.

Suggested by miod@, ok kettenis@ miod@

Remove old poll/select wakeup mechanism.

Also remove unneeded seltrue() and selfalse().

OK mpi@ jsg@

We only have one syscall table: inline sysent/SYS_MAXSYSCALL and
SYS_syscall as the nosys() function into the MD syscall entry
routines and the SYSCALL_DEBUG support. Adjust alpha's syscall
check to match the other archs. Also, make sysent const to get it
into .rodata.

With that, 'struct emul' is unused: delete it and all its references

ok millert@

kernel: introduce per-CPU panic(9) message buffers

Add a 512-byte buffer (ci_panicbuf) to each cpu_info struct on each
platform for use by panic(9). The first panic on a given CPU writes
its message to this buffer. Subsequent panics on a given CPU print
the panic message to the console but do not modify the buffer. This
aids debugging in two cases:

- If 2+ CPUs panic simultaneously there is no risk of garbled messages
in the panic buffer.

- If a CPU panics and then the operator causes a second panic while
using ddb(4), the operator can still recall the first failure on
a particular CPU.

Misc. changes to support this bigger change:

- Set panicstr atomically to identify the first CPU to reach panic().

- Tweak db_show_panic_cmd() to print all panic messages across all
CPUs. Prefix the first panic with an asterisk ('*').

- Prefer db_printf() to printf() during a panic if we have it.
Apparently it disturbs less global state.

- On amd64, tweak fault() to write the local panic buffer. This needs
more work.

Prompted by bluhm@ and deraadt@. Mostly written by deraadt@.
Discussed with bluhm@, deraadt@ and kettenis@.

Borne from a discussion on tech@ about making panic(9) more MP-safe:

https://marc.info/?l=openbsd-tech&m=162086462316143&w=2

ok kettenis@, visa@, bluhm@, deraadt@

time to add NET_ASSERT_WLOCKED()

with moving towards NET_RLOCK...() we need NET_ASSERT_WLOCKED()
to check caller owns netlock exclusively.

OK @bluhm

Simplify sleep_setup API to two operations in preparation for splitting
the SCHED_LOCK().

Putting a thread on a sleep queue is reduce to the following:

sleep_setup();
/* check condition or release lock */
sleep_finish();

Previous version ok cheloha@, jmatthew@, ok claudio@

Remove obsolete vnode operation vector declarations.

OK bluhm@, claudio@, mpi@, semarie@

Make NET_LOCK() assertions conditional to DIAGNOSTIC

This saves about 2.5 KiB off amd64's RAMDISK after gzip compression.

OK deraadt@, mpi@, cheloha@

tsleep(9): add global "nowake" channel for threads avoiding wakeup(9)

It would be convenient if there were a channel a thread could sleep on
to indicate they do not want any wakeup(9) broadcasts. The easiest way
to do this is to add an "int nowake" to kern_synch.c and extern it in
sys/systm.h. You use it like this:

#include <sys/systm.h>

tsleep_nsec(&nowait, ...);

There is now no need to handroll a local dead channel, e.g.

int chan;

tsleep_nsec(&chan, ...);

which expands the stack. Local dead channels will be replaced with
&nowake in later patches.

One possible problem with this "one global channel" approach is sleep
queue congestion. If you have lots of threads sleeping on &nowake you
might slow down a wakeup(9) on a different channel that hashes into
the same queue. Unsure how much of problem this actually is, if at all.

NetBSD and FreeBSD have a "pause" interface in the kernel that chooses
a suitable channel automatically. To keep things simple and avoid
adding a new interface we will start with this global channel.

Discussed with mpi@, claudio@, kettenis@, and deraadt@.

Basically designed by kettenis@, who vetoed my other proposals.

Bugs caught by deraadt@, tb@, and patrick@.

Declare hw_{prod,serial,uuid,vendor,ver} in <sys/systm.h>.

OK deraadt@, mpi@

dev/rndvar.h no longer has statistical interfaces (removed during various
conversion steps). it only contains kernel prototypes for 4 interfaces,
all of which legitimately belong in sys/systm.h, which are already included
by all enqueue_randomness() users.

Document the various flavors of NET_LOCK() and rename the reader version.

Since our last concurrency mistake only ioctl(2) ans sysctl(2) code path
take the reader lock. This is mostly for documentation purpose as long as
the softnet thread is converted back to use a read lock.

dlg@ said that comments should be good enough.

ok sashan@

tsleep_nsec(9): add MAXTSLP macro, the maximum sleep duration

This macro will be useful for truncating durations below INFSLP
(UINT64_MAX) when converting from a timespec or timeval to a count
of nanoseconds before calling tsleep_nsec(9), msleep_nsec(9), or
rwsleep_nsec(9).

A relative timespec can hold many more nanoseconds than a uint64_t
can. TIMESPEC_TO_NSEC() and TIMEVAL_TO_NSEC() check for overflow,
returning UINT64_MAX if the conversion would overflow a uint64_t.

Thus, MAXTSLP will make it easy to avoid inadvertently passing INFSLP
to tsleep_nsec(9) et al. when the caller intended to set a timeout.

The code in such a case might look something like this:

uint64_t nsecs = MIN(TIMESPEC_TO_NSEC(&ts), MAXTSLP);

The macro may also be useful for rejecting intervals that are "too large",
e.g. for sockets with timeouts, if the timeout duration is to be stored
as a uint64_t in an object in the kernel. The code in such a case might
look something like this:

case SIOCTIMEOUT:
{
struct timeval *tv = (struct timeval *)data;
uint64_t nsecs;

if (tv->tv_sec < 0 || !timerisvalid(tv))
return EINVAL;

nsecs = TIMEVAL_TO_NSEC(tv);
if (nsecs > MAXTSLP)
return EOVERFLOW;

obj.timeout = nsecs;
break;
}

Idea suggested by visa@.

ok visa@

Move kernel locking inside the sleep machinery. This enables calling
rwsleep(9) with PCATCH and rw_enter(9) with RW_INTR without the kernel
lock. In addition, now tsleep(9) with PCATCH should be safe to use
without the kernel lock if the sleep is purely time-based.

Tested by anton@, cheloha@, chris@
OK anton@, cheloha@

softclock: move softintr registration/scheduling into timeout module

softclock() is scheduled from hardclock(9) because long ago callouts were
processed from hardclock(9) directly. The introduction of timeout(9) circa
2000 moved all callout processing into a dedicated module, but the softclock
scheduling stayed behind in hardclock(9).

We can move all the softclock() "stuff" into the timeout module to make
kern_clock.c a bit cleaner. Neither initclocks() nor hardclock(9) need
to "know" about softclock(). The initial softclock() softintr registration
can be done from timeout_proc_init() and softclock() can be scheduled
from timeout_hardclock_update().

ok visa@

Add tsleep_nsec(9), msleep_nsec(9), and rwsleep_nsec(9).

Equivalent to their unsuffixed counterparts except that (a) they take
a timeout in terms of nanoseconds, and (b) INFSLP, aka UINT64_MAX (not
zero) indicates that a timeout should not be set.

For now, zero nanoseconds is not a strictly valid invocation: we log a
warning on DIAGNOSTIC kernels if we see such a call. We still sleep
until the next tick in such a case, however. In the future this could
become some sort of poll... TBD.

To facilitate conversions to these interfaces: add inline conversion
functions to sys/time.h for turning your timeout into nanoseconds.

Also do a few easy conversions for warmup and to demonstrate how
further conversions should be done.

Lots of input from mpi@ and ratchov@. Additional input from tedu@,
deraadt@, mortimer@, millert@, and claudio@.

Partly inspired by FreeBSD r247787.

positive feedback from deraadt@, ok mpi@

Remove file name and line number output from witness(4)

Reduce code clutter by removing the file name and line number output
from witness(4). Typically it is easy enough to locate offending locks
using the stack traces that are shown in lock order conflict reports.
Tricky cases can be tracked using sysctl kern.witness.locktrace=1 .

This patch additionally removes the witness(4) wrapper for mutexes.
Now each mutex implementation has to invoke the WITNESS_*() macros
in order to utilize the checker.

Discussed with and OK dlg@, OK mpi@

Add sleep_finish_all(), which provides the common combo of sleep_finish(),
sleep_finish_timeout(), and sleep_finish_signal() with error preferencing,
and then use it in five places.

ok mpi@

Do not panic from ddb(4) when a lock requirement isn't fulfilled.

Extend the logic already present for panic() to any DDB-related
operation such that if ddb(4) is entered because of a fault or
other trap it is still possible to call 'boot reboot'.

While here stop printing splassert() messages as well, to not fill
the buffer.

ok visa@, deraadt@

Use a temporary chacha instance to fill large randomdata sections. Avoids
grabbing the rnglock repeatedly.

ok deraadt@ djm@

Show uvm_fault and trace when typing show panic on a page fault'd kernel

Currently there is only support for amd64, if this change settles
I will add support for the rest of the architectures.

OK kettenis@.

add code to provide simple wait condition handling.

this will be used to replace the bare sleep_state handling in a
bunch of places, starting with the barriers.

Do not call splassert_fail() if splassert_ctl is <= 0.

This matches splassert(9)s behavior and prevent noise when a CPU
panic(9) and set splassert_ctl to 0.

Found the hardway by sthen@