Make `logread_filterops' MP safe. For that purpose use `log_mtx' mutex(9)
protecting message buffer.

ok bluhm

struct sleep_state is no longer used, remove it.
Also remove the priority argument to sleep_finish() the code can use
the p_flag P_SINTR flag to know if the signal check is needed or not.
OK cheloha@ kettenis@ mpi@

First step at removing struct sleep_state.

Pass the timeout and sleep priority not only to sleep_setup() but also
to sleep_finish(). With that sls_timeout and sls_catch can be removed
from struct sleep_state.

The timeout is now setup first thing in sleep_finish() and no longer as
last thing in sleep_setup(). This should not cause a noticeable difference
since the code run between sleep_setup() and sleep_finish() is minimal.

OK kettenis@

Remove unused device poll functions.

Also remove unneeded includes of <sys/poll.h> and <sys/select.h>.

Some addenda from jsg@.

OK miod@ mpi@

Unlock sendsyslog(2). Console output still requires kernel lock to be
held but this path is only followed while `syslogf' socket is not set.

New `syslogf_rwlock' used to protect `syslogf' access.

ok bluhm@

Early daemons like dhcpleased(8), slaacd(8), unwind(8), resolvd(8)
are started before syslogd(8). This resulted in ugly sendsyslog(2)
dropped logs and the real message was lost.
Create a temporary stash for log messages within the kernel. It
has a limited size of 100 messages, and each message is truncated
to 8192 bytes. When the stash is exhausted, the well-known dropped
message is generated with a counter. After syslogd(8) has setup
everything, it sends a debug line through libc to flush the kernel
stash. Then syslogd receives all messages from the kernel before
the usual logs.
OK deraadt@ visa@

Simplify sleep_setup API to two operations in preparation for splitting
the SCHED_LOCK().

Putting a thread on a sleep queue is reduce to the following:

sleep_setup();
/* check condition or release lock */
sleep_finish();

Previous version ok cheloha@, jmatthew@, ok claudio@

Fix boot-time crash on sparc64

On sparc64, initmsgbuf() is invoked before curcpu() is usable
on the boot processor. Consequently, it is unsafe to use mutexes
during the message buffer initialization. Avoid such use by skipping
log_mtx when appending a newline from initmsgbuf().

Use mbp instead of msgbufp as the buffer argument to the putchar routine
for consistency.

Bug reported and fix suggested by miod@

Refactor klist insertion and removal

Rename klist_{insert,remove}() to klist_{insert,remove}_locked().
These functions assume that the caller has locked the klist. The current
state of locking remains intact because the kernel lock is still used
with all klists.

Add new functions klist_insert() and klist_remove() that lock the klist
internally. This allows some code simplification.

OK mpi@

Serialize msgbuf access with a mutex.

This introduces a system-wide mutex that serializes msgbuf operations.
The mutex controls access to all modifiable fields of struct msgbuf.
It also covers logsoftc.sc_state.

To avoid adding extra lock order constraints that would affect use of
printf(9), the code does not take new locks when the log mutex is held.

The code assumes that there is at most one thread using logread(). This
keeps the logic simple. If there was more than one reader, logread()
might return the same data to different readers. Also, log wakeup might
not be reliable with multiple threads.

Tested in snaps for two weeks.

OK mpi@

Fix kn_data returned by filt_logread().

Take into account the circular nature of the message buffer when
computing the number of available bytes. Move the computation into
a separate function and use it with the kevent(2) and ioctl(2)
interfaces.

OK mpi@

Remove an unnecessary field from struct msgbuf.

OK mvs@

Abstract the head of knote lists. This allows extending the lists,
for example, with locking assertions.

OK mpi@, anton@

Replace field f_isfd with field f_flags in struct filterops to allow
adding more filter properties without cluttering the struct.

OK mpi@, anton@

Convert infinite sleeps to tsleep_nsec(9).

ok bluhm@

Unify handling of ioctls FIOSETOWN/SIOCSPGRP/TIOCSPGRP and
FIOGETOWN/SIOCGPGRP/TIOCGPGRP. Do this by determining the meaning of
the ID parameter inside the sigio code. Also add cases for FIOSETOWN
and FIOGETOWN where there have been TIOCSPGRP and TIOCGPGRP before.
These changes allow removing the ID translation from sys_fcntl() and
sys_ioctl().

Idea from NetBSD

OK mpi@, claudio@

Change /dev/log to use pgsigio(9) and sigio_init(9) like it was done for
bpf(4) and tun(4) instead of using hand rolled code using csignal().
OK visa@

Use C99 designated initializers with struct filterops. In addition,
make the structs const so that the data are put in .rodata.

OK mpi@, deraadt@, anton@, bluhm@

The console buffer is allocated during startup. initconsbuf() is
only called from main(). There allocation must not fail, so better
use M_WAITOK and remove error handling. As it is not a temporary
buffer, M_TTYS is more appropriate.
OK deraadt@ mpi@

Make log tick interval independent of hz and reduce the frequency
so that the timeout would not be scheduled on every tick.

Discussed with and OK mpi@

Work around locking issues in logwakeup(). Instead of actually waking up
waiters, just set a flag in logwakeup(). The flag is later noted through
periodic polling. This lets the wakeup code run with sufficient locking.

logwakeup() is a very tricky place to take locks because the function
can be called in many different contexts. By not requiring locks in
the routine helps to keep printf(9) as usable as possible.

OK mpi@

dosendsyslog() must only pass ktrgenio(9) userspace buffers that it can
use copyin() on. While here: just put the struct iovec for ktrace on the
stack instead of mallocing and freeing it.

problem debugged by patrick@
ok deraadt@ mpi@

Use FNONBLOCK instead of SS_NBIO to check/indicate that the I/O mode
for sockets is non-blocking.

This allows us to G/C SS_NBIO. Having to keep the two flags in sync
in a mp-safe way is complicated.

This change introduce a behavior change in sosplice(), it can now
always block. However this should not matter much due to the socket
lock being taken beforhand.

ok bluhm@, benno@, visa@

Remove almost unused `flags' argument of suser().

The account flag `ASU' will no longer be set but that makes suser()
mpsafe since it no longer mess with a per-process field.

No objection from millert@, ok tedu@, bluhm@

Print the pid of the most recent program that failed to send a log
via sendsyslog(2) along with the corresponding errno.

Help when troubleshooting which program is triggering an error, like
an overflow.

ok bluhm@

sendsyslog should take a const char * everywhere.

okay bluhm@, deraadt@

Kernel sendsyslog(2), libc syslog(3), and syslogd(8) restrict and
truncate the length of a syslog message to 8192 bytes. Use one
global define LOG_MAXLINE for all of them.
OK deraadt@ millert@

Both syslog(3) and syslogd(8) truncate the message at 8192 bytes.
Do the same in sendsyslog(2) and document the behavior.
reported by Ilja Van Sprundel; OK millert@ deraadt@

branches: 1.50.4;
Reorder FREF() and FRELE() in a way that the the global variable
syslogf always points to a file object with increased reference
count. This makes the implementation independent from the fact
whether changing the reference counter may sleep.
pointed out by Mateusz Guzik; OK deraadt@

There was a race in dosendsyslog() which resulted in a crash.
sosend(syslogf->f_data, ...) could be called with a NULL pointer.
syslogf was not NULL, f_data was NULL and f_count was 1. The file
structure is ref counted, but the global variable syslogf is not
protected. So it may change during sleep and dosendsyslog() possibly
used a different socket at each access. Solution is to access
syslogf only once, use a local copy, and do the ref counting there.
OK millert@ deraadt@

branches: 1.48.2;
As klog dropped message has no ifdef small kernel, sendsyslog
should not have it either. While there bring some variables in
sync between both functions.
OK deraadt@

It is annoying that the dmesg buffer can overflow and loose messages
undetected during debugging. To make clear what happens, count the
dropped bytes and write message buffer full to syslogd. This also
helps to have a reliable log system.
OK deraadt@ millert@ tedu@

As logging to dmesg with msgbuf_putchar() can happen in any interrupt
handler, every access to msg buf counters should be protected by
splhigh(). This is already done in some places, make it consistenly
everywhere.
OK mpi@ deraadt@

sys_osendsyslog can go away; we have transitioned to the new version with
a flags argument
ok guenther sthen

All msg buf counters are long, so lmin(9) should be used for them.
OK deraadt@ natano@

Kill trailing whitespaces.

Use constty and cn_devvp for checking wether to use cnwrite(). That
makes console redirection with senssyslog(2) and LOG_CONS work
again. Also merge the two if else if else blocks into one.
OK deraadt@

Allow sendsyslog(2) with LOG_CONS even when /dev/console has not
been opened during init(8). Only log with cnwrite() if cn_devvp
exists, otherwise use cnputc() as fallback. While there move extern
declarations to dev/cons.h.
input and OK deraadt@

Backout the previous fix for the sendsyslog(2) with LOG_CONS solution.
Permanently holding /dev/console open in the kernel works only until
init(8) calls revoke(2). After that the console device vnode cannot
be used anymore. It still resulted in a hanging init(8) if it tried
to syslog(3) something. With the backout also dmesg -s works again.

If sendsyslog(2) is called with LOG_CONS before syslogd(8) has been
started and before init(8) has opened the console, the kernel could
crash as the console device has not been initialized. Open
/dev/console in the kernel before starting init(8) and keep it open.
This way sendsyslog(2) can be called early in the system.
OK beck@ deraadt@

Rename the system call sendsyslog2 to sendsyslog. Keep the old one
as osendsyslog for a while. The three argument variant is the only
one that will stay.
input kettenis@; OK deraadt@

Convert to uiomove(); from Martin Natano, thanks!

ok deraadt@

In sendsyslogd(2) strip off syslog priority when logging to console.
OK deraadt@ millert@

The pointer buf is a user space string which was directly passed
to tputchar() and could crash the kernel. Better use cnwrite() in
sendsyslog2() for writing to console. It takes a struct uio which
does the copyin() automatically. In addition cnwrite() outputs to
the real console or to a redirected one, whichever is appropriate.
One drawback is that the syslog priority cannot be stripped off
easily.
OK deraadt@

remove stale lint annotations

Add sendsyslog2(), which accepts the syslog/openlog "logopt" flag
LOG_CONS. If syslogd is not accepting messages, direct them to the console.
This allows us to remove the direct /dev/console opening code from the
bowels of libc. Of course, that forgotten code was exposed by pledge.
ok kettenis millert beck

The must be no space after the syslog priority in the sendsyslog(2)
dropped message error log.
OK benno@

To make logging to local syslog reliable, log a message about failed
log atempts. sendsyslog(2) is a good place to detect and report
the problem.
OK deraadt@

Pass a thread pointer instead of its file descriptor table to getsock(9).

Diff from Vitaliy Makkoveev.

Manpage tweak and ok millert@

Remove some includes include-what-you-use claims don't
have any direct symbols used. Tested for indirect use by compiling
amd64/i386/sparc64 kernels.

ok tedu@ deraadt@

First step towards making uiomove() take a size_t size argument:
- rename uiomove() to uiomovei() and update all its users.
- introduce uiomove(), which is similar to uiomovei() but with a size_t.
- rewrite uiomovei() as an uiomove() wrapper.
ok kettenis@

Many architectures call initmsgbuf() really really early, before uvm is
initialized. Calling malloc(9) at that point is not a good idea. So
initialize consbuf later.

Fixes dmesg -s on sparc64 (and probably a few other architectures).

ok miod@, deraadt@

Add dmesg -s support, to view the output of rc(8) system startup messages.
Help and feedback by Theo and Miod.
OK deraadt@, manpage-ok jmc@

yet more mallocarray() changes.

ok tedu@ deraadt@

convert bcopy to memcpy. ok millert

pass size argument to free()
ok doug tedu

I/O ktrace of sendsyslog(2) did not work. As uiomove() adjusts
iov_len to 0, we need a propper length calculation. While there,
use -1 for the file descriptor because 0 is reserved for stdin.
OK deraadt@ guenther@

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Add new sendsyslog(const char *, size_t) system call which simply passes
a message up into syslogd's /dev/log interface. This will be used to make
syslog_r work during file descriptor exhaustion, or inside sandboxes which
prohibit socket, connect, sendto, etc.
The system call is being added about a week before the library and daemon
changes.
ok guenther tedu miod matthew

Eliminates struct pcred by moving the real and saved ugids into
struct ucred; struct process then directly links to the ucred

Based on a discussion at c2k10 or so before noting that FreeBSD and
NetBSD did this too.

ok matthew@

bzero -> memset

kqueue attach functions should return an errno or 0, not a plain 1. Fix
the obvious cases to return EINVAL and ENXIO.

ok tedu deraadt

Every selwakeup() should have a matching KNOTE() (even if kqueue isn't
supported it doesn't do any harm), so put the KNOTE() in selwakeup() itself and
remove it from any occurences where both are used, except one for kqueue itself
and one in sys_pipe.c (where the selwakeup is under a PIPE_SEL flag).

Based on a diff from tedu.

ok deraadt

unsupported ioctl's should be returned with ENOTTY
not -1;

ok miod@

ansi and deregister. No binary change.

knf

Try to start log message buffer on a new line. Helps install scripts avoid
detecting phantom devices when scanning dmesg output.

Suggested by miod@, ok deraadt@.

Replace select backends with poll backends. selscan() and pollscan()
now call the poll backend. With this change we implement greater
poll(2) functionality instead of emulating it via the select backend.
Adapted from NetBSD and including some changes from FreeBSD.
Tested by many, deraadt@ OK

remove caddr_t casts. it's just silly to cast something when the function
takes a void *. convert uiomove to take a void * as well. ok deraadt@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

kqfilter for the log. niels said i can commit it if it works
and i converted syslog to libevent and it works now (;

branches: 1.7.6;


Move offsetof define into sys/param.h

enlarge msgbuf, somewhat line netbsd did

branches: 1.5.12;
it is definately wrong to initialize the pgid at open() time because the
uid/euid used for checking in csignal() is controlled by whoever used
TIOCSPGRP previously.

for non-tty TIOCSPGRP/F_SETOWN/FIOSETOWN pgid setting calls, store uid
and euid as well, then deliver them using new csignal() interface
which ensures that pgid setting process is permitted to signal the
pgid process(es). Thanks to newsham@aloha.net for extensive help and
discussion.

partial sync with netbsd 960418, more to come

From NetBSD: 960217 merge

branches: 1.1.1;
Initial revision

struct sleep_state is no longer used, remove it.
Also remove the priority argument to sleep_finish() the code can use
the p_flag P_SINTR flag to know if the signal check is needed or not.
OK cheloha@ kettenis@ mpi@

First step at removing struct sleep_state.

Pass the timeout and sleep priority not only to sleep_setup() but also
to sleep_finish(). With that sls_timeout and sls_catch can be removed
from struct sleep_state.

The timeout is now setup first thing in sleep_finish() and no longer as
last thing in sleep_setup(). This should not cause a noticeable difference
since the code run between sleep_setup() and sleep_finish() is minimal.

OK kettenis@

Remove unused device poll functions.

Also remove unneeded includes of <sys/poll.h> and <sys/select.h>.

Some addenda from jsg@.

OK miod@ mpi@

Unlock sendsyslog(2). Console output still requires kernel lock to be
held but this path is only followed while `syslogf' socket is not set.

New `syslogf_rwlock' used to protect `syslogf' access.

ok bluhm@

Early daemons like dhcpleased(8), slaacd(8), unwind(8), resolvd(8)
are started before syslogd(8). This resulted in ugly sendsyslog(2)
dropped logs and the real message was lost.
Create a temporary stash for log messages within the kernel. It
has a limited size of 100 messages, and each message is truncated
to 8192 bytes. When the stash is exhausted, the well-known dropped
message is generated with a counter. After syslogd(8) has setup
everything, it sends a debug line through libc to flush the kernel
stash. Then syslogd receives all messages from the kernel before
the usual logs.
OK deraadt@ visa@

Simplify sleep_setup API to two operations in preparation for splitting
the SCHED_LOCK().

Putting a thread on a sleep queue is reduce to the following:

sleep_setup();
/* check condition or release lock */
sleep_finish();

Previous version ok cheloha@, jmatthew@, ok claudio@

Fix boot-time crash on sparc64

On sparc64, initmsgbuf() is invoked before curcpu() is usable
on the boot processor. Consequently, it is unsafe to use mutexes
during the message buffer initialization. Avoid such use by skipping
log_mtx when appending a newline from initmsgbuf().

Use mbp instead of msgbufp as the buffer argument to the putchar routine
for consistency.

Bug reported and fix suggested by miod@

Refactor klist insertion and removal

Rename klist_{insert,remove}() to klist_{insert,remove}_locked().
These functions assume that the caller has locked the klist. The current
state of locking remains intact because the kernel lock is still used
with all klists.

Add new functions klist_insert() and klist_remove() that lock the klist
internally. This allows some code simplification.

OK mpi@

Serialize msgbuf access with a mutex.

This introduces a system-wide mutex that serializes msgbuf operations.
The mutex controls access to all modifiable fields of struct msgbuf.
It also covers logsoftc.sc_state.

To avoid adding extra lock order constraints that would affect use of
printf(9), the code does not take new locks when the log mutex is held.

The code assumes that there is at most one thread using logread(). This
keeps the logic simple. If there was more than one reader, logread()
might return the same data to different readers. Also, log wakeup might
not be reliable with multiple threads.

Tested in snaps for two weeks.

OK mpi@

Fix kn_data returned by filt_logread().

Take into account the circular nature of the message buffer when
computing the number of available bytes. Move the computation into
a separate function and use it with the kevent(2) and ioctl(2)
interfaces.

OK mpi@

Remove an unnecessary field from struct msgbuf.

OK mvs@

Abstract the head of knote lists. This allows extending the lists,
for example, with locking assertions.

OK mpi@, anton@

Replace field f_isfd with field f_flags in struct filterops to allow
adding more filter properties without cluttering the struct.

OK mpi@, anton@

Convert infinite sleeps to tsleep_nsec(9).

ok bluhm@

Unify handling of ioctls FIOSETOWN/SIOCSPGRP/TIOCSPGRP and
FIOGETOWN/SIOCGPGRP/TIOCGPGRP. Do this by determining the meaning of
the ID parameter inside the sigio code. Also add cases for FIOSETOWN
and FIOGETOWN where there have been TIOCSPGRP and TIOCGPGRP before.
These changes allow removing the ID translation from sys_fcntl() and
sys_ioctl().

Idea from NetBSD

OK mpi@, claudio@

Change /dev/log to use pgsigio(9) and sigio_init(9) like it was done for
bpf(4) and tun(4) instead of using hand rolled code using csignal().
OK visa@

Use C99 designated initializers with struct filterops. In addition,
make the structs const so that the data are put in .rodata.

OK mpi@, deraadt@, anton@, bluhm@

The console buffer is allocated during startup. initconsbuf() is
only called from main(). There allocation must not fail, so better
use M_WAITOK and remove error handling. As it is not a temporary
buffer, M_TTYS is more appropriate.
OK deraadt@ mpi@

Make log tick interval independent of hz and reduce the frequency
so that the timeout would not be scheduled on every tick.

Discussed with and OK mpi@

Work around locking issues in logwakeup(). Instead of actually waking up
waiters, just set a flag in logwakeup(). The flag is later noted through
periodic polling. This lets the wakeup code run with sufficient locking.

logwakeup() is a very tricky place to take locks because the function
can be called in many different contexts. By not requiring locks in
the routine helps to keep printf(9) as usable as possible.

OK mpi@

dosendsyslog() must only pass ktrgenio(9) userspace buffers that it can
use copyin() on. While here: just put the struct iovec for ktrace on the
stack instead of mallocing and freeing it.

problem debugged by patrick@
ok deraadt@ mpi@

Use FNONBLOCK instead of SS_NBIO to check/indicate that the I/O mode
for sockets is non-blocking.

This allows us to G/C SS_NBIO. Having to keep the two flags in sync
in a mp-safe way is complicated.

This change introduce a behavior change in sosplice(), it can now
always block. However this should not matter much due to the socket
lock being taken beforhand.

ok bluhm@, benno@, visa@

Remove almost unused `flags' argument of suser().

The account flag `ASU' will no longer be set but that makes suser()
mpsafe since it no longer mess with a per-process field.

No objection from millert@, ok tedu@, bluhm@

Print the pid of the most recent program that failed to send a log
via sendsyslog(2) along with the corresponding errno.

Help when troubleshooting which program is triggering an error, like
an overflow.

ok bluhm@

sendsyslog should take a const char * everywhere.

okay bluhm@, deraadt@

Kernel sendsyslog(2), libc syslog(3), and syslogd(8) restrict and
truncate the length of a syslog message to 8192 bytes. Use one
global define LOG_MAXLINE for all of them.
OK deraadt@ millert@

Both syslog(3) and syslogd(8) truncate the message at 8192 bytes.
Do the same in sendsyslog(2) and document the behavior.
reported by Ilja Van Sprundel; OK millert@ deraadt@

branches: 1.50.4;
Reorder FREF() and FRELE() in a way that the the global variable
syslogf always points to a file object with increased reference
count. This makes the implementation independent from the fact
whether changing the reference counter may sleep.
pointed out by Mateusz Guzik; OK deraadt@

There was a race in dosendsyslog() which resulted in a crash.
sosend(syslogf->f_data, ...) could be called with a NULL pointer.
syslogf was not NULL, f_data was NULL and f_count was 1. The file
structure is ref counted, but the global variable syslogf is not
protected. So it may change during sleep and dosendsyslog() possibly
used a different socket at each access. Solution is to access
syslogf only once, use a local copy, and do the ref counting there.
OK millert@ deraadt@

branches: 1.48.2;
As klog dropped message has no ifdef small kernel, sendsyslog
should not have it either. While there bring some variables in
sync between both functions.
OK deraadt@

It is annoying that the dmesg buffer can overflow and loose messages
undetected during debugging. To make clear what happens, count the
dropped bytes and write message buffer full to syslogd. This also
helps to have a reliable log system.
OK deraadt@ millert@ tedu@

As logging to dmesg with msgbuf_putchar() can happen in any interrupt
handler, every access to msg buf counters should be protected by
splhigh(). This is already done in some places, make it consistenly
everywhere.
OK mpi@ deraadt@

sys_osendsyslog can go away; we have transitioned to the new version with
a flags argument
ok guenther sthen

All msg buf counters are long, so lmin(9) should be used for them.
OK deraadt@ natano@

Kill trailing whitespaces.

Use constty and cn_devvp for checking wether to use cnwrite(). That
makes console redirection with senssyslog(2) and LOG_CONS work
again. Also merge the two if else if else blocks into one.
OK deraadt@

Allow sendsyslog(2) with LOG_CONS even when /dev/console has not
been opened during init(8). Only log with cnwrite() if cn_devvp
exists, otherwise use cnputc() as fallback. While there move extern
declarations to dev/cons.h.
input and OK deraadt@

Backout the previous fix for the sendsyslog(2) with LOG_CONS solution.
Permanently holding /dev/console open in the kernel works only until
init(8) calls revoke(2). After that the console device vnode cannot
be used anymore. It still resulted in a hanging init(8) if it tried
to syslog(3) something. With the backout also dmesg -s works again.

If sendsyslog(2) is called with LOG_CONS before syslogd(8) has been
started and before init(8) has opened the console, the kernel could
crash as the console device has not been initialized. Open
/dev/console in the kernel before starting init(8) and keep it open.
This way sendsyslog(2) can be called early in the system.
OK beck@ deraadt@

Rename the system call sendsyslog2 to sendsyslog. Keep the old one
as osendsyslog for a while. The three argument variant is the only
one that will stay.
input kettenis@; OK deraadt@

Convert to uiomove(); from Martin Natano, thanks!

ok deraadt@

In sendsyslogd(2) strip off syslog priority when logging to console.
OK deraadt@ millert@

The pointer buf is a user space string which was directly passed
to tputchar() and could crash the kernel. Better use cnwrite() in
sendsyslog2() for writing to console. It takes a struct uio which
does the copyin() automatically. In addition cnwrite() outputs to
the real console or to a redirected one, whichever is appropriate.
One drawback is that the syslog priority cannot be stripped off
easily.
OK deraadt@

remove stale lint annotations

Add sendsyslog2(), which accepts the syslog/openlog "logopt" flag
LOG_CONS. If syslogd is not accepting messages, direct them to the console.
This allows us to remove the direct /dev/console opening code from the
bowels of libc. Of course, that forgotten code was exposed by pledge.
ok kettenis millert beck

The must be no space after the syslog priority in the sendsyslog(2)
dropped message error log.
OK benno@

To make logging to local syslog reliable, log a message about failed
log atempts. sendsyslog(2) is a good place to detect and report
the problem.
OK deraadt@

Pass a thread pointer instead of its file descriptor table to getsock(9).

Diff from Vitaliy Makkoveev.

Manpage tweak and ok millert@

Remove some includes include-what-you-use claims don't
have any direct symbols used. Tested for indirect use by compiling
amd64/i386/sparc64 kernels.

ok tedu@ deraadt@

First step towards making uiomove() take a size_t size argument:
- rename uiomove() to uiomovei() and update all its users.
- introduce uiomove(), which is similar to uiomovei() but with a size_t.
- rewrite uiomovei() as an uiomove() wrapper.
ok kettenis@

Many architectures call initmsgbuf() really really early, before uvm is
initialized. Calling malloc(9) at that point is not a good idea. So
initialize consbuf later.

Fixes dmesg -s on sparc64 (and probably a few other architectures).

ok miod@, deraadt@

Add dmesg -s support, to view the output of rc(8) system startup messages.
Help and feedback by Theo and Miod.
OK deraadt@, manpage-ok jmc@

yet more mallocarray() changes.

ok tedu@ deraadt@

convert bcopy to memcpy. ok millert

pass size argument to free()
ok doug tedu

I/O ktrace of sendsyslog(2) did not work. As uiomove() adjusts
iov_len to 0, we need a propper length calculation. While there,
use -1 for the file descriptor because 0 is reserved for stdin.
OK deraadt@ guenther@

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Add new sendsyslog(const char *, size_t) system call which simply passes
a message up into syslogd's /dev/log interface. This will be used to make
syslog_r work during file descriptor exhaustion, or inside sandboxes which
prohibit socket, connect, sendto, etc.
The system call is being added about a week before the library and daemon
changes.
ok guenther tedu miod matthew

Eliminates struct pcred by moving the real and saved ugids into
struct ucred; struct process then directly links to the ucred

Based on a discussion at c2k10 or so before noting that FreeBSD and
NetBSD did this too.

ok matthew@

bzero -> memset

kqueue attach functions should return an errno or 0, not a plain 1. Fix
the obvious cases to return EINVAL and ENXIO.

ok tedu deraadt

Every selwakeup() should have a matching KNOTE() (even if kqueue isn't
supported it doesn't do any harm), so put the KNOTE() in selwakeup() itself and
remove it from any occurences where both are used, except one for kqueue itself
and one in sys_pipe.c (where the selwakeup is under a PIPE_SEL flag).

Based on a diff from tedu.

ok deraadt

unsupported ioctl's should be returned with ENOTTY
not -1;

ok miod@

ansi and deregister. No binary change.

knf

Try to start log message buffer on a new line. Helps install scripts avoid
detecting phantom devices when scanning dmesg output.

Suggested by miod@, ok deraadt@.

Replace select backends with poll backends. selscan() and pollscan()
now call the poll backend. With this change we implement greater
poll(2) functionality instead of emulating it via the select backend.
Adapted from NetBSD and including some changes from FreeBSD.
Tested by many, deraadt@ OK

remove caddr_t casts. it's just silly to cast something when the function
takes a void *. convert uiomove to take a void * as well. ok deraadt@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

kqfilter for the log. niels said i can commit it if it works
and i converted syslog to libevent and it works now (;

branches: 1.7.6;


Move offsetof define into sys/param.h

enlarge msgbuf, somewhat line netbsd did

branches: 1.5.12;
it is definately wrong to initialize the pgid at open() time because the
uid/euid used for checking in csignal() is controlled by whoever used
TIOCSPGRP previously.

for non-tty TIOCSPGRP/F_SETOWN/FIOSETOWN pgid setting calls, store uid
and euid as well, then deliver them using new csignal() interface
which ensures that pgid setting process is permitted to signal the
pgid process(es). Thanks to newsham@aloha.net for extensive help and
discussion.

partial sync with netbsd 960418, more to come

From NetBSD: 960217 merge

branches: 1.1.1;
Initial revision

First step at removing struct sleep_state.

Pass the timeout and sleep priority not only to sleep_setup() but also
to sleep_finish(). With that sls_timeout and sls_catch can be removed
from struct sleep_state.

The timeout is now setup first thing in sleep_finish() and no longer as
last thing in sleep_setup(). This should not cause a noticeable difference
since the code run between sleep_setup() and sleep_finish() is minimal.

OK kettenis@

Remove unused device poll functions.

Also remove unneeded includes of <sys/poll.h> and <sys/select.h>.

Some addenda from jsg@.

OK miod@ mpi@

Unlock sendsyslog(2). Console output still requires kernel lock to be
held but this path is only followed while `syslogf' socket is not set.

New `syslogf_rwlock' used to protect `syslogf' access.

ok bluhm@

Early daemons like dhcpleased(8), slaacd(8), unwind(8), resolvd(8)
are started before syslogd(8). This resulted in ugly sendsyslog(2)
dropped logs and the real message was lost.
Create a temporary stash for log messages within the kernel. It
has a limited size of 100 messages, and each message is truncated
to 8192 bytes. When the stash is exhausted, the well-known dropped
message is generated with a counter. After syslogd(8) has setup
everything, it sends a debug line through libc to flush the kernel
stash. Then syslogd receives all messages from the kernel before
the usual logs.
OK deraadt@ visa@

Simplify sleep_setup API to two operations in preparation for splitting
the SCHED_LOCK().

Putting a thread on a sleep queue is reduce to the following:

sleep_setup();
/* check condition or release lock */
sleep_finish();

Previous version ok cheloha@, jmatthew@, ok claudio@

Fix boot-time crash on sparc64

On sparc64, initmsgbuf() is invoked before curcpu() is usable
on the boot processor. Consequently, it is unsafe to use mutexes
during the message buffer initialization. Avoid such use by skipping
log_mtx when appending a newline from initmsgbuf().

Use mbp instead of msgbufp as the buffer argument to the putchar routine
for consistency.

Bug reported and fix suggested by miod@

Refactor klist insertion and removal

Rename klist_{insert,remove}() to klist_{insert,remove}_locked().
These functions assume that the caller has locked the klist. The current
state of locking remains intact because the kernel lock is still used
with all klists.

Add new functions klist_insert() and klist_remove() that lock the klist
internally. This allows some code simplification.

OK mpi@

Serialize msgbuf access with a mutex.

This introduces a system-wide mutex that serializes msgbuf operations.
The mutex controls access to all modifiable fields of struct msgbuf.
It also covers logsoftc.sc_state.

To avoid adding extra lock order constraints that would affect use of
printf(9), the code does not take new locks when the log mutex is held.

The code assumes that there is at most one thread using logread(). This
keeps the logic simple. If there was more than one reader, logread()
might return the same data to different readers. Also, log wakeup might
not be reliable with multiple threads.

Tested in snaps for two weeks.

OK mpi@

Fix kn_data returned by filt_logread().

Take into account the circular nature of the message buffer when
computing the number of available bytes. Move the computation into
a separate function and use it with the kevent(2) and ioctl(2)
interfaces.

OK mpi@

Remove an unnecessary field from struct msgbuf.

OK mvs@

Abstract the head of knote lists. This allows extending the lists,
for example, with locking assertions.

OK mpi@, anton@

Replace field f_isfd with field f_flags in struct filterops to allow
adding more filter properties without cluttering the struct.

OK mpi@, anton@

Convert infinite sleeps to tsleep_nsec(9).

ok bluhm@

Unify handling of ioctls FIOSETOWN/SIOCSPGRP/TIOCSPGRP and
FIOGETOWN/SIOCGPGRP/TIOCGPGRP. Do this by determining the meaning of
the ID parameter inside the sigio code. Also add cases for FIOSETOWN
and FIOGETOWN where there have been TIOCSPGRP and TIOCGPGRP before.
These changes allow removing the ID translation from sys_fcntl() and
sys_ioctl().

Idea from NetBSD

OK mpi@, claudio@

Change /dev/log to use pgsigio(9) and sigio_init(9) like it was done for
bpf(4) and tun(4) instead of using hand rolled code using csignal().
OK visa@

Use C99 designated initializers with struct filterops. In addition,
make the structs const so that the data are put in .rodata.

OK mpi@, deraadt@, anton@, bluhm@

The console buffer is allocated during startup. initconsbuf() is
only called from main(). There allocation must not fail, so better
use M_WAITOK and remove error handling. As it is not a temporary
buffer, M_TTYS is more appropriate.
OK deraadt@ mpi@

Make log tick interval independent of hz and reduce the frequency
so that the timeout would not be scheduled on every tick.

Discussed with and OK mpi@

Work around locking issues in logwakeup(). Instead of actually waking up
waiters, just set a flag in logwakeup(). The flag is later noted through
periodic polling. This lets the wakeup code run with sufficient locking.

logwakeup() is a very tricky place to take locks because the function
can be called in many different contexts. By not requiring locks in
the routine helps to keep printf(9) as usable as possible.

OK mpi@

dosendsyslog() must only pass ktrgenio(9) userspace buffers that it can
use copyin() on. While here: just put the struct iovec for ktrace on the
stack instead of mallocing and freeing it.

problem debugged by patrick@
ok deraadt@ mpi@

Use FNONBLOCK instead of SS_NBIO to check/indicate that the I/O mode
for sockets is non-blocking.

This allows us to G/C SS_NBIO. Having to keep the two flags in sync
in a mp-safe way is complicated.

This change introduce a behavior change in sosplice(), it can now
always block. However this should not matter much due to the socket
lock being taken beforhand.

ok bluhm@, benno@, visa@

Remove almost unused `flags' argument of suser().

The account flag `ASU' will no longer be set but that makes suser()
mpsafe since it no longer mess with a per-process field.

No objection from millert@, ok tedu@, bluhm@

Print the pid of the most recent program that failed to send a log
via sendsyslog(2) along with the corresponding errno.

Help when troubleshooting which program is triggering an error, like
an overflow.

ok bluhm@

sendsyslog should take a const char * everywhere.

okay bluhm@, deraadt@

Kernel sendsyslog(2), libc syslog(3), and syslogd(8) restrict and
truncate the length of a syslog message to 8192 bytes. Use one
global define LOG_MAXLINE for all of them.
OK deraadt@ millert@

Both syslog(3) and syslogd(8) truncate the message at 8192 bytes.
Do the same in sendsyslog(2) and document the behavior.
reported by Ilja Van Sprundel; OK millert@ deraadt@

branches: 1.50.4;
Reorder FREF() and FRELE() in a way that the the global variable
syslogf always points to a file object with increased reference
count. This makes the implementation independent from the fact
whether changing the reference counter may sleep.
pointed out by Mateusz Guzik; OK deraadt@

There was a race in dosendsyslog() which resulted in a crash.
sosend(syslogf->f_data, ...) could be called with a NULL pointer.
syslogf was not NULL, f_data was NULL and f_count was 1. The file
structure is ref counted, but the global variable syslogf is not
protected. So it may change during sleep and dosendsyslog() possibly
used a different socket at each access. Solution is to access
syslogf only once, use a local copy, and do the ref counting there.
OK millert@ deraadt@

branches: 1.48.2;
As klog dropped message has no ifdef small kernel, sendsyslog
should not have it either. While there bring some variables in
sync between both functions.
OK deraadt@

It is annoying that the dmesg buffer can overflow and loose messages
undetected during debugging. To make clear what happens, count the
dropped bytes and write message buffer full to syslogd. This also
helps to have a reliable log system.
OK deraadt@ millert@ tedu@

As logging to dmesg with msgbuf_putchar() can happen in any interrupt
handler, every access to msg buf counters should be protected by
splhigh(). This is already done in some places, make it consistenly
everywhere.
OK mpi@ deraadt@

sys_osendsyslog can go away; we have transitioned to the new version with
a flags argument
ok guenther sthen

All msg buf counters are long, so lmin(9) should be used for them.
OK deraadt@ natano@

Kill trailing whitespaces.

Use constty and cn_devvp for checking wether to use cnwrite(). That
makes console redirection with senssyslog(2) and LOG_CONS work
again. Also merge the two if else if else blocks into one.
OK deraadt@

Allow sendsyslog(2) with LOG_CONS even when /dev/console has not
been opened during init(8). Only log with cnwrite() if cn_devvp
exists, otherwise use cnputc() as fallback. While there move extern
declarations to dev/cons.h.
input and OK deraadt@

Backout the previous fix for the sendsyslog(2) with LOG_CONS solution.
Permanently holding /dev/console open in the kernel works only until
init(8) calls revoke(2). After that the console device vnode cannot
be used anymore. It still resulted in a hanging init(8) if it tried
to syslog(3) something. With the backout also dmesg -s works again.

If sendsyslog(2) is called with LOG_CONS before syslogd(8) has been
started and before init(8) has opened the console, the kernel could
crash as the console device has not been initialized. Open
/dev/console in the kernel before starting init(8) and keep it open.
This way sendsyslog(2) can be called early in the system.
OK beck@ deraadt@

Rename the system call sendsyslog2 to sendsyslog. Keep the old one
as osendsyslog for a while. The three argument variant is the only
one that will stay.
input kettenis@; OK deraadt@

Convert to uiomove(); from Martin Natano, thanks!

ok deraadt@

In sendsyslogd(2) strip off syslog priority when logging to console.
OK deraadt@ millert@

The pointer buf is a user space string which was directly passed
to tputchar() and could crash the kernel. Better use cnwrite() in
sendsyslog2() for writing to console. It takes a struct uio which
does the copyin() automatically. In addition cnwrite() outputs to
the real console or to a redirected one, whichever is appropriate.
One drawback is that the syslog priority cannot be stripped off
easily.
OK deraadt@

remove stale lint annotations

Add sendsyslog2(), which accepts the syslog/openlog "logopt" flag
LOG_CONS. If syslogd is not accepting messages, direct them to the console.
This allows us to remove the direct /dev/console opening code from the
bowels of libc. Of course, that forgotten code was exposed by pledge.
ok kettenis millert beck

The must be no space after the syslog priority in the sendsyslog(2)
dropped message error log.
OK benno@

To make logging to local syslog reliable, log a message about failed
log atempts. sendsyslog(2) is a good place to detect and report
the problem.
OK deraadt@

Pass a thread pointer instead of its file descriptor table to getsock(9).

Diff from Vitaliy Makkoveev.

Manpage tweak and ok millert@

Remove some includes include-what-you-use claims don't
have any direct symbols used. Tested for indirect use by compiling
amd64/i386/sparc64 kernels.

ok tedu@ deraadt@

First step towards making uiomove() take a size_t size argument:
- rename uiomove() to uiomovei() and update all its users.
- introduce uiomove(), which is similar to uiomovei() but with a size_t.
- rewrite uiomovei() as an uiomove() wrapper.
ok kettenis@

Many architectures call initmsgbuf() really really early, before uvm is
initialized. Calling malloc(9) at that point is not a good idea. So
initialize consbuf later.

Fixes dmesg -s on sparc64 (and probably a few other architectures).

ok miod@, deraadt@

Add dmesg -s support, to view the output of rc(8) system startup messages.
Help and feedback by Theo and Miod.
OK deraadt@, manpage-ok jmc@

yet more mallocarray() changes.

ok tedu@ deraadt@

convert bcopy to memcpy. ok millert

pass size argument to free()
ok doug tedu

I/O ktrace of sendsyslog(2) did not work. As uiomove() adjusts
iov_len to 0, we need a propper length calculation. While there,
use -1 for the file descriptor because 0 is reserved for stdin.
OK deraadt@ guenther@

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Add new sendsyslog(const char *, size_t) system call which simply passes
a message up into syslogd's /dev/log interface. This will be used to make
syslog_r work during file descriptor exhaustion, or inside sandboxes which
prohibit socket, connect, sendto, etc.
The system call is being added about a week before the library and daemon
changes.
ok guenther tedu miod matthew

Eliminates struct pcred by moving the real and saved ugids into
struct ucred; struct process then directly links to the ucred

Based on a discussion at c2k10 or so before noting that FreeBSD and
NetBSD did this too.

ok matthew@

bzero -> memset

kqueue attach functions should return an errno or 0, not a plain 1. Fix
the obvious cases to return EINVAL and ENXIO.

ok tedu deraadt

Every selwakeup() should have a matching KNOTE() (even if kqueue isn't
supported it doesn't do any harm), so put the KNOTE() in selwakeup() itself and
remove it from any occurences where both are used, except one for kqueue itself
and one in sys_pipe.c (where the selwakeup is under a PIPE_SEL flag).

Based on a diff from tedu.

ok deraadt

unsupported ioctl's should be returned with ENOTTY
not -1;

ok miod@

ansi and deregister. No binary change.

knf

Try to start log message buffer on a new line. Helps install scripts avoid
detecting phantom devices when scanning dmesg output.

Suggested by miod@, ok deraadt@.

Replace select backends with poll backends. selscan() and pollscan()
now call the poll backend. With this change we implement greater
poll(2) functionality instead of emulating it via the select backend.
Adapted from NetBSD and including some changes from FreeBSD.
Tested by many, deraadt@ OK

remove caddr_t casts. it's just silly to cast something when the function
takes a void *. convert uiomove to take a void * as well. ok deraadt@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

kqfilter for the log. niels said i can commit it if it works
and i converted syslog to libevent and it works now (;

branches: 1.7.6;


Move offsetof define into sys/param.h

enlarge msgbuf, somewhat line netbsd did

branches: 1.5.12;
it is definately wrong to initialize the pgid at open() time because the
uid/euid used for checking in csignal() is controlled by whoever used
TIOCSPGRP previously.

for non-tty TIOCSPGRP/F_SETOWN/FIOSETOWN pgid setting calls, store uid
and euid as well, then deliver them using new csignal() interface
which ensures that pgid setting process is permitted to signal the
pgid process(es). Thanks to newsham@aloha.net for extensive help and
discussion.

partial sync with netbsd 960418, more to come

From NetBSD: 960217 merge

branches: 1.1.1;
Initial revision

Remove unused device poll functions.

Also remove unneeded includes of <sys/poll.h> and <sys/select.h>.

Some addenda from jsg@.

OK miod@ mpi@

Unlock sendsyslog(2). Console output still requires kernel lock to be
held but this path is only followed while `syslogf' socket is not set.

New `syslogf_rwlock' used to protect `syslogf' access.

ok bluhm@

Early daemons like dhcpleased(8), slaacd(8), unwind(8), resolvd(8)
are started before syslogd(8). This resulted in ugly sendsyslog(2)
dropped logs and the real message was lost.
Create a temporary stash for log messages within the kernel. It
has a limited size of 100 messages, and each message is truncated
to 8192 bytes. When the stash is exhausted, the well-known dropped
message is generated with a counter. After syslogd(8) has setup
everything, it sends a debug line through libc to flush the kernel
stash. Then syslogd receives all messages from the kernel before
the usual logs.
OK deraadt@ visa@

Simplify sleep_setup API to two operations in preparation for splitting
the SCHED_LOCK().

Putting a thread on a sleep queue is reduce to the following:

sleep_setup();
/* check condition or release lock */
sleep_finish();

Previous version ok cheloha@, jmatthew@, ok claudio@

Fix boot-time crash on sparc64

On sparc64, initmsgbuf() is invoked before curcpu() is usable
on the boot processor. Consequently, it is unsafe to use mutexes
during the message buffer initialization. Avoid such use by skipping
log_mtx when appending a newline from initmsgbuf().

Use mbp instead of msgbufp as the buffer argument to the putchar routine
for consistency.

Bug reported and fix suggested by miod@

Refactor klist insertion and removal

Rename klist_{insert,remove}() to klist_{insert,remove}_locked().
These functions assume that the caller has locked the klist. The current
state of locking remains intact because the kernel lock is still used
with all klists.

Add new functions klist_insert() and klist_remove() that lock the klist
internally. This allows some code simplification.

OK mpi@

Serialize msgbuf access with a mutex.

This introduces a system-wide mutex that serializes msgbuf operations.
The mutex controls access to all modifiable fields of struct msgbuf.
It also covers logsoftc.sc_state.

To avoid adding extra lock order constraints that would affect use of
printf(9), the code does not take new locks when the log mutex is held.

The code assumes that there is at most one thread using logread(). This
keeps the logic simple. If there was more than one reader, logread()
might return the same data to different readers. Also, log wakeup might
not be reliable with multiple threads.

Tested in snaps for two weeks.

OK mpi@

Fix kn_data returned by filt_logread().

Take into account the circular nature of the message buffer when
computing the number of available bytes. Move the computation into
a separate function and use it with the kevent(2) and ioctl(2)
interfaces.

OK mpi@

Remove an unnecessary field from struct msgbuf.

OK mvs@

Abstract the head of knote lists. This allows extending the lists,
for example, with locking assertions.

OK mpi@, anton@

Replace field f_isfd with field f_flags in struct filterops to allow
adding more filter properties without cluttering the struct.

OK mpi@, anton@

Convert infinite sleeps to tsleep_nsec(9).

ok bluhm@

Unify handling of ioctls FIOSETOWN/SIOCSPGRP/TIOCSPGRP and
FIOGETOWN/SIOCGPGRP/TIOCGPGRP. Do this by determining the meaning of
the ID parameter inside the sigio code. Also add cases for FIOSETOWN
and FIOGETOWN where there have been TIOCSPGRP and TIOCGPGRP before.
These changes allow removing the ID translation from sys_fcntl() and
sys_ioctl().

Idea from NetBSD

OK mpi@, claudio@

Change /dev/log to use pgsigio(9) and sigio_init(9) like it was done for
bpf(4) and tun(4) instead of using hand rolled code using csignal().
OK visa@

Use C99 designated initializers with struct filterops. In addition,
make the structs const so that the data are put in .rodata.

OK mpi@, deraadt@, anton@, bluhm@

The console buffer is allocated during startup. initconsbuf() is
only called from main(). There allocation must not fail, so better
use M_WAITOK and remove error handling. As it is not a temporary
buffer, M_TTYS is more appropriate.
OK deraadt@ mpi@

Make log tick interval independent of hz and reduce the frequency
so that the timeout would not be scheduled on every tick.

Discussed with and OK mpi@

Work around locking issues in logwakeup(). Instead of actually waking up
waiters, just set a flag in logwakeup(). The flag is later noted through
periodic polling. This lets the wakeup code run with sufficient locking.

logwakeup() is a very tricky place to take locks because the function
can be called in many different contexts. By not requiring locks in
the routine helps to keep printf(9) as usable as possible.

OK mpi@

dosendsyslog() must only pass ktrgenio(9) userspace buffers that it can
use copyin() on. While here: just put the struct iovec for ktrace on the
stack instead of mallocing and freeing it.

problem debugged by patrick@
ok deraadt@ mpi@

Use FNONBLOCK instead of SS_NBIO to check/indicate that the I/O mode
for sockets is non-blocking.

This allows us to G/C SS_NBIO. Having to keep the two flags in sync
in a mp-safe way is complicated.

This change introduce a behavior change in sosplice(), it can now
always block. However this should not matter much due to the socket
lock being taken beforhand.

ok bluhm@, benno@, visa@

Remove almost unused `flags' argument of suser().

The account flag `ASU' will no longer be set but that makes suser()
mpsafe since it no longer mess with a per-process field.

No objection from millert@, ok tedu@, bluhm@

Print the pid of the most recent program that failed to send a log
via sendsyslog(2) along with the corresponding errno.

Help when troubleshooting which program is triggering an error, like
an overflow.

ok bluhm@

sendsyslog should take a const char * everywhere.

okay bluhm@, deraadt@

Kernel sendsyslog(2), libc syslog(3), and syslogd(8) restrict and
truncate the length of a syslog message to 8192 bytes. Use one
global define LOG_MAXLINE for all of them.
OK deraadt@ millert@

Both syslog(3) and syslogd(8) truncate the message at 8192 bytes.
Do the same in sendsyslog(2) and document the behavior.
reported by Ilja Van Sprundel; OK millert@ deraadt@

branches: 1.50.4;
Reorder FREF() and FRELE() in a way that the the global variable
syslogf always points to a file object with increased reference
count. This makes the implementation independent from the fact
whether changing the reference counter may sleep.
pointed out by Mateusz Guzik; OK deraadt@

There was a race in dosendsyslog() which resulted in a crash.
sosend(syslogf->f_data, ...) could be called with a NULL pointer.
syslogf was not NULL, f_data was NULL and f_count was 1. The file
structure is ref counted, but the global variable syslogf is not
protected. So it may change during sleep and dosendsyslog() possibly
used a different socket at each access. Solution is to access
syslogf only once, use a local copy, and do the ref counting there.
OK millert@ deraadt@

branches: 1.48.2;
As klog dropped message has no ifdef small kernel, sendsyslog
should not have it either. While there bring some variables in
sync between both functions.
OK deraadt@

It is annoying that the dmesg buffer can overflow and loose messages
undetected during debugging. To make clear what happens, count the
dropped bytes and write message buffer full to syslogd. This also
helps to have a reliable log system.
OK deraadt@ millert@ tedu@

As logging to dmesg with msgbuf_putchar() can happen in any interrupt
handler, every access to msg buf counters should be protected by
splhigh(). This is already done in some places, make it consistenly
everywhere.
OK mpi@ deraadt@

sys_osendsyslog can go away; we have transitioned to the new version with
a flags argument
ok guenther sthen

All msg buf counters are long, so lmin(9) should be used for them.
OK deraadt@ natano@

Kill trailing whitespaces.

Use constty and cn_devvp for checking wether to use cnwrite(). That
makes console redirection with senssyslog(2) and LOG_CONS work
again. Also merge the two if else if else blocks into one.
OK deraadt@

Allow sendsyslog(2) with LOG_CONS even when /dev/console has not
been opened during init(8). Only log with cnwrite() if cn_devvp
exists, otherwise use cnputc() as fallback. While there move extern
declarations to dev/cons.h.
input and OK deraadt@

Backout the previous fix for the sendsyslog(2) with LOG_CONS solution.
Permanently holding /dev/console open in the kernel works only until
init(8) calls revoke(2). After that the console device vnode cannot
be used anymore. It still resulted in a hanging init(8) if it tried
to syslog(3) something. With the backout also dmesg -s works again.

If sendsyslog(2) is called with LOG_CONS before syslogd(8) has been
started and before init(8) has opened the console, the kernel could
crash as the console device has not been initialized. Open
/dev/console in the kernel before starting init(8) and keep it open.
This way sendsyslog(2) can be called early in the system.
OK beck@ deraadt@

Rename the system call sendsyslog2 to sendsyslog. Keep the old one
as osendsyslog for a while. The three argument variant is the only
one that will stay.
input kettenis@; OK deraadt@

Convert to uiomove(); from Martin Natano, thanks!

ok deraadt@

In sendsyslogd(2) strip off syslog priority when logging to console.
OK deraadt@ millert@

The pointer buf is a user space string which was directly passed
to tputchar() and could crash the kernel. Better use cnwrite() in
sendsyslog2() for writing to console. It takes a struct uio which
does the copyin() automatically. In addition cnwrite() outputs to
the real console or to a redirected one, whichever is appropriate.
One drawback is that the syslog priority cannot be stripped off
easily.
OK deraadt@

remove stale lint annotations

Add sendsyslog2(), which accepts the syslog/openlog "logopt" flag
LOG_CONS. If syslogd is not accepting messages, direct them to the console.
This allows us to remove the direct /dev/console opening code from the
bowels of libc. Of course, that forgotten code was exposed by pledge.
ok kettenis millert beck

The must be no space after the syslog priority in the sendsyslog(2)
dropped message error log.
OK benno@

To make logging to local syslog reliable, log a message about failed
log atempts. sendsyslog(2) is a good place to detect and report
the problem.
OK deraadt@

Pass a thread pointer instead of its file descriptor table to getsock(9).

Diff from Vitaliy Makkoveev.

Manpage tweak and ok millert@

Remove some includes include-what-you-use claims don't
have any direct symbols used. Tested for indirect use by compiling
amd64/i386/sparc64 kernels.

ok tedu@ deraadt@

First step towards making uiomove() take a size_t size argument:
- rename uiomove() to uiomovei() and update all its users.
- introduce uiomove(), which is similar to uiomovei() but with a size_t.
- rewrite uiomovei() as an uiomove() wrapper.
ok kettenis@

Many architectures call initmsgbuf() really really early, before uvm is
initialized. Calling malloc(9) at that point is not a good idea. So
initialize consbuf later.

Fixes dmesg -s on sparc64 (and probably a few other architectures).

ok miod@, deraadt@

Add dmesg -s support, to view the output of rc(8) system startup messages.
Help and feedback by Theo and Miod.
OK deraadt@, manpage-ok jmc@

yet more mallocarray() changes.

ok tedu@ deraadt@

convert bcopy to memcpy. ok millert

pass size argument to free()
ok doug tedu

I/O ktrace of sendsyslog(2) did not work. As uiomove() adjusts
iov_len to 0, we need a propper length calculation. While there,
use -1 for the file descriptor because 0 is reserved for stdin.
OK deraadt@ guenther@

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Add new sendsyslog(const char *, size_t) system call which simply passes
a message up into syslogd's /dev/log interface. This will be used to make
syslog_r work during file descriptor exhaustion, or inside sandboxes which
prohibit socket, connect, sendto, etc.
The system call is being added about a week before the library and daemon
changes.
ok guenther tedu miod matthew

Eliminates struct pcred by moving the real and saved ugids into
struct ucred; struct process then directly links to the ucred

Based on a discussion at c2k10 or so before noting that FreeBSD and
NetBSD did this too.

ok matthew@

bzero -> memset

kqueue attach functions should return an errno or 0, not a plain 1. Fix
the obvious cases to return EINVAL and ENXIO.

ok tedu deraadt

Every selwakeup() should have a matching KNOTE() (even if kqueue isn't
supported it doesn't do any harm), so put the KNOTE() in selwakeup() itself and
remove it from any occurences where both are used, except one for kqueue itself
and one in sys_pipe.c (where the selwakeup is under a PIPE_SEL flag).

Based on a diff from tedu.

ok deraadt

unsupported ioctl's should be returned with ENOTTY
not -1;

ok miod@

ansi and deregister. No binary change.

knf

Try to start log message buffer on a new line. Helps install scripts avoid
detecting phantom devices when scanning dmesg output.

Suggested by miod@, ok deraadt@.

Replace select backends with poll backends. selscan() and pollscan()
now call the poll backend. With this change we implement greater
poll(2) functionality instead of emulating it via the select backend.
Adapted from NetBSD and including some changes from FreeBSD.
Tested by many, deraadt@ OK

remove caddr_t casts. it's just silly to cast something when the function
takes a void *. convert uiomove to take a void * as well. ok deraadt@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

kqfilter for the log. niels said i can commit it if it works
and i converted syslog to libevent and it works now (;

branches: 1.7.6;


Move offsetof define into sys/param.h

enlarge msgbuf, somewhat line netbsd did

branches: 1.5.12;
it is definately wrong to initialize the pgid at open() time because the
uid/euid used for checking in csignal() is controlled by whoever used
TIOCSPGRP previously.

for non-tty TIOCSPGRP/F_SETOWN/FIOSETOWN pgid setting calls, store uid
and euid as well, then deliver them using new csignal() interface
which ensures that pgid setting process is permitted to signal the
pgid process(es). Thanks to newsham@aloha.net for extensive help and
discussion.

partial sync with netbsd 960418, more to come

From NetBSD: 960217 merge

branches: 1.1.1;
Initial revision

Unlock sendsyslog(2). Console output still requires kernel lock to be
held but this path is only followed while `syslogf' socket is not set.

New `syslogf_rwlock' used to protect `syslogf' access.

ok bluhm@

Early daemons like dhcpleased(8), slaacd(8), unwind(8), resolvd(8)
are started before syslogd(8). This resulted in ugly sendsyslog(2)
dropped logs and the real message was lost.
Create a temporary stash for log messages within the kernel. It
has a limited size of 100 messages, and each message is truncated
to 8192 bytes. When the stash is exhausted, the well-known dropped
message is generated with a counter. After syslogd(8) has setup
everything, it sends a debug line through libc to flush the kernel
stash. Then syslogd receives all messages from the kernel before
the usual logs.
OK deraadt@ visa@

Simplify sleep_setup API to two operations in preparation for splitting
the SCHED_LOCK().

Putting a thread on a sleep queue is reduce to the following:

sleep_setup();
/* check condition or release lock */
sleep_finish();

Previous version ok cheloha@, jmatthew@, ok claudio@

Fix boot-time crash on sparc64

On sparc64, initmsgbuf() is invoked before curcpu() is usable
on the boot processor. Consequently, it is unsafe to use mutexes
during the message buffer initialization. Avoid such use by skipping
log_mtx when appending a newline from initmsgbuf().

Use mbp instead of msgbufp as the buffer argument to the putchar routine
for consistency.

Bug reported and fix suggested by miod@

Refactor klist insertion and removal

Rename klist_{insert,remove}() to klist_{insert,remove}_locked().
These functions assume that the caller has locked the klist. The current
state of locking remains intact because the kernel lock is still used
with all klists.

Add new functions klist_insert() and klist_remove() that lock the klist
internally. This allows some code simplification.

OK mpi@

Serialize msgbuf access with a mutex.

This introduces a system-wide mutex that serializes msgbuf operations.
The mutex controls access to all modifiable fields of struct msgbuf.
It also covers logsoftc.sc_state.

To avoid adding extra lock order constraints that would affect use of
printf(9), the code does not take new locks when the log mutex is held.

The code assumes that there is at most one thread using logread(). This
keeps the logic simple. If there was more than one reader, logread()
might return the same data to different readers. Also, log wakeup might
not be reliable with multiple threads.

Tested in snaps for two weeks.

OK mpi@

Fix kn_data returned by filt_logread().

Take into account the circular nature of the message buffer when
computing the number of available bytes. Move the computation into
a separate function and use it with the kevent(2) and ioctl(2)
interfaces.

OK mpi@

Remove an unnecessary field from struct msgbuf.

OK mvs@

Abstract the head of knote lists. This allows extending the lists,
for example, with locking assertions.

OK mpi@, anton@

Replace field f_isfd with field f_flags in struct filterops to allow
adding more filter properties without cluttering the struct.

OK mpi@, anton@

Convert infinite sleeps to tsleep_nsec(9).

ok bluhm@

Unify handling of ioctls FIOSETOWN/SIOCSPGRP/TIOCSPGRP and
FIOGETOWN/SIOCGPGRP/TIOCGPGRP. Do this by determining the meaning of
the ID parameter inside the sigio code. Also add cases for FIOSETOWN
and FIOGETOWN where there have been TIOCSPGRP and TIOCGPGRP before.
These changes allow removing the ID translation from sys_fcntl() and
sys_ioctl().

Idea from NetBSD

OK mpi@, claudio@

Change /dev/log to use pgsigio(9) and sigio_init(9) like it was done for
bpf(4) and tun(4) instead of using hand rolled code using csignal().
OK visa@

Use C99 designated initializers with struct filterops. In addition,
make the structs const so that the data are put in .rodata.

OK mpi@, deraadt@, anton@, bluhm@

The console buffer is allocated during startup. initconsbuf() is
only called from main(). There allocation must not fail, so better
use M_WAITOK and remove error handling. As it is not a temporary
buffer, M_TTYS is more appropriate.
OK deraadt@ mpi@

Make log tick interval independent of hz and reduce the frequency
so that the timeout would not be scheduled on every tick.

Discussed with and OK mpi@

Work around locking issues in logwakeup(). Instead of actually waking up
waiters, just set a flag in logwakeup(). The flag is later noted through
periodic polling. This lets the wakeup code run with sufficient locking.

logwakeup() is a very tricky place to take locks because the function
can be called in many different contexts. By not requiring locks in
the routine helps to keep printf(9) as usable as possible.

OK mpi@

dosendsyslog() must only pass ktrgenio(9) userspace buffers that it can
use copyin() on. While here: just put the struct iovec for ktrace on the
stack instead of mallocing and freeing it.

problem debugged by patrick@
ok deraadt@ mpi@

Use FNONBLOCK instead of SS_NBIO to check/indicate that the I/O mode
for sockets is non-blocking.

This allows us to G/C SS_NBIO. Having to keep the two flags in sync
in a mp-safe way is complicated.

This change introduce a behavior change in sosplice(), it can now
always block. However this should not matter much due to the socket
lock being taken beforhand.

ok bluhm@, benno@, visa@

Remove almost unused `flags' argument of suser().

The account flag `ASU' will no longer be set but that makes suser()
mpsafe since it no longer mess with a per-process field.

No objection from millert@, ok tedu@, bluhm@

Print the pid of the most recent program that failed to send a log
via sendsyslog(2) along with the corresponding errno.

Help when troubleshooting which program is triggering an error, like
an overflow.

ok bluhm@

sendsyslog should take a const char * everywhere.

okay bluhm@, deraadt@

Kernel sendsyslog(2), libc syslog(3), and syslogd(8) restrict and
truncate the length of a syslog message to 8192 bytes. Use one
global define LOG_MAXLINE for all of them.
OK deraadt@ millert@

Both syslog(3) and syslogd(8) truncate the message at 8192 bytes.
Do the same in sendsyslog(2) and document the behavior.
reported by Ilja Van Sprundel; OK millert@ deraadt@

branches: 1.50.4;
Reorder FREF() and FRELE() in a way that the the global variable
syslogf always points to a file object with increased reference
count. This makes the implementation independent from the fact
whether changing the reference counter may sleep.
pointed out by Mateusz Guzik; OK deraadt@

There was a race in dosendsyslog() which resulted in a crash.
sosend(syslogf->f_data, ...) could be called with a NULL pointer.
syslogf was not NULL, f_data was NULL and f_count was 1. The file
structure is ref counted, but the global variable syslogf is not
protected. So it may change during sleep and dosendsyslog() possibly
used a different socket at each access. Solution is to access
syslogf only once, use a local copy, and do the ref counting there.
OK millert@ deraadt@

branches: 1.48.2;
As klog dropped message has no ifdef small kernel, sendsyslog
should not have it either. While there bring some variables in
sync between both functions.
OK deraadt@

It is annoying that the dmesg buffer can overflow and loose messages
undetected during debugging. To make clear what happens, count the
dropped bytes and write message buffer full to syslogd. This also
helps to have a reliable log system.
OK deraadt@ millert@ tedu@

As logging to dmesg with msgbuf_putchar() can happen in any interrupt
handler, every access to msg buf counters should be protected by
splhigh(). This is already done in some places, make it consistenly
everywhere.
OK mpi@ deraadt@

sys_osendsyslog can go away; we have transitioned to the new version with
a flags argument
ok guenther sthen

All msg buf counters are long, so lmin(9) should be used for them.
OK deraadt@ natano@

Kill trailing whitespaces.

Use constty and cn_devvp for checking wether to use cnwrite(). That
makes console redirection with senssyslog(2) and LOG_CONS work
again. Also merge the two if else if else blocks into one.
OK deraadt@

Allow sendsyslog(2) with LOG_CONS even when /dev/console has not
been opened during init(8). Only log with cnwrite() if cn_devvp
exists, otherwise use cnputc() as fallback. While there move extern
declarations to dev/cons.h.
input and OK deraadt@

Backout the previous fix for the sendsyslog(2) with LOG_CONS solution.
Permanently holding /dev/console open in the kernel works only until
init(8) calls revoke(2). After that the console device vnode cannot
be used anymore. It still resulted in a hanging init(8) if it tried
to syslog(3) something. With the backout also dmesg -s works again.

If sendsyslog(2) is called with LOG_CONS before syslogd(8) has been
started and before init(8) has opened the console, the kernel could
crash as the console device has not been initialized. Open
/dev/console in the kernel before starting init(8) and keep it open.
This way sendsyslog(2) can be called early in the system.
OK beck@ deraadt@

Rename the system call sendsyslog2 to sendsyslog. Keep the old one
as osendsyslog for a while. The three argument variant is the only
one that will stay.
input kettenis@; OK deraadt@

Convert to uiomove(); from Martin Natano, thanks!

ok deraadt@

In sendsyslogd(2) strip off syslog priority when logging to console.
OK deraadt@ millert@

The pointer buf is a user space string which was directly passed
to tputchar() and could crash the kernel. Better use cnwrite() in
sendsyslog2() for writing to console. It takes a struct uio which
does the copyin() automatically. In addition cnwrite() outputs to
the real console or to a redirected one, whichever is appropriate.
One drawback is that the syslog priority cannot be stripped off
easily.
OK deraadt@

remove stale lint annotations

Add sendsyslog2(), which accepts the syslog/openlog "logopt" flag
LOG_CONS. If syslogd is not accepting messages, direct them to the console.
This allows us to remove the direct /dev/console opening code from the
bowels of libc. Of course, that forgotten code was exposed by pledge.
ok kettenis millert beck

The must be no space after the syslog priority in the sendsyslog(2)
dropped message error log.
OK benno@

To make logging to local syslog reliable, log a message about failed
log atempts. sendsyslog(2) is a good place to detect and report
the problem.
OK deraadt@

Pass a thread pointer instead of its file descriptor table to getsock(9).

Diff from Vitaliy Makkoveev.

Manpage tweak and ok millert@

Remove some includes include-what-you-use claims don't
have any direct symbols used. Tested for indirect use by compiling
amd64/i386/sparc64 kernels.

ok tedu@ deraadt@

First step towards making uiomove() take a size_t size argument:
- rename uiomove() to uiomovei() and update all its users.
- introduce uiomove(), which is similar to uiomovei() but with a size_t.
- rewrite uiomovei() as an uiomove() wrapper.
ok kettenis@

Many architectures call initmsgbuf() really really early, before uvm is
initialized. Calling malloc(9) at that point is not a good idea. So
initialize consbuf later.

Fixes dmesg -s on sparc64 (and probably a few other architectures).

ok miod@, deraadt@

Add dmesg -s support, to view the output of rc(8) system startup messages.
Help and feedback by Theo and Miod.
OK deraadt@, manpage-ok jmc@

yet more mallocarray() changes.

ok tedu@ deraadt@

convert bcopy to memcpy. ok millert

pass size argument to free()
ok doug tedu

I/O ktrace of sendsyslog(2) did not work. As uiomove() adjusts
iov_len to 0, we need a propper length calculation. While there,
use -1 for the file descriptor because 0 is reserved for stdin.
OK deraadt@ guenther@

add a size argument to free. will be used soon, but for now default to 0.
after discussions with beck deraadt kettenis.

Add new sendsyslog(const char *, size_t) system call which simply passes
a message up into syslogd's /dev/log interface. This will be used to make
syslog_r work during file descriptor exhaustion, or inside sandboxes which
prohibit socket, connect, sendto, etc.
The system call is being added about a week before the library and daemon
changes.
ok guenther tedu miod matthew

Eliminates struct pcred by moving the real and saved ugids into
struct ucred; struct process then directly links to the ucred

Based on a discussion at c2k10 or so before noting that FreeBSD and
NetBSD did this too.

ok matthew@

bzero -> memset

kqueue attach functions should return an errno or 0, not a plain 1. Fix
the obvious cases to return EINVAL and ENXIO.

ok tedu deraadt

Every selwakeup() should have a matching KNOTE() (even if kqueue isn't
supported it doesn't do any harm), so put the KNOTE() in selwakeup() itself and
remove it from any occurences where both are used, except one for kqueue itself
and one in sys_pipe.c (where the selwakeup is under a PIPE_SEL flag).

Based on a diff from tedu.

ok deraadt

unsupported ioctl's should be returned with ENOTTY
not -1;

ok miod@

ansi and deregister. No binary change.

knf

Try to start log message buffer on a new line. Helps install scripts avoid
detecting phantom devices when scanning dmesg output.

Suggested by miod@, ok deraadt@.

Replace select backends with poll backends. selscan() and pollscan()
now call the poll backend. With this change we implement greater
poll(2) functionality instead of emulating it via the select backend.
Adapted from NetBSD and including some changes from FreeBSD.
Tested by many, deraadt@ OK

remove caddr_t casts. it's just silly to cast something when the function
takes a void *. convert uiomove to take a void * as well. ok deraadt@

Remove the advertising clause in the UCB license which Berkeley
rescinded 22 July 1999. Proofed by myself and Theo.

kqfilter for the log. niels said i can commit it if it works
and i converted syslog to libevent and it works now (;

branches: 1.7.6;


Move offsetof define into sys/param.h

enlarge msgbuf, somewhat line netbsd did

branches: 1.5.12;
it is definately wrong to initialize the pgid at open() time because the
uid/euid used for checking in csignal() is controlled by whoever used
TIOCSPGRP previously.

for non-tty TIOCSPGRP/F_SETOWN/FIOSETOWN pgid setting calls, store uid
and euid as well, then deliver them using new csignal() interface
which ensures that pgid setting process is permitted to signal the
pgid process(es). Thanks to newsham@aloha.net for extensive help and
discussion.

partial sync with netbsd 960418, more to come

From NetBSD: 960217 merge

branches: 1.1.1;
Initial revision

Early daemons like dhcpleased(8), slaacd(8), unwind(8), resolvd(8)
are started before syslogd(8). This resulted in ugly sendsyslog(2)
dropped logs and the real message was lost.
Create a temporary stash for log messages within the kernel. It
has a limited size of 100 messages, and each message is truncated
to 8192 bytes. When the stash is exhausted, the well-known dropped
message is generated with a counter. After syslogd(8) has setup
everything, it sends a debug line through libc to flush the kernel
stash. Then syslogd receives all messages from the kernel before
the usual logs.
OK deraadt@ visa@

Simplify sleep_setup API to two operations in preparation for splitting
the SCHED_LOCK().

Putting a thread on a sleep queue is reduce to the following:

sleep_setup();
/* check condition or release lock */
sleep_finish();

Previous version ok cheloha@, jmatthew@, ok claudio@

Fix boot-time crash on sparc64

On sparc64, initmsgbuf() is invoked before curcpu() is usable
on the boot processor. Consequently, it is unsafe to use mutexes
during the message buffer initialization. Avoid such use by skipping
log_mtx when appending a newline from initmsgbuf().

Use mbp instead of msgbufp as the buffer argument to the putchar routine
for consistency.

Bug reported and fix suggested by miod@

Refactor klist insertion and removal

Rename klist_{insert,remove}() to klist_{insert,remove}_locked().
These functions assume that the caller has locked the klist. The current
state of locking remains intact because the kernel lock is still used
with all klists.

Add new functions klist_insert() and klist_remove() that lock the klist
internally. This allows some code simplification.

OK mpi@

Serialize msgbuf access with a mutex.

This introduces a system-wide mutex that serializes msgbuf operations.
The mutex controls access to all modifiable fields of struct msgbuf.
It also covers logsoftc.sc_state.

To avoid adding extra lock order constraints that would affect use of
printf(9), the code does not take new locks when the log mutex is held.

The code assumes that there is at most one thread using logread(). This
keeps the logic simple. If there was more than one reader, logread()
might return the same data to different readers. Also, log wakeup might
not be reliable with multiple threads.

Tested in snaps for two weeks.

OK mpi@

Fix kn_data returned by filt_logread().

Take into account the circular nature of the message buffer when
computing the number of available bytes. Move the computation into
a separate function and use it with the kevent(2) and ioctl(2)
interfaces.

OK mpi@

Remove an unnecessary field from struct msgbuf.

OK mvs@

Abstract the head of knote lists. This allows extending the lists,
for example, with locking assertions.

OK mpi@, anton@

Replace field f_isfd with field f_flags in struct filterops to allow
adding more filter properties without cluttering the struct.

OK mpi@, anton@

Convert infinite sleeps to tsleep_nsec(9).

ok bluhm@

Unify handling of ioctls FIOSETOWN/SIOCSPGRP/TIOCSPGRP and
FIOGETOWN/SIOCGPGRP/TIOCGPGRP. Do this by determining the meaning of
the ID parameter inside the sigio code. Also add cases for FIOSETOWN
and FIOGETOWN where there have been TIOCSPGRP and TIOCGPGRP before.
These changes allow removing the ID translation from sys_fcntl() and
sys_ioctl().

Idea from NetBSD

OK mpi@, claudio@

Change /dev/log to use pgsigio(9) and sigio_init(9) like it was done for
bpf(4) and tun(4) instead of using hand rolled code using csignal().
OK visa@