Move the initialization of the random hash for addresses earlier so that
it does not happen under a spin lock context (when it is first used).

KNF (sort #includes and remove duplicate sys/cpu.h)

Move proc_lock into the data segment. It was dynamically allocated because
at the time we had mutex_obj_alloc() but not __cacheline_aligned.

Merge vfs_cache.c from the ad-namecache branch. With this the namecache
index becomes per-directory (initially, a red-black tree). The remaining
changes on the branch to namei()/getcwd() will be merged in the future.

Use 4K pages on ARM_MMU_EXTENDED platforms (all armv[67] except RPI) by
creating a new pool l1ttpl for the userland L1 translation table which
needs to be 8KB and 8KB aligned.

Limit the pool to maxproc and add hooks to allow the sysctl changing of
maxproc to adjust the pool.

This comes at a 5% performance penalty for build.sh -j8 kernel on a
Tegra TK1.

branches: 1.223.2;
- Eliminate the global "boottime" variable, which was being accessed
without any synchronization against changes by e.g. clock_settime().
- Replace with new getbinboottime() / getnanoboottime() / getmicroboottime()
functions (naming mirrors that of other time access functions in kern_tc.c).
It returns the (maybe-converted) value of timebasebin, which also tracks
our estimate of when the system was booted (i.e. the legacy "boottime" was
redundant).

XXX There needs to be a lockless synchronization mechanism for reading
timebasebin, but this is a problem in kern_tc.c that pre-existed these
"boottime" changes. At least now the problem is centralized in one location.

remove kern.panic_now -- crashme panic node replaces it.

As discussed in tech-kern:

- make sysctl kern.expose_address tri-state:
0: no access
1: access to processes with open /dev/kmem
2: access to everyone
defaults:
0: KASLR kernels
1: non-KASLR kernels

- improve efficiency by calling get_expose_address() per sysctl, not per
process.

- don't expose addresses for linux procfs

- welcome to 8.99.27, changes to fill_*proc ABI

Expose addresses depending on the KASLR setting (from mrg@). Restores the
status quo of exposing kernel addresses if there is no KASLR.

Fix kernel pointer leaks in the kern.lwp sysctl.

Provide a sysctl kern.expose_address to expose kernel addresses in
sysctl structure returns for non-root. Defaults to off. Turning it
on will restore sockstat/fstat and friends for regular users.

CTL_DEBUG_MAXID is only used to size a static array that the compiler
can do just fine itself. use the compiler and remove the define.

Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)

- Cleanup for dynamic sysctl:
- Remove unused *_NAMES macros for sysctl.
- Remove unused *_MAXID for sysctls.
- Move CTL_MACHDEP sysctl definitions for m68k into m68k/include/cpu.h and
use them on all m68k machines.

branches: 1.214.2; 1.214.4;
Add a proper defflag for GPROF, and include opt_gprof.h, otherwise we're
not gonna go very far.

remove checks for failure after memory allocation calls that cannot fail:

kmem_alloc() with KM_SLEEP
kmem_zalloc() with KM_SLEEP
percpu_alloc()
pserialize_create()
psref_class_create()

all of these paths include an assertion that the allocation has not failed,
so callers should not assert that again.

Remove the "target" argment from vfs_drainvnodes() as it is
always equal to "desiredvnodes" and move its definition
from sys/vnode.h to sys/vnode_impl.h.

Extend vfs_drainvnodes() to also wait for deferred vrele to flush
and replace the call to vrele_flush() with a call to vfs_drainvnodes().

branches: 1.211.2;
Add a new kern.messages sysctl to allow kernel message verbosity to be
altered after boot.

Fixes PR kern/46539 using patch submitted by Nat Sloss.

Whether or not the semaphore code is loaded as a module or built-in, its
sysctl data belongs with the module code. Move it from kern/init_sysctl.c
to kern/uipc_sem.c

While here, add a new sysctl variable kern.posix.semcnt (current count of
semaphores) to complement the existing kern.posix.semmax (maximum number
of semaphores).

Move a bunch of sysctl nodes from init_sysctl (kitchen sink sysctl file)
to init_sysctl_base (only base kernel defs). Main motivation was to
fix sysconf(_SC_NPROCESSORS) for Rumprun. As reported by neeraj on irc,
it returned -1 before this fix, so we were doing imaginary computing.

Move hw.machine and hw.machine_arch sysctls to base so rump can use them

This allows uname(3) and uname(1) to work on rump kernels.

group msgbuf sysctls with the msgbuf code
(init_sysctl.c -> subr_log.c)

More preparation for modularizing the SYSVxxx options. Here we
change the kern.ipc.sysvxxx sysctls into dynamic values, so each
sub-component of SYSVxxx can declare its own availability.

move clock sysctls from init_sysctl.c to kern_clock.c

branches: 1.204.4;
BUILDINFO part 2: expose sysctl kern.buildinfo

Add a global vnode cache:

- vcache_get() retrieves a referenced and initialised vnode / fs node pair.
- vcache_remove() removes a vnode / fs node pair from the cache.

On cache miss vcache_get() calls new vfs operation vfs_loadvnode() to
initialise a vnode / fs node pair. This call is guaranteed exclusive,
no other thread will try to load this vnode / fs node pair.

Convert ufs/ext2fs, ufs/ffs and ufs/mfs to use this interface.

Remove now unused ufs/ufs_ihash

Discussed on tech-kern.

Welcome to 6.99.41

branches: 1.202.2;
- create cpu_{g,s}etmodel() and hide cpu_model from direct access.

Ensure that the top level sysctl nodes (kern, vfs, net, ...) exist before
the sysctl link sets are processed, and remove redundancy.

Shaves >13kB off of an amd64 GENERIC, not to mention >1k duplicate
lines of code.

Add kern.{ostype,osrelease,osrevision,version} kern.domainname,
kern.rawpartition sysctl support to rump kernel.
Moved the sysctl support that is shared between rump and normal
kernels to init_sysctl_base.c as rump cannot use init_sysctl.c
in order to avoid code duplication. Agreed with pooka@.

Put cprng sysctls into subr_cprng.c. Also, make sysctl_prng static
in subr_cprng and get rid of SYSCTL_PRIVATE namespace leak macro.

Fixes ping(8) when run against a standalone rump kernel due to appearance
of the kern.urandom sysctl node (in case someone was wondering ...)

GC various arrays defined and used in kern_proc.c

branches: 1.197.6;
calculate vnode cache size based on the resource it gets allocated from
this stops setting kern.maxvnodes to high so it exhausts available space in kmem

http://mail-index.netbsd.org/tech-kern/2013/03/08/msg015095.html

Add a kern.configname sysctl object.

Move boottime50 and its associated sysctl into the compat module. As
noted on tech-kern. Should fix PR/47579.

OK christos@

Will request pull-up to 6.0 in a few days.

Make the inclusion of <sys/cprng.h> a private matter for sysctl. No reason
to expose the rest of the kernel to it.

split device_t/softc for all remaining drivers.
replace "struct device *" with "device_t".
use device_xname(), device_unit(), etc.

put all kern socket sysctls in the same place

Add sanity check to sysctl_kern_maxvnodes.

branches: 1.190.2;
Add some pre-processor magic to verify that the type of the data item
passed to sysctl_createv() actually matches the declared type for
the item itself.
In the places where the caller specifies a function and a structure
address (typically the 'softc') an explicit (void *) cast is now needed.
Fixes bugs in sys/dev/acpi/asus_acpi.c sys/dev/bluetooth/bcsp.c
sys/kern/vfs_bio.c sys/miscfs/syncfs/sync_subr.c and setting
AcpiGbl_EnableAmlDebugObject.
(mostly passing the address of a uint64_t when typed as CTLTYPE_INT).
I've test built quite a few kernels, but there may be some unfixed MD
fallout. Most likely passing &char[] to char *.
Also add CTLFLAG_UNSIGNED for unsiged decimals - not set yet.

remove bogus check.

P1003_1B_SEMAPHORE is no longer optional.

Remove COMPAT_SA / KERN_SA. Welcome to 6.99.3!
Approved by core@.

branches: 1.186.2;

Separate /dev/random pseudodevice implemenation from kernel entropy pool
implementation. Rewrite pseudodevice code to use cprng_strong(9).

The new pseudodevice is cloning, so each caller gets bits from a stream
generated with its own key. Users of /dev/urandom get their generators
keyed on a "best effort" basis -- the kernel will rekey generators
whenever the entropy pool hits the high water mark -- while users of
/dev/random get their generators rekeyed every time key-length bits
are output.

The underlying cprng_strong API can use AES-256 or AES-128, but we use
AES-128 because of concerns about related-key attacks on AES-256. This
improves performance (and reduces entropy pool depletion) significantly
for users of /dev/urandom but does cause users of /dev/random to rekey
twice as often.

Also fixes various bugs (including some missing locking and a reseed-counter
overflow in the CTR_DRBG code) found while testing this.

For long reads, this generator is approximately 20 times as fast as the
old generator (dd with bs=64K yields 53MB/sec on 2Ghz Core2 instead of
2.5MB/sec) and also uses a separate mutex per instance so concurrency
is greatly improved. For reads of typical key sizes for modern
cryptosystems (16-32 bytes) performance is about the same as the old
code: a little better for 32 bytes, a little worse for 16 bytes.

branches: 1.185.2;
An undocumented behavior of the sysctl kern.arandom node used to allow
sucking up to 8192 bytes out of the kernel arc4random() generator at a
time. Supposedly some very old application code uses this to rekey
other instances of RC4 in userspace (a truly great idea). Reduce the
limit to 256 bytes -- and note that it will probably be reduced to
sizeof(int) in the future, since this node is so documented.

First step of random number subsystem rework described in
<20111022023242.BA26F14A158@mail.netbsd.org>. This change includes
the following:

An initial cleanup and minor reorganization of the entropy pool
code in sys/dev/rnd.c and sys/dev/rndpool.c. Several bugs are
fixed. Some effort is made to accumulate entropy more quickly at
boot time.

A generic interface, "rndsink", is added, for stream generators to
request that they be re-keyed with good quality entropy from the pool
as soon as it is available.

The arc4random()/arc4randbytes() implementation in libkern is
adjusted to use the rndsink interface for rekeying, which helps
address the problem of low-quality keys at boot time.

An implementation of the FIPS 140-2 statistical tests for random
number generator quality is provided (libkern/rngtest.c). This
is based on Greg Rose's implementation from Qualcomm.

A new random stream generator, nist_ctr_drbg, is provided. It is
based on an implementation of the NIST SP800-90 CTR_DRBG by
Henric Jungheim. This generator users AES in a modified counter
mode to generate a backtracking-resistant random stream.

An abstraction layer, "cprng", is provided for in-kernel consumers
of randomness. The arc4random/arc4randbytes API is deprecated for
in-kernel use. It is replaced by "cprng_strong". The current
cprng_fast implementation wraps the existing arc4random
implementation. The current cprng_strong implementation wraps the
new CTR_DRBG implementation. Both interfaces are rekeyed from
the entropy pool automatically at intervals justifiable from best
current cryptographic practice.

In some quick tests, cprng_fast() is about the same speed as
the old arc4randbytes(), and cprng_strong() is about 20% faster
than rnd_extract_data(). Performance is expected to improve.

The AES code in src/crypto/rijndael is no longer an optional
kernel component, as it is required by cprng_strong, which is
not an optional kernel component.

The entropy pool output is subjected to the rngtest tests at
startup time; if it fails, the system will reboot. There is
approximately a 3/10000 chance of a false positive from these
tests. Entropy pool _input_ from hardware random numbers is
subjected to the rngtest tests at attach time, as well as the
FIPS continuous-output test, to detect bad or stuck hardware
RNGs; if any are detected, they are detached, but the system
continues to run.

A problem with rndctl(8) is fixed -- datastructures with
pointers in arrays are no longer passed to userspace (this
was not a security problem, but rather a major issue for
compat32). A new kernel will require a new rndctl.

The sysctl kern.arandom() and kern.urandom() nodes are hooked
up to the new generators, but the /dev/*random pseudodevices
are not, yet.

Manual pages for the new kernel interfaces are forthcoming.

branches: 1.183.2;
Add getlabelusesmbr(), as proposed in
http://mail-index.netbsd.org/tech-userlevel/2011/08/25/msg005404.html
This is used by disk tools such as disklabel(8) to dynamically decide is
the undelyling platform uses a disklabel-in-mbr-partition or not
(instead of using a compile-time list of ports).
getlabelusesmbr() reads the sysctl kern.labelusesmbr, takes its value from the
machdep #define LABELUSESMBR.
For evbmips, make LABELUSESMBR 1 if the platform uses pmon
as bootloader, and 0 (the previous value) otherwise.

When KERN_SA is not defined, kern.no_sa_support is a constant (1). So
add CTLFLAG_IMMEDIATE to flags. Make the macro block logically reversed so
it looks more natural when reading.

Reported by Peter Tworek on tech-kern@.

Add some needed __UNCONST

vfs_drainvnodes: drop lwp argument, remove variable name in prototype.

avoid code duplication.

migrate compat32 handling with previous

pointed out by Lars Heidieker

Move sysctl routines from init_sysctl.c to kern_descrip.c (for
descriptors) and kern_proc.c (for processes). This makes them
usable in a rump kernel, in case somebody was wondering.

Use the L_ flags instead of the P_ flags for lwps.

branches: 1.175.2; 1.175.4;
Remove pfind() and pgfind(), fix locking in various broken uses of these.
Rename real routines to proc_find() and pgrp_find(), remove PFIND_* flags
and have consistent behaviour. Provide proc_find_raw() for special cases.
Fix memory leak in sysctl_proc_corename().

COMPAT_LINUX: rework ptrace() locking, minimise differences between
different versions per-arch.

Note: while this change adds some formal cosmetics for COMPAT_DARWIN and
COMPAT_IRIX - locking there is utterly broken (for ages).

Fixes PR/43176.

Set kinfo_lwp to 0 before filling it so that if someone removes
variable assignments from here, kernel memory does not leak to
userspace.

Bug found, a little bit suprisingly, by the atf ps test which failed
due to the column width between the -o holdcnt column being too
wide due to the contents displayed being garbage.

branches: 1.173.2;
sysctl_doeproc: don't follow a possibly stale pointer.

branches: 1.172.2;
Minimize unnecessary differences in rump.

When reporting open files using sysctl, don't use 'filehead' to fetch files,
as we don't have a process context to authorize on. Instead, traverse the
file descriptor table of each process -- as we already do in one case.

Introduce a "marker" we can use to mark files we've seen in an iteration, as
the same file can be referenced more than once.

Hopefully this availability of filtering by process also makes life easier
for those who are interested in implementing process "containers" etc.

Report L_INMEM in the lwp info as well.

Always set L_INMEM to maintain binary compatibility.

Remove uarea swap-out functionality:

- Addresses the issue described in PR/38828.
- Some simplification in threading and sleepq subsystems.
- Eliminates pmap_collect() and, as a side note, allows pmap optimisations.
- Eliminates XS_CTL_DATA_ONSTACK in scsipi code.
- Avoids few scans on LWP list and thus potentially long holds of proc_lock.
- Cuts ~1.5k lines of code. Reduces amd64 kernel size by ~4k.
- Removes __SWAP_BROKEN cases.

Tested on x86, mips, acorn32 (thanks <mpumford>) and partly tested on
acorn26 (thanks to <bjh21>).

Discussed on <tech-kern>, reviewed by <ad>.

Chop init_sysctl into base nodes (init_sysctl_base.c) and the
kitchen sink (init_sysctl.c). Further surgery may be needed down
the line.

Expose the kernel's boothowto(9) variable through the sysctl
kern.boothowto variable.

Part of the /etc/rc silent changes requested in PR 41946
and proposed in tech-userlevel.

provide compatibility for the older variant of kern.consdev, which used
a 32 bit dev_t. Reported by mrg.

More changes to improve kern_descrip.c.

- Avoid atomics in more places.
- Remove the per-descriptor mutex, and just use filedesc_t::fd_lock.
It was only being used to synchronize close, and in any case we needed
to take fd_lock to free the descriptor slot.
- Optimize certain paths for the <NDFDFILE case.
- Sprinkle more comments and assertions.
- Cache more stuff in filedesc_t.
- Fix numerous minor bugs spotted along the way.
- Restructure how the open files array is maintained, for clarity and so
that we can eliminate the membar_consumer() call in fd_getfile(). This is
mostly syntactic sugar; the main functional change is that fd_nfiles now
lives alongside the open file array.

Some measurements with libmicro:

- simple file syscalls are like close() are between 1 to 10% faster.
- some nice improvements, e.g. poll(1000) which is ~50% faster.

sysctl_doeproc:
- simplify.
- KERN_PROC: fix possible stale proc pointer dereference.
- KERN_PROC: don't do copyout with proc_lock held.

don't forget to skip marker processes.

sysctl_doeproc: fix a bug in rev.1.135.
don't forget to mark our marker process PK_MARKER.
this fixes crashes in sched_pstats, etc.

- add new RLIMIT_AS (aka RLIMIT_VMEM) resource that limits the total
address space available to processes. this limit exists in most other
modern unix variants, and like most of them, our defaults are unlimited.
remove the old mmap / rlimit.datasize hack.

- adds the VMCMD_STACK flag to all the stack-creation vmcmd callers.
it is currently unused, but was added a few years ago.

- add a pair of new process size values to kinfo_proc2{}. one is the
total size of the process memory map, and the other is the total size
adjusted for unused stack space (since most processes have a lot of
this...)

- patch sh, and csh to notice RLIMIT_AS. (in some cases, the alias
RLIMIT_VMEM was already present and used if availble.)

- patch ps, top and systat to notice the new k_vm_vsize member of
kinfo_proc2{}.

- update irix, svr4, svr4_32, linux and osf1 emulations to support
this information. (freebsd could be done, but that it's best left
as part of the full-update of compat/freebsd.)


this addresses PR 7897. it also gives correct memory usage values,
which have never been entirely correct (since mmap), and have been
very incorrect since jemalloc() was enabled.

tested on i386 and sparc64, build tested on several other platforms.

thanks to many folks for feedback and testing but most espcially
chuq and yamt for critical suggestions that lead to this patch not
having a special ugliness i wasn't happy with anyway :-)

like KERN_FILE2: *do* update "needed" when there is no count. we want
userland to know what sort of size to provide..

while here, slightly normalise the previous to init_sysctl.c.

always calculate "needed" for KERN_FILE2 calls. this allows a caller
to get an estimate of the needed space, like the intention is.

Don't bother with file_t::f_iflags any more, as it's not used.
Noted by mrg@.

Use "defopt MODULAR" in sys/conf/files, and #include "opt_modular.h"
in all kernel sources that use the MODULAR option.
Proposed in tech-kern on 18 Jan 2009.

branches: 1.155.2;
whitespace nit

malloc -> kmem_alloc.

merge christos-time_t

Rename specfs_lock as device_lock and move it from specfs to devsw.
Relaxes kernel dependency on vfs.

PR/40002: Daniel Horecki: sockstat doesn't work for user with sysctl
security.curtain=1

If the kauth call failed, we'd silently continue the loop, but the error
code would remain and eventually "leak" to userspace. Reset the error to
zero when continuing.

Tested by snj@ and myself. Okay snj@.

Allow the POSIX semaphore code to be loaded as a module.

branches: 1.149.2; 1.149.4;
Set kern.posix_semaphores are runtime so it can be a module.
(Picked wrong header the last time.)

Set kern.posix_semaphores are runtime so it can be a module.

rename proc_representative_lwp to proc_active_lwp and clarify it is for
ps display purposes. suggested by rmind.

Select a "representative" lwp instead of the first lwp in the list. The
first lwp in the list is the last created and in the firefox and gtk-gnash
case this is usually a zombie, so the status in ps was ZLl. This now picks
the lwp in order ONPROC > RUN > SLEEP > STOP > SUSPENDED > IDL > DEAD > ZOMB
and breaks ties using cpticks.

Merge wrstuden-revivesa into HEAD.

make l_flags contain more stuff. Fixes top thread display where system processes
were always displayed.

branches: 1.143.2;
Remove proc_representative_lwp(), use a simple LIST_FIRST() instead.
OK by <ad>.

PR kern/38927: processes getting stuck in uvm_map (cv_timedwait), hanging
machine

Assume that a vnode (and associated data structures) costs 2kB in the
worst imaginable case. Don't allow sysctl to set desiredvnodes to a
value that would use more than 75% of KVA or 75% of physical memory.

- PPWAIT is need only be locked by proc_lock, so move it to proc::p_lflag.
- Remove a few needless lock acquires from exec/fork/exit.
- Sprinkle branch hints.

No functional change.

branches: 1.140.2;
Kill devsw_lock and just use specfs_lock. The two would need merging
in order to prevent unload of modules when a device that they provide
is still open.

don't forget to fill in the emulation.

Use cpu_index(), not ci_cpuid.

branches: 1.137.2;
KERN_FILE_BYPID: fix locking botch.

Don't try grabbing a zombie's p_reflock.

PR kern/37917 /bin/ps no longer shows zombies

Remove clause 3 and 4 from TNF licenses

branches: 1.133.2;
Merge proc::p_mutex and proc::p_smutex into a single adaptive mutex, since
we no longer need to guard against access from hardware interrupt handlers.

Additionally, if cloning a process with CLONE_SIGHAND, arrange to have the
child process share the parent's lock so that signal state may be kept in
sync. Partially addresses PR kern/37437.

Network protocol interrupts can now block on locks, so merge the globals
proclist_mutex and proclist_lock into a single adaptive mutex (proc_lock).
Implications:

- Inspecting process state requires thread context, so signals can no longer
be sent from a hardware interrupt handler. Signal activity must be
deferred to a soft interrupt or kthread.

- As the proc state locking is simplified, it's now safe to take exit()
and wait() out from under kernel_lock.

- The system spends less time at IPL_SCHED, and there is less lock activity.

branches: 1.131.2;
- l_wmesg is not always valid. check l_wchan when using l_wmesg.
should fix a crash reported by Juan RP on current-users@.
- ttyinfo: lock lwp when accessing l_wmesg.
- fill_lwp: add an assertion.

use device_xname() where appropriate
OK martin

Revert rev 1.126-1.128. The original code was correct and rmind and I
didn't look correctly at them.

When copying l_name and l_wmesg use KI_LNAMELEN and KI_WMESGLEN
respectively, so that we don't care if l_name/wmesg is longer
than kl_name/wmesg and the KASSERTs added in previous can go away.

Fix previous: use the length of l->l_foo not kl->l_foo and add
two KASSERTs to check for max lenght limits before copying.

As suggested by rmind@.

fill_lwp: when copying l_wmesg and l_name, use the size of the string
not of the variable.

Found and ok by rmind@.

branches: 1.125.2;
Make rusage collection per-LWP and collate in the appropriate places.
cloned threads need a little bit more work but the locking needs to
be fixed first.

Catch up with descriptor handling changes. See kern_descrip.c revision
1.173 for details.

Convert to ansi definitions from old-style definitons.

branches: 1.122.2; 1.122.6;
Another locking botch.

More file/proc locking fixes.

Tons of process scope changes.

- Add a KAUTH_PROCESS_SCHEDULER action, to handle scheduler related
requests, and add specific requests for set/get scheduler policy and
set/get scheduler parameters.

- Add a KAUTH_PROCESS_KEVENT_FILTER action, to handle kevent(2) related
requests.

- Add a KAUTH_DEVICE_TTY_STI action to handle requests to TIOCSTI.

- Add requests for the KAUTH_PROCESS_CANSEE action, indicating what
process information is being looked at (entry itself, args, env,
open files).

- Add requests for the KAUTH_PROCESS_RLIMIT action indicating set/get.

- Add requests for the KAUTH_PROCESS_CORENAME action indicating set/get.

- Make bsd44 secmodel code handle the newly added rqeuests appropriately.

All of the above make it possible to issue finer-grained kauth(9) calls in
many places, removing some KAUTH_GENERIC_ISSUSER requests.

- Remove the "CAN" from KAUTH_PROCESS_CAN{KTRACE,PROCFS,PTRACE,SIGNAL}.

Discussed with christos@ and yamt@.

sysctl_kern_proc_args: avoid zero length allocation.

Patch up sysctl locking:

- Lock processes, credentials, filehead etc correctly.
- Acquire a read hold on sysctl_treelock if only doing a query.
- Don't wire down the output buffer. It doesn't work correctly and the code
regularly does long term sleeps with it held - it's not worth it.
- Don't hold locks other than sysctl_lock while doing copyout().
- Drop sysctl_lock while doing copyout / allocating memory in a few places.
- Don't take kernel_lock for sysctl.
- Fix a number of bugs spotted along the way

Remove systrace. Ok core@.

Add PaX ASLR (Address Space Layout Randomization) [from elad and myself]

For regular (non PIE) executables randomization is enabled for:
1. The data segment
2. The stack

For PIE executables(*) randomization is enabled for:
1. The program itself
2. All shared libraries
3. The data segment
4. The stack

(*) To generate a PIE executable:
- compile everything with -fPIC
- link with -shared-libgcc -Wl,-pie

This feature is experimental, and might change. To use selectively add
options PAX_ASLR=0
in your kernel.

Currently we are using 12 bits for the stack, program, and data segment and
16 or 24 bits for mmap, depending on __LP64__.

use binuptime for l_stime/l_rtime.

- Use KAUTH_ARG() instead of casts,
- Don't ignore return value of settime() in sysctl_kern_rtc_offset(), as
suggested by yamt@.

Note: the kauth(9) call in sysctl_kern_rtc_offset() is bogus, but this will
be addressed separately.

branches: 1.113.2; 1.113.4; 1.113.6;
Merge scheduler changes from the vmlocking branch. All discussed on
tech-kern:

- Invert priority space so that zero is the lowest priority. Rearrange
number and type of priority levels into bands. Add new bands like
'kernel real time'.
- Ignore the priority level passed to tsleep. Compute priority for
sleep dynamically.
- For SCHED_4BSD, make priority adjustment per-LWP, not per-process.

branches: 1.112.2;
machine/{bus,cpu,intr}.h -> sys/{bus,cpu,intr}.h

branches: 1.111.2;
Don't fail to produce the argument vector if the program has modified it
by deleting arguments. This is a popular practice, and failing means that
ps(1) prints (programname). For example this is what XtOpenDisplay() with
-geometry. This used to work before 2.0H, and the behavior is allowed and
hinted by POSIX. Found out by Anon Ymous.

- fix comment sentence capitalization.
- whitespace cleanup.
No functional changes.

Add _SC_NPROCESSORS_ONLN and _SC_NPROCESSORS_CONF for sysconf(). These
are extensions but are provided by many Unix systems.

sysctl_kern_lwp: Use a correct variable when rechecking if LWP still
exists after relocking. Found via CID: 4689. OK by <dsl>.

Merge from vmlocking: don't hold scheduler locks across copyout().

Add kern.no_sa_support to easily detect whether a kernel supports
Scheduler Activation or not. This is a negative name as ld.so.conf
conditionals threat undefined sysctls like 0.

branches: 1.105.2; 1.105.4;
Changes to make ktrace LKM friendly and reduce ifdef KTRACE. Proposed
on tech-kern.

branches: 1.104.2;
remove a homegrown definition of CPU_INFO_FOREACH.

branches: 1.103.2; 1.103.6;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

Add a flags parameter to kauth_cred_get/setgroups() so that sys_set/setgroups
can copy directly to/from userspace.
Avoids exposing the implementation of the group list as an array to code
outside kern_auth.c.
compat code and man page need updating.

merge yamt-idlelwp branch. asked by core@. some ports still needs work.

from doc/BRANCHES:

idle lwp, and some changes depending on it.

1. separate context switching and thread scheduling.
(cf. gmcgarry_ctxsw)
2. implement idle lwp.
3. clean up related MD/MI interfaces.
4. make scheduler(s) modular.

Remove proc->p_ru and the 'rusage' pool.
I think it existed to cache the numbers in kernel memory of a zombie when
proc->p_stats was part of the 'u' area - so got freed earlier and wouldn't
(easily) be accessible from a separate process. However since both the
p_ru and p_stats fields are freed at the same time it is no longer needed.
Ride the recent 4.99.19 version change.

branches: 1.99.2;
Add the LWP's runtime to kinfo_lwp.

branches: 1.98.2;
- Make the proclist_lock a mutex. The write:read ratio is unfavourable,
and mutexes are cheaper use than RW locks.
- LOCK_ASSERT -> KASSERT in some places.
- Hold proclist_lock/kernel_lock longer in a couple of places.

Change the process/lwp flags seen by userland via sysctl back to the
P_*/L_* naming convention, and rename the in-kernel flags to avoid
conflict. (P_ -> PK_, L_ -> LW_ ). Add back the (now unused) LSDEAD
constant.

Restores source compatibility with pre-newlock2 tools like ps or top.

Reviewed by Andrew Doran.

branches: 1.96.2;
Count the number of CPUs at boot and stash in 'ncpu'. Eventually should
have each CPU register at attach, so we can figure out the topology for
the scheduler.

Merge newlock2 to head.

Don't rely on KAUTH_PROCESS_CANSEE for environment just yet,
otherwise we're allowing anyone to read the environment unless
curtain is enabled.

From yamt@.

branches: 1.93.2;
Move Veriexec's sysctl(9) setup routine and helper to kern_verifiedexec.c.

PR/34837: Mindaguas: Add SysV SHM dynamic reallocation and locking to the
physical memory

implement kern.arandom properly, instead of lying about it and only filling
the first 4 bytes of the array with random data.

add the emulation in kinfo_proc2

Back out previous (p_flag2).

In 30 minutes from now Jason Thorpe will come up with an implementation
of a proplib dictionary in struct proc, so adding an int doesn't really
make any sense.

Until we figure out the Perfect Way of adding flags to processes, add
a p_flag2. No objections on tech-kern@.

Input from simonb@, thanks!

correct dcopyout #define for !KTRACE case.

Add a -t+S flag to ktrace for tracing activity related to sysctl. MIB
names will be displayed, with data readen and written as well.

branches: 1.85.2;
Don't use KAUTH_RESULT_* where it's not applicable.
Prompted by yamt@.

When getting the program argument or environement string, we previously
assumed that all the strings were stored in a row, separated by NUL chars,
at the address pointed bu argv[0] (or envp[0]).

This was wrong: if the program changed argvs[0], we still read the
first string correctly, but the next strings did contain unexpected data.

The fix: read the whole argv (or envp) array, then copy the string one by
one, using their addresses in argv (or agrp)

First take at security model abstraction.

- Add a few scopes to the kernel: system, network, and machdep.

- Add a few more actions/sub-actions (requests), and start using them as
opposed to the KAUTH_GENERIC_ISSUSER place-holders.

- Introduce a basic set of listeners that implement our "traditional"
security model, called "bsd44". This is the default (and only) model we
have at the moment.

- Update all relevant documentation.

- Add some code and docs to help folks who want to actually use this stuff:

* There's a sample overlay model, sitting on-top of "bsd44", for
fast experimenting with tweaking just a subset of an existing model.

This is pretty cool because it's *really* straightforward to do stuff
you had to use ugly hacks for until now...

* And of course, documentation describing how to do the above for quick
reference, including code samples.

All of these changes were tested for regressions using a Python-based
testsuite that will be (I hope) available soon via pkgsrc. Information
about the tests, and how to write new ones, can be found on:

http://kauth.linbsd.org/kauthwiki

NOTE FOR DEVELOPERS: *PLEASE* don't add any code that does any of the
following:

- Uses a KAUTH_GENERIC_ISSUSER kauth(9) request,
- Checks 'securelevel' directly,
- Checks a uid/gid directly.

(or if you feel you have to, contact me first)

This is still work in progress; It's far from being done, but now it'll
be a lot easier.

Relevant mailing list threads:

http://mail-index.netbsd.org/tech-security/2006/01/25/0011.html
http://mail-index.netbsd.org/tech-security/2006/03/24/0001.html
http://mail-index.netbsd.org/tech-security/2006/04/18/0000.html
http://mail-index.netbsd.org/tech-security/2006/05/15/0000.html
http://mail-index.netbsd.org/tech-security/2006/08/01/0000.html
http://mail-index.netbsd.org/tech-security/2006/08/25/0000.html

Many thanks to YAMAMOTO Takashi, Matt Thomas, and Christos Zoulas for help
stablizing kauth(9).

Full credit for the regression tests, making sure these changes didn't break
anything, goes to Matt Fleming and Jaime Fournier.

Happy birthday Randi! :)

When colecting a 32 bit process' argument or environement vector, we need
to convert 32 bits pointers to the 64 bit environement

branches: 1.81.4;
at the request of elad, as veriexec.h has returned, revert the changes
from 2006-07-25.

mechanically go through and
s,include "veriexec.h",include <sys/verified_exec.h>,
as the former has apparently gone away.

some fixes:
- adapt to NVERIEXEC in init_sysctl.c.
- we now need "veriexec.h" for NVERIEXEC.
- "opt_verified_exec.h" -> "opt_veriexec.h", and include it only where
it is needed.

Use the LWP cached credentials where sane.

- Don't cast kauth_cred_t to (struct ucred *), just set pc_ucred = NULL.
- Fill ucred::cr_ref.

CURTAIN() -> KAUTH_GENERIC_CANSEE.

move security.setid_core.* to kern.coredump.setid.*, as requested by yamt@.

Don't leak memory on success. Allocate only the type of struct that we'll
need for efficiency.

don't allocate too much stuff on the stack.

sysctl_security_setidcorename: don't allocate MAXPATHLEN bytes on stack.

branches: 1.71.2;
remove unnecessary arguments from kauth_authorize_process.
ie. make it similar to the one found in apple TN.

sysctl_kern_file, sysctl_kern_file2: don't abuse kauth_authorize_process
for non-process objects.

sysctl_kern_file2: fix an indent.

branches: 1.68.2;
integrate kauth.

Move securelevel-specific stuff to its own file.

Make i/o statistics collection more generic, include tape drives and
nfs mounts in the set of devices that statistics will be reported on.

PR/32809: Pavel Cahyna: Conflicting flags in l_flag and p_flag are causing
ps(1) to print incorrect information. Annotate the flags in the header files
to make sure that flags are not being re-used and move flags so that there
are no conflicts.

When DIAGNOSTIC is defined, provide a kern.panic_now sysctl to conviniently
and reliably panic the system

branches: 1.63.2; 1.63.4; 1.63.6;
merge yamt-uio_vmspace branch.

- use vmspace rather than proc or lwp where appropriate.
the latter is more natural to specify an address space.
(and less likely to be abused for random purposes.)
- fix a swdmover race.

for some random places, use PNBUF_GET/PUT rather than
- on-stack buffer
- malloc(MAXPATHLEN)

branches: 1.61.2;
implement a security.setid_core node as discussed on tech-kern@ and
tech-security@.

branches: 1.60.2;
remove security node sysctl objects; they are now created using CTL_CREATE.

branches: 1.59.2;
u_intN_t -> uintN_t

merge ktrace-lwp.

- make settime take timespec.
- avoid wrapping of time in settime.
- pass struct proc down so that we can log a detailed message.

sysctl_kern_proc_args: don't assume that the process is
resident while we are sleeping.

Implement curtain in KERN_{PROC,PROC2,FILE,FILE2,PROC_ARGS}.
While I'm here, disable curtain by default.

Introduce ``security.curtain'', new node for security features and
settings, and new variable for controlling access to objects based
on user-id.

Implement kern.hardclock_ticks.

Fix a tyop in a comment.

Remove the tape stats from here, they caused issues on non-scsipi
architectures.

Don't include tape stats functions if no devices configured.

Add tape statistics gathering functions.

#ifdef VERIFIED_EXEC

defopt verified_exec.

branches: 1.46.2;
Comment in new cp_id implementation was wrong since I abandoned
rewriting it in favor of some testing and then never got back to it.
It's better now.

Add a new sysctl 'cp_id' that returns the array of cpu id values. Requested by
me, implemented by atatat.

Fix sysctl handling for raise-only variables. This affected the veriexec
node entirely. Reported by Nino Dehne.

Properly fix the constipated lossage wrt -Wcast-qual and the sysctl
code. I know it's not the prettiest code, but it seems to work rather
well in spite of itself.

Revert previous ('_ncpus' is now 'ncpus' again).
MI variable names have precedence.

Rename 'ncpus' to '_ncpus', otherwise we shadow sparc/sparc64's 'ncpus'
when MULTIPROCESSOR is defined.

- add const.
- remove unnecessary casts.
- add __UNCONST casts and mark them with XXXUNCONST as necessary.

Add indication for number of fingerprinted files on each device.

When a table is created for a new device, a new variable is created
under the kern.veriexec.count node named "dev_<id>". For example,
dev_0, dev_3, etc.

Some changes in veriexec.

New features:

- Add a veriexec_report() routine to make most reporting consistent and
remove some common code.
- Add 'strict' mode that controls how veriexec behaves.
- Add sysctl knobs:
o kern.veriexec.verbose controls verbosity levels. Value: 0, 1.
o kern.veriexec.strict controls strict level. Values: 0, 1, 2. See
documentation in sysctl(3) for details.
o kern.veriexec.algorithms returns a string with a space separated
list of supported hashing algorithms in veriexec.
- Updated documentation in man pages for sysctl(3) and sysctl(8).

Bug fixes:

- veriexec_removechk(): Code cleanup + handle FINGERPRINT_NOTEVAL
correctly.
- exec_script(): Don't pass 0 as flag when executing a script; use the
defined VERIEXEC_INDIRECT - which is 1. Makes indirect execution
enforcement work.
- Fix some printing formats and types..

be explicit in the description for POSIX saved set-id that this is for
POSIX-style, not sane-style. (ie, add "POSIX " to the description.)

branches: 1.36.2;
Revert the change that made kern.file2 and net.*.*.pcblist into nodes
instead of structs. It had other deleterious side-effects that are
rather nasty. Another solution must be found.

Change types of kern.file2 and net.*.*.pcblist to NODE

Add kern.file2. As kern.proc2 is to kern.proc, so is kern.file2 to
kern.file, namely a 32/64 bit clean sysctl interface to the same data.
It also borrows a few things from struct vnode (if applicable) and
from struct proc, just to tie things together a bit more.

You can walk this list "by file" or "by pid". The former method is
similar to kern.file but omits the filehead, and the latter can give
you duplicates if multiple processes have the same struct file open,
but tells you which process it is.

nuke trailing whitespace

branches: 1.32.4; 1.32.6;
introduce a function, proclist_foreach_call, to iterate all procs on
a proclist and call the specified function for each of them.
primarily to fix a procfs locking problem, but i think that it's useful for
others as well.

while i'm here, introduce PROCLIST_FOREACH macro, which is similar to
LIST_FOREACH but skips marker entries which are used by proclist_foreach_call.

branches: 1.31.2;
The message buffer datum instrumented by KERN_MSGBUFSIZE is actually a
long, not an int, and this causes "problems" on LP64be machines
(sparc64, etc). Assign the value to a temporary int and instrument
that instead. Should be fine until someone wants a message buffer
larger than two gigabytes.

(off_t)(long) is wrong when it comes to kernel addresses [because on a 32 bit
machine if the high bit is set they turn negative]. Make an intermediate cast
to unsigned long.

Fix a comment.
Approved by Andrew Brown.

s/the the/the/ (only in sources that aren't regularly imported from
elsewhere).

Prefer that kern.hostid is printed in hex, not as a signed decimal,
and avoid accidental sign-extension when setting it.

Lots of sysctl descriptions (if someone wants to help out here, that
would be good) mostly copied from sysctl(3). This takes care of the
top-level, most of kern.* and hw.* (modulo the ath and bge stuff), and
all of proc.*.

If you don't want the added rodata in your kernel, use "options
SYSCTL_NO_DESCR" in your kernel config.

Clear out the struct kinfo_drivers before stuffing things into it.
Avoids leaking garbage from the stack (left over from the earlier
call to sysctl_locate()).

branches: 1.24.2;
Tango on sysctl_createv() and flags. The flags have all been renamed,
and sysctl_createv() now uses more arguments.

- move kern.somaxkva sysctl stuff from init_sysctl.c to uipc_socket.c.
- when changing its value, wakeup sokva waiters.

Use KERN_PROCSLOP for struct kinfo_proc and KERN_LWPSLOP for
struct kinfo_lwp, and not vice versa.

Should solve the issue with top dying because it's unable to "allocate
memory".

Use new PTRTOUINT64() macro instead of local PTRTOINT64() macro.

Avoid dereferencing l...it might be NULL

Sysctl functions called for "generic" nodes should forward "query"
requests (where possible), rather than returning errors.

Adjust error returns in kern.cp_time when a specific processor is
being requested so that (1) the uniprocessor case and the
multiprocessor case are more similar and (2) so that we return ENOENT
when a non-existent processor is requested (which is both more
sensible and follows the general order of things anyway).

Rename sysctl_kern_hostname() to sysctl_setlen() and use it also for
domainname. Note that there's no need to copy rnode since we're not
changing any of it, nor protecting anything from change.

Thanks to martin for initial work.

RCSid police

After changing hostname, adjust hostnamelen.
This closes PR kern/23907.

Make kern.rtc_offset writable at securelevel <= 0.
This allows boot-time adjustment when a machine runs other OSes with
RTC == localtime.

update a comment to match with the previous change (rev.1.12).

restore functionality to decrease kern.maxvnodes which
has been backed out during sysctl rework.

In sysctl_kern_lwp adjust offsets into the mib entries so that
they are now correct. Fixes problems with "ps -s" not working.
Also use KERN_LWPSLOP instead of KERN_PROCSLOP.

Both changes from Andrew Brown.

Make kern.dump_on_panic writeable again, too

Make kern.sbmax writeable again as well.

From a follow-on to PR kern/23695 by a Mr. Davis, which I missed at a
quick glance.

Make kern.logsigexit writeable again.

Fixes PR kern/23695.

Add missing break.

Also make declaration of sysctl_kern_maxptys() depend on NPTY > 0.
Makes the mvme68k RAMDISK kernel compile again.

Fix kern.cp_time for MULTIPROCESSOR kernels: calculate size of result
correctly, free original instead of incremented pointer, copy results for
n = -2 case too, so top shows correct stats.
Additionaly, rearange code for better readability (from Andrew).

Include opt_posix.h for the P1003_1B_SEMAPHORE define.
Include <machine/cpu.h> just to be sure.

We can not count CPUs at sysctl initialization time - so don't make
hw.ncpu an immediate value.

#include "opt_multiprocessor.h"

This makes hw.ncpu and kern.cp_time work better on those platforms.

Dynamic sysctl.

Gone are the old kern_sysctl(), cpu_sysctl(), hw_sysctl(),
vfs_sysctl(), etc, routines, along with sysctl_int() et al. Now all
nodes are registered with the tree, and nodes can be added (or
removed) easily, and I/O to and from the tree is handled generically.

Since the nodes are registered with the tree, the mapping from name to
number (and back again) can now be discovered, instead of having to be
hard coded. Adding new nodes to the tree is likewise much simpler --
the new infrastructure handles almost all the work for simple types,
and just about anything else can be done with a small helper function.

All existing nodes are where they were before (numerically speaking),
so all existing consumers of sysctl information should notice no
difference.

PS - I'm sorry, but there's a distinct lack of documentation at the
moment. I'm working on sysctl(3/8/9) right now, and I promise to
watch out for buses.

KNF (sort #includes and remove duplicate sys/cpu.h)

Move proc_lock into the data segment. It was dynamically allocated because
at the time we had mutex_obj_alloc() but not __cacheline_aligned.

Merge vfs_cache.c from the ad-namecache branch. With this the namecache
index becomes per-directory (initially, a red-black tree). The remaining
changes on the branch to namei()/getcwd() will be merged in the future.

Use 4K pages on ARM_MMU_EXTENDED platforms (all armv[67] except RPI) by
creating a new pool l1ttpl for the userland L1 translation table which
needs to be 8KB and 8KB aligned.

Limit the pool to maxproc and add hooks to allow the sysctl changing of
maxproc to adjust the pool.

This comes at a 5% performance penalty for build.sh -j8 kernel on a
Tegra TK1.

branches: 1.223.2;
- Eliminate the global "boottime" variable, which was being accessed
without any synchronization against changes by e.g. clock_settime().
- Replace with new getbinboottime() / getnanoboottime() / getmicroboottime()
functions (naming mirrors that of other time access functions in kern_tc.c).
It returns the (maybe-converted) value of timebasebin, which also tracks
our estimate of when the system was booted (i.e. the legacy "boottime" was
redundant).

XXX There needs to be a lockless synchronization mechanism for reading
timebasebin, but this is a problem in kern_tc.c that pre-existed these
"boottime" changes. At least now the problem is centralized in one location.

remove kern.panic_now -- crashme panic node replaces it.

As discussed in tech-kern:

- make sysctl kern.expose_address tri-state:
0: no access
1: access to processes with open /dev/kmem
2: access to everyone
defaults:
0: KASLR kernels
1: non-KASLR kernels

- improve efficiency by calling get_expose_address() per sysctl, not per
process.

- don't expose addresses for linux procfs

- welcome to 8.99.27, changes to fill_*proc ABI

Expose addresses depending on the KASLR setting (from mrg@). Restores the
status quo of exposing kernel addresses if there is no KASLR.

Fix kernel pointer leaks in the kern.lwp sysctl.

Provide a sysctl kern.expose_address to expose kernel addresses in
sysctl structure returns for non-root. Defaults to off. Turning it
on will restore sockstat/fstat and friends for regular users.

CTL_DEBUG_MAXID is only used to size a static array that the compiler
can do just fine itself. use the compiler and remove the define.

Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)

- Cleanup for dynamic sysctl:
- Remove unused *_NAMES macros for sysctl.
- Remove unused *_MAXID for sysctls.
- Move CTL_MACHDEP sysctl definitions for m68k into m68k/include/cpu.h and
use them on all m68k machines.

branches: 1.214.2; 1.214.4;
Add a proper defflag for GPROF, and include opt_gprof.h, otherwise we're
not gonna go very far.

remove checks for failure after memory allocation calls that cannot fail:

kmem_alloc() with KM_SLEEP
kmem_zalloc() with KM_SLEEP
percpu_alloc()
pserialize_create()
psref_class_create()

all of these paths include an assertion that the allocation has not failed,
so callers should not assert that again.

Remove the "target" argment from vfs_drainvnodes() as it is
always equal to "desiredvnodes" and move its definition
from sys/vnode.h to sys/vnode_impl.h.

Extend vfs_drainvnodes() to also wait for deferred vrele to flush
and replace the call to vrele_flush() with a call to vfs_drainvnodes().

branches: 1.211.2;
Add a new kern.messages sysctl to allow kernel message verbosity to be
altered after boot.

Fixes PR kern/46539 using patch submitted by Nat Sloss.

Whether or not the semaphore code is loaded as a module or built-in, its
sysctl data belongs with the module code. Move it from kern/init_sysctl.c
to kern/uipc_sem.c

While here, add a new sysctl variable kern.posix.semcnt (current count of
semaphores) to complement the existing kern.posix.semmax (maximum number
of semaphores).

Move a bunch of sysctl nodes from init_sysctl (kitchen sink sysctl file)
to init_sysctl_base (only base kernel defs). Main motivation was to
fix sysconf(_SC_NPROCESSORS) for Rumprun. As reported by neeraj on irc,
it returned -1 before this fix, so we were doing imaginary computing.

Move hw.machine and hw.machine_arch sysctls to base so rump can use them

This allows uname(3) and uname(1) to work on rump kernels.

group msgbuf sysctls with the msgbuf code
(init_sysctl.c -> subr_log.c)

More preparation for modularizing the SYSVxxx options. Here we
change the kern.ipc.sysvxxx sysctls into dynamic values, so each
sub-component of SYSVxxx can declare its own availability.

move clock sysctls from init_sysctl.c to kern_clock.c

branches: 1.204.4;
BUILDINFO part 2: expose sysctl kern.buildinfo

Add a global vnode cache:

- vcache_get() retrieves a referenced and initialised vnode / fs node pair.
- vcache_remove() removes a vnode / fs node pair from the cache.

On cache miss vcache_get() calls new vfs operation vfs_loadvnode() to
initialise a vnode / fs node pair. This call is guaranteed exclusive,
no other thread will try to load this vnode / fs node pair.

Convert ufs/ext2fs, ufs/ffs and ufs/mfs to use this interface.

Remove now unused ufs/ufs_ihash

Discussed on tech-kern.

Welcome to 6.99.41

branches: 1.202.2;
- create cpu_{g,s}etmodel() and hide cpu_model from direct access.

Ensure that the top level sysctl nodes (kern, vfs, net, ...) exist before
the sysctl link sets are processed, and remove redundancy.

Shaves >13kB off of an amd64 GENERIC, not to mention >1k duplicate
lines of code.

Add kern.{ostype,osrelease,osrevision,version} kern.domainname,
kern.rawpartition sysctl support to rump kernel.
Moved the sysctl support that is shared between rump and normal
kernels to init_sysctl_base.c as rump cannot use init_sysctl.c
in order to avoid code duplication. Agreed with pooka@.

Put cprng sysctls into subr_cprng.c. Also, make sysctl_prng static
in subr_cprng and get rid of SYSCTL_PRIVATE namespace leak macro.

Fixes ping(8) when run against a standalone rump kernel due to appearance
of the kern.urandom sysctl node (in case someone was wondering ...)

GC various arrays defined and used in kern_proc.c

branches: 1.197.6;
calculate vnode cache size based on the resource it gets allocated from
this stops setting kern.maxvnodes to high so it exhausts available space in kmem

http://mail-index.netbsd.org/tech-kern/2013/03/08/msg015095.html

Add a kern.configname sysctl object.

Move boottime50 and its associated sysctl into the compat module. As
noted on tech-kern. Should fix PR/47579.

OK christos@

Will request pull-up to 6.0 in a few days.

Make the inclusion of <sys/cprng.h> a private matter for sysctl. No reason
to expose the rest of the kernel to it.

split device_t/softc for all remaining drivers.
replace "struct device *" with "device_t".
use device_xname(), device_unit(), etc.

put all kern socket sysctls in the same place

Add sanity check to sysctl_kern_maxvnodes.

branches: 1.190.2;
Add some pre-processor magic to verify that the type of the data item
passed to sysctl_createv() actually matches the declared type for
the item itself.
In the places where the caller specifies a function and a structure
address (typically the 'softc') an explicit (void *) cast is now needed.
Fixes bugs in sys/dev/acpi/asus_acpi.c sys/dev/bluetooth/bcsp.c
sys/kern/vfs_bio.c sys/miscfs/syncfs/sync_subr.c and setting
AcpiGbl_EnableAmlDebugObject.
(mostly passing the address of a uint64_t when typed as CTLTYPE_INT).
I've test built quite a few kernels, but there may be some unfixed MD
fallout. Most likely passing &char[] to char *.
Also add CTLFLAG_UNSIGNED for unsiged decimals - not set yet.

remove bogus check.

P1003_1B_SEMAPHORE is no longer optional.

Remove COMPAT_SA / KERN_SA. Welcome to 6.99.3!
Approved by core@.

branches: 1.186.2;

Separate /dev/random pseudodevice implemenation from kernel entropy pool
implementation. Rewrite pseudodevice code to use cprng_strong(9).

The new pseudodevice is cloning, so each caller gets bits from a stream
generated with its own key. Users of /dev/urandom get their generators
keyed on a "best effort" basis -- the kernel will rekey generators
whenever the entropy pool hits the high water mark -- while users of
/dev/random get their generators rekeyed every time key-length bits
are output.

The underlying cprng_strong API can use AES-256 or AES-128, but we use
AES-128 because of concerns about related-key attacks on AES-256. This
improves performance (and reduces entropy pool depletion) significantly
for users of /dev/urandom but does cause users of /dev/random to rekey
twice as often.

Also fixes various bugs (including some missing locking and a reseed-counter
overflow in the CTR_DRBG code) found while testing this.

For long reads, this generator is approximately 20 times as fast as the
old generator (dd with bs=64K yields 53MB/sec on 2Ghz Core2 instead of
2.5MB/sec) and also uses a separate mutex per instance so concurrency
is greatly improved. For reads of typical key sizes for modern
cryptosystems (16-32 bytes) performance is about the same as the old
code: a little better for 32 bytes, a little worse for 16 bytes.

branches: 1.185.2;
An undocumented behavior of the sysctl kern.arandom node used to allow
sucking up to 8192 bytes out of the kernel arc4random() generator at a
time. Supposedly some very old application code uses this to rekey
other instances of RC4 in userspace (a truly great idea). Reduce the
limit to 256 bytes -- and note that it will probably be reduced to
sizeof(int) in the future, since this node is so documented.

First step of random number subsystem rework described in
<20111022023242.BA26F14A158@mail.netbsd.org>. This change includes
the following:

An initial cleanup and minor reorganization of the entropy pool
code in sys/dev/rnd.c and sys/dev/rndpool.c. Several bugs are
fixed. Some effort is made to accumulate entropy more quickly at
boot time.

A generic interface, "rndsink", is added, for stream generators to
request that they be re-keyed with good quality entropy from the pool
as soon as it is available.

The arc4random()/arc4randbytes() implementation in libkern is
adjusted to use the rndsink interface for rekeying, which helps
address the problem of low-quality keys at boot time.

An implementation of the FIPS 140-2 statistical tests for random
number generator quality is provided (libkern/rngtest.c). This
is based on Greg Rose's implementation from Qualcomm.

A new random stream generator, nist_ctr_drbg, is provided. It is
based on an implementation of the NIST SP800-90 CTR_DRBG by
Henric Jungheim. This generator users AES in a modified counter
mode to generate a backtracking-resistant random stream.

An abstraction layer, "cprng", is provided for in-kernel consumers
of randomness. The arc4random/arc4randbytes API is deprecated for
in-kernel use. It is replaced by "cprng_strong". The current
cprng_fast implementation wraps the existing arc4random
implementation. The current cprng_strong implementation wraps the
new CTR_DRBG implementation. Both interfaces are rekeyed from
the entropy pool automatically at intervals justifiable from best
current cryptographic practice.

In some quick tests, cprng_fast() is about the same speed as
the old arc4randbytes(), and cprng_strong() is about 20% faster
than rnd_extract_data(). Performance is expected to improve.

The AES code in src/crypto/rijndael is no longer an optional
kernel component, as it is required by cprng_strong, which is
not an optional kernel component.

The entropy pool output is subjected to the rngtest tests at
startup time; if it fails, the system will reboot. There is
approximately a 3/10000 chance of a false positive from these
tests. Entropy pool _input_ from hardware random numbers is
subjected to the rngtest tests at attach time, as well as the
FIPS continuous-output test, to detect bad or stuck hardware
RNGs; if any are detected, they are detached, but the system
continues to run.

A problem with rndctl(8) is fixed -- datastructures with
pointers in arrays are no longer passed to userspace (this
was not a security problem, but rather a major issue for
compat32). A new kernel will require a new rndctl.

The sysctl kern.arandom() and kern.urandom() nodes are hooked
up to the new generators, but the /dev/*random pseudodevices
are not, yet.

Manual pages for the new kernel interfaces are forthcoming.

branches: 1.183.2;
Add getlabelusesmbr(), as proposed in
http://mail-index.netbsd.org/tech-userlevel/2011/08/25/msg005404.html
This is used by disk tools such as disklabel(8) to dynamically decide is
the undelyling platform uses a disklabel-in-mbr-partition or not
(instead of using a compile-time list of ports).
getlabelusesmbr() reads the sysctl kern.labelusesmbr, takes its value from the
machdep #define LABELUSESMBR.
For evbmips, make LABELUSESMBR 1 if the platform uses pmon
as bootloader, and 0 (the previous value) otherwise.

When KERN_SA is not defined, kern.no_sa_support is a constant (1). So
add CTLFLAG_IMMEDIATE to flags. Make the macro block logically reversed so
it looks more natural when reading.

Reported by Peter Tworek on tech-kern@.

Add some needed __UNCONST

vfs_drainvnodes: drop lwp argument, remove variable name in prototype.

avoid code duplication.

migrate compat32 handling with previous

pointed out by Lars Heidieker

Move sysctl routines from init_sysctl.c to kern_descrip.c (for
descriptors) and kern_proc.c (for processes). This makes them
usable in a rump kernel, in case somebody was wondering.

Use the L_ flags instead of the P_ flags for lwps.

branches: 1.175.2; 1.175.4;
Remove pfind() and pgfind(), fix locking in various broken uses of these.
Rename real routines to proc_find() and pgrp_find(), remove PFIND_* flags
and have consistent behaviour. Provide proc_find_raw() for special cases.
Fix memory leak in sysctl_proc_corename().

COMPAT_LINUX: rework ptrace() locking, minimise differences between
different versions per-arch.

Note: while this change adds some formal cosmetics for COMPAT_DARWIN and
COMPAT_IRIX - locking there is utterly broken (for ages).

Fixes PR/43176.

Set kinfo_lwp to 0 before filling it so that if someone removes
variable assignments from here, kernel memory does not leak to
userspace.

Bug found, a little bit suprisingly, by the atf ps test which failed
due to the column width between the -o holdcnt column being too
wide due to the contents displayed being garbage.

branches: 1.173.2;
sysctl_doeproc: don't follow a possibly stale pointer.

branches: 1.172.2;
Minimize unnecessary differences in rump.

When reporting open files using sysctl, don't use 'filehead' to fetch files,
as we don't have a process context to authorize on. Instead, traverse the
file descriptor table of each process -- as we already do in one case.

Introduce a "marker" we can use to mark files we've seen in an iteration, as
the same file can be referenced more than once.

Hopefully this availability of filtering by process also makes life easier
for those who are interested in implementing process "containers" etc.

Report L_INMEM in the lwp info as well.

Always set L_INMEM to maintain binary compatibility.

Remove uarea swap-out functionality:

- Addresses the issue described in PR/38828.
- Some simplification in threading and sleepq subsystems.
- Eliminates pmap_collect() and, as a side note, allows pmap optimisations.
- Eliminates XS_CTL_DATA_ONSTACK in scsipi code.
- Avoids few scans on LWP list and thus potentially long holds of proc_lock.
- Cuts ~1.5k lines of code. Reduces amd64 kernel size by ~4k.
- Removes __SWAP_BROKEN cases.

Tested on x86, mips, acorn32 (thanks <mpumford>) and partly tested on
acorn26 (thanks to <bjh21>).

Discussed on <tech-kern>, reviewed by <ad>.

Chop init_sysctl into base nodes (init_sysctl_base.c) and the
kitchen sink (init_sysctl.c). Further surgery may be needed down
the line.

Expose the kernel's boothowto(9) variable through the sysctl
kern.boothowto variable.

Part of the /etc/rc silent changes requested in PR 41946
and proposed in tech-userlevel.

provide compatibility for the older variant of kern.consdev, which used
a 32 bit dev_t. Reported by mrg.

More changes to improve kern_descrip.c.

- Avoid atomics in more places.
- Remove the per-descriptor mutex, and just use filedesc_t::fd_lock.
It was only being used to synchronize close, and in any case we needed
to take fd_lock to free the descriptor slot.
- Optimize certain paths for the <NDFDFILE case.
- Sprinkle more comments and assertions.
- Cache more stuff in filedesc_t.
- Fix numerous minor bugs spotted along the way.
- Restructure how the open files array is maintained, for clarity and so
that we can eliminate the membar_consumer() call in fd_getfile(). This is
mostly syntactic sugar; the main functional change is that fd_nfiles now
lives alongside the open file array.

Some measurements with libmicro:

- simple file syscalls are like close() are between 1 to 10% faster.
- some nice improvements, e.g. poll(1000) which is ~50% faster.

sysctl_doeproc:
- simplify.
- KERN_PROC: fix possible stale proc pointer dereference.
- KERN_PROC: don't do copyout with proc_lock held.

don't forget to skip marker processes.

sysctl_doeproc: fix a bug in rev.1.135.
don't forget to mark our marker process PK_MARKER.
this fixes crashes in sched_pstats, etc.

- add new RLIMIT_AS (aka RLIMIT_VMEM) resource that limits the total
address space available to processes. this limit exists in most other
modern unix variants, and like most of them, our defaults are unlimited.
remove the old mmap / rlimit.datasize hack.

- adds the VMCMD_STACK flag to all the stack-creation vmcmd callers.
it is currently unused, but was added a few years ago.

- add a pair of new process size values to kinfo_proc2{}. one is the
total size of the process memory map, and the other is the total size
adjusted for unused stack space (since most processes have a lot of
this...)

- patch sh, and csh to notice RLIMIT_AS. (in some cases, the alias
RLIMIT_VMEM was already present and used if availble.)

- patch ps, top and systat to notice the new k_vm_vsize member of
kinfo_proc2{}.

- update irix, svr4, svr4_32, linux and osf1 emulations to support
this information. (freebsd could be done, but that it's best left
as part of the full-update of compat/freebsd.)


this addresses PR 7897. it also gives correct memory usage values,
which have never been entirely correct (since mmap), and have been
very incorrect since jemalloc() was enabled.

tested on i386 and sparc64, build tested on several other platforms.

thanks to many folks for feedback and testing but most espcially
chuq and yamt for critical suggestions that lead to this patch not
having a special ugliness i wasn't happy with anyway :-)

like KERN_FILE2: *do* update "needed" when there is no count. we want
userland to know what sort of size to provide..

while here, slightly normalise the previous to init_sysctl.c.

always calculate "needed" for KERN_FILE2 calls. this allows a caller
to get an estimate of the needed space, like the intention is.

Don't bother with file_t::f_iflags any more, as it's not used.
Noted by mrg@.

Use "defopt MODULAR" in sys/conf/files, and #include "opt_modular.h"
in all kernel sources that use the MODULAR option.
Proposed in tech-kern on 18 Jan 2009.

branches: 1.155.2;
whitespace nit

malloc -> kmem_alloc.

merge christos-time_t

Rename specfs_lock as device_lock and move it from specfs to devsw.
Relaxes kernel dependency on vfs.

PR/40002: Daniel Horecki: sockstat doesn't work for user with sysctl
security.curtain=1

If the kauth call failed, we'd silently continue the loop, but the error
code would remain and eventually "leak" to userspace. Reset the error to
zero when continuing.

Tested by snj@ and myself. Okay snj@.

Allow the POSIX semaphore code to be loaded as a module.

branches: 1.149.2; 1.149.4;
Set kern.posix_semaphores are runtime so it can be a module.
(Picked wrong header the last time.)

Set kern.posix_semaphores are runtime so it can be a module.

rename proc_representative_lwp to proc_active_lwp and clarify it is for
ps display purposes. suggested by rmind.

Select a "representative" lwp instead of the first lwp in the list. The
first lwp in the list is the last created and in the firefox and gtk-gnash
case this is usually a zombie, so the status in ps was ZLl. This now picks
the lwp in order ONPROC > RUN > SLEEP > STOP > SUSPENDED > IDL > DEAD > ZOMB
and breaks ties using cpticks.

Merge wrstuden-revivesa into HEAD.

make l_flags contain more stuff. Fixes top thread display where system processes
were always displayed.

branches: 1.143.2;
Remove proc_representative_lwp(), use a simple LIST_FIRST() instead.
OK by <ad>.

PR kern/38927: processes getting stuck in uvm_map (cv_timedwait), hanging
machine

Assume that a vnode (and associated data structures) costs 2kB in the
worst imaginable case. Don't allow sysctl to set desiredvnodes to a
value that would use more than 75% of KVA or 75% of physical memory.

- PPWAIT is need only be locked by proc_lock, so move it to proc::p_lflag.
- Remove a few needless lock acquires from exec/fork/exit.
- Sprinkle branch hints.

No functional change.

branches: 1.140.2;
Kill devsw_lock and just use specfs_lock. The two would need merging
in order to prevent unload of modules when a device that they provide
is still open.

don't forget to fill in the emulation.

Use cpu_index(), not ci_cpuid.

branches: 1.137.2;
KERN_FILE_BYPID: fix locking botch.

Don't try grabbing a zombie's p_reflock.

PR kern/37917 /bin/ps no longer shows zombies

Remove clause 3 and 4 from TNF licenses

branches: 1.133.2;
Merge proc::p_mutex and proc::p_smutex into a single adaptive mutex, since
we no longer need to guard against access from hardware interrupt handlers.

Additionally, if cloning a process with CLONE_SIGHAND, arrange to have the
child process share the parent's lock so that signal state may be kept in
sync. Partially addresses PR kern/37437.

Network protocol interrupts can now block on locks, so merge the globals
proclist_mutex and proclist_lock into a single adaptive mutex (proc_lock).
Implications:

- Inspecting process state requires thread context, so signals can no longer
be sent from a hardware interrupt handler. Signal activity must be
deferred to a soft interrupt or kthread.

- As the proc state locking is simplified, it's now safe to take exit()
and wait() out from under kernel_lock.

- The system spends less time at IPL_SCHED, and there is less lock activity.

branches: 1.131.2;
- l_wmesg is not always valid. check l_wchan when using l_wmesg.
should fix a crash reported by Juan RP on current-users@.
- ttyinfo: lock lwp when accessing l_wmesg.
- fill_lwp: add an assertion.

use device_xname() where appropriate
OK martin

Revert rev 1.126-1.128. The original code was correct and rmind and I
didn't look correctly at them.

When copying l_name and l_wmesg use KI_LNAMELEN and KI_WMESGLEN
respectively, so that we don't care if l_name/wmesg is longer
than kl_name/wmesg and the KASSERTs added in previous can go away.

Fix previous: use the length of l->l_foo not kl->l_foo and add
two KASSERTs to check for max lenght limits before copying.

As suggested by rmind@.

fill_lwp: when copying l_wmesg and l_name, use the size of the string
not of the variable.

Found and ok by rmind@.

branches: 1.125.2;
Make rusage collection per-LWP and collate in the appropriate places.
cloned threads need a little bit more work but the locking needs to
be fixed first.

Catch up with descriptor handling changes. See kern_descrip.c revision
1.173 for details.

Convert to ansi definitions from old-style definitons.

branches: 1.122.2; 1.122.6;
Another locking botch.

More file/proc locking fixes.

Tons of process scope changes.

- Add a KAUTH_PROCESS_SCHEDULER action, to handle scheduler related
requests, and add specific requests for set/get scheduler policy and
set/get scheduler parameters.

- Add a KAUTH_PROCESS_KEVENT_FILTER action, to handle kevent(2) related
requests.

- Add a KAUTH_DEVICE_TTY_STI action to handle requests to TIOCSTI.

- Add requests for the KAUTH_PROCESS_CANSEE action, indicating what
process information is being looked at (entry itself, args, env,
open files).

- Add requests for the KAUTH_PROCESS_RLIMIT action indicating set/get.

- Add requests for the KAUTH_PROCESS_CORENAME action indicating set/get.

- Make bsd44 secmodel code handle the newly added rqeuests appropriately.

All of the above make it possible to issue finer-grained kauth(9) calls in
many places, removing some KAUTH_GENERIC_ISSUSER requests.

- Remove the "CAN" from KAUTH_PROCESS_CAN{KTRACE,PROCFS,PTRACE,SIGNAL}.

Discussed with christos@ and yamt@.

sysctl_kern_proc_args: avoid zero length allocation.

Patch up sysctl locking:

- Lock processes, credentials, filehead etc correctly.
- Acquire a read hold on sysctl_treelock if only doing a query.
- Don't wire down the output buffer. It doesn't work correctly and the code
regularly does long term sleeps with it held - it's not worth it.
- Don't hold locks other than sysctl_lock while doing copyout().
- Drop sysctl_lock while doing copyout / allocating memory in a few places.
- Don't take kernel_lock for sysctl.
- Fix a number of bugs spotted along the way

Remove systrace. Ok core@.

Add PaX ASLR (Address Space Layout Randomization) [from elad and myself]

For regular (non PIE) executables randomization is enabled for:
1. The data segment
2. The stack

For PIE executables(*) randomization is enabled for:
1. The program itself
2. All shared libraries
3. The data segment
4. The stack

(*) To generate a PIE executable:
- compile everything with -fPIC
- link with -shared-libgcc -Wl,-pie

This feature is experimental, and might change. To use selectively add
options PAX_ASLR=0
in your kernel.

Currently we are using 12 bits for the stack, program, and data segment and
16 or 24 bits for mmap, depending on __LP64__.

use binuptime for l_stime/l_rtime.

- Use KAUTH_ARG() instead of casts,
- Don't ignore return value of settime() in sysctl_kern_rtc_offset(), as
suggested by yamt@.

Note: the kauth(9) call in sysctl_kern_rtc_offset() is bogus, but this will
be addressed separately.

branches: 1.113.2; 1.113.4; 1.113.6;
Merge scheduler changes from the vmlocking branch. All discussed on
tech-kern:

- Invert priority space so that zero is the lowest priority. Rearrange
number and type of priority levels into bands. Add new bands like
'kernel real time'.
- Ignore the priority level passed to tsleep. Compute priority for
sleep dynamically.
- For SCHED_4BSD, make priority adjustment per-LWP, not per-process.

branches: 1.112.2;
machine/{bus,cpu,intr}.h -> sys/{bus,cpu,intr}.h

branches: 1.111.2;
Don't fail to produce the argument vector if the program has modified it
by deleting arguments. This is a popular practice, and failing means that
ps(1) prints (programname). For example this is what XtOpenDisplay() with
-geometry. This used to work before 2.0H, and the behavior is allowed and
hinted by POSIX. Found out by Anon Ymous.

- fix comment sentence capitalization.
- whitespace cleanup.
No functional changes.

Add _SC_NPROCESSORS_ONLN and _SC_NPROCESSORS_CONF for sysconf(). These
are extensions but are provided by many Unix systems.

sysctl_kern_lwp: Use a correct variable when rechecking if LWP still
exists after relocking. Found via CID: 4689. OK by <dsl>.

Merge from vmlocking: don't hold scheduler locks across copyout().

Add kern.no_sa_support to easily detect whether a kernel supports
Scheduler Activation or not. This is a negative name as ld.so.conf
conditionals threat undefined sysctls like 0.

branches: 1.105.2; 1.105.4;
Changes to make ktrace LKM friendly and reduce ifdef KTRACE. Proposed
on tech-kern.

branches: 1.104.2;
remove a homegrown definition of CPU_INFO_FOREACH.

branches: 1.103.2; 1.103.6;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

Add a flags parameter to kauth_cred_get/setgroups() so that sys_set/setgroups
can copy directly to/from userspace.
Avoids exposing the implementation of the group list as an array to code
outside kern_auth.c.
compat code and man page need updating.

merge yamt-idlelwp branch. asked by core@. some ports still needs work.

from doc/BRANCHES:

idle lwp, and some changes depending on it.

1. separate context switching and thread scheduling.
(cf. gmcgarry_ctxsw)
2. implement idle lwp.
3. clean up related MD/MI interfaces.
4. make scheduler(s) modular.

Remove proc->p_ru and the 'rusage' pool.
I think it existed to cache the numbers in kernel memory of a zombie when
proc->p_stats was part of the 'u' area - so got freed earlier and wouldn't
(easily) be accessible from a separate process. However since both the
p_ru and p_stats fields are freed at the same time it is no longer needed.
Ride the recent 4.99.19 version change.

branches: 1.99.2;
Add the LWP's runtime to kinfo_lwp.

branches: 1.98.2;
- Make the proclist_lock a mutex. The write:read ratio is unfavourable,
and mutexes are cheaper use than RW locks.
- LOCK_ASSERT -> KASSERT in some places.
- Hold proclist_lock/kernel_lock longer in a couple of places.

Change the process/lwp flags seen by userland via sysctl back to the
P_*/L_* naming convention, and rename the in-kernel flags to avoid
conflict. (P_ -> PK_, L_ -> LW_ ). Add back the (now unused) LSDEAD
constant.

Restores source compatibility with pre-newlock2 tools like ps or top.

Reviewed by Andrew Doran.

branches: 1.96.2;
Count the number of CPUs at boot and stash in 'ncpu'. Eventually should
have each CPU register at attach, so we can figure out the topology for
the scheduler.

Merge newlock2 to head.

Don't rely on KAUTH_PROCESS_CANSEE for environment just yet,
otherwise we're allowing anyone to read the environment unless
curtain is enabled.

From yamt@.

branches: 1.93.2;
Move Veriexec's sysctl(9) setup routine and helper to kern_verifiedexec.c.

PR/34837: Mindaguas: Add SysV SHM dynamic reallocation and locking to the
physical memory

implement kern.arandom properly, instead of lying about it and only filling
the first 4 bytes of the array with random data.

add the emulation in kinfo_proc2

Back out previous (p_flag2).

In 30 minutes from now Jason Thorpe will come up with an implementation
of a proplib dictionary in struct proc, so adding an int doesn't really
make any sense.

Until we figure out the Perfect Way of adding flags to processes, add
a p_flag2. No objections on tech-kern@.

Input from simonb@, thanks!

correct dcopyout #define for !KTRACE case.

Add a -t+S flag to ktrace for tracing activity related to sysctl. MIB
names will be displayed, with data readen and written as well.

branches: 1.85.2;
Don't use KAUTH_RESULT_* where it's not applicable.
Prompted by yamt@.

When getting the program argument or environement string, we previously
assumed that all the strings were stored in a row, separated by NUL chars,
at the address pointed bu argv[0] (or envp[0]).

This was wrong: if the program changed argvs[0], we still read the
first string correctly, but the next strings did contain unexpected data.

The fix: read the whole argv (or envp) array, then copy the string one by
one, using their addresses in argv (or agrp)

First take at security model abstraction.

- Add a few scopes to the kernel: system, network, and machdep.

- Add a few more actions/sub-actions (requests), and start using them as
opposed to the KAUTH_GENERIC_ISSUSER place-holders.

- Introduce a basic set of listeners that implement our "traditional"
security model, called "bsd44". This is the default (and only) model we
have at the moment.

- Update all relevant documentation.

- Add some code and docs to help folks who want to actually use this stuff:

* There's a sample overlay model, sitting on-top of "bsd44", for
fast experimenting with tweaking just a subset of an existing model.

This is pretty cool because it's *really* straightforward to do stuff
you had to use ugly hacks for until now...

* And of course, documentation describing how to do the above for quick
reference, including code samples.

All of these changes were tested for regressions using a Python-based
testsuite that will be (I hope) available soon via pkgsrc. Information
about the tests, and how to write new ones, can be found on:

http://kauth.linbsd.org/kauthwiki

NOTE FOR DEVELOPERS: *PLEASE* don't add any code that does any of the
following:

- Uses a KAUTH_GENERIC_ISSUSER kauth(9) request,
- Checks 'securelevel' directly,
- Checks a uid/gid directly.

(or if you feel you have to, contact me first)

This is still work in progress; It's far from being done, but now it'll
be a lot easier.

Relevant mailing list threads:

http://mail-index.netbsd.org/tech-security/2006/01/25/0011.html
http://mail-index.netbsd.org/tech-security/2006/03/24/0001.html
http://mail-index.netbsd.org/tech-security/2006/04/18/0000.html
http://mail-index.netbsd.org/tech-security/2006/05/15/0000.html
http://mail-index.netbsd.org/tech-security/2006/08/01/0000.html
http://mail-index.netbsd.org/tech-security/2006/08/25/0000.html

Many thanks to YAMAMOTO Takashi, Matt Thomas, and Christos Zoulas for help
stablizing kauth(9).

Full credit for the regression tests, making sure these changes didn't break
anything, goes to Matt Fleming and Jaime Fournier.

Happy birthday Randi! :)

When colecting a 32 bit process' argument or environement vector, we need
to convert 32 bits pointers to the 64 bit environement

branches: 1.81.4;
at the request of elad, as veriexec.h has returned, revert the changes
from 2006-07-25.

mechanically go through and
s,include "veriexec.h",include <sys/verified_exec.h>,
as the former has apparently gone away.

some fixes:
- adapt to NVERIEXEC in init_sysctl.c.
- we now need "veriexec.h" for NVERIEXEC.
- "opt_verified_exec.h" -> "opt_veriexec.h", and include it only where
it is needed.

Use the LWP cached credentials where sane.

- Don't cast kauth_cred_t to (struct ucred *), just set pc_ucred = NULL.
- Fill ucred::cr_ref.

CURTAIN() -> KAUTH_GENERIC_CANSEE.

move security.setid_core.* to kern.coredump.setid.*, as requested by yamt@.

Don't leak memory on success. Allocate only the type of struct that we'll
need for efficiency.

don't allocate too much stuff on the stack.