Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r253078: audit_proc_coredump: check return value of audit_new

Approved by: re (kib)

MFC: r227309 (partial)

Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs.

The SYSCTL_NODE macro defines a list that stores all child-elements of
that node. If there's no SYSCTL_DECL macro anywhere else, there's no
reason why it shouldn't be static.

MFC r243719,r243720,r243722,r243723,r243726,r243727,r243746:

r243719:

IFp4 @208450:

Remove redundant call to AUDIT_ARG_UPATH1().
Path will be remembered by the following NDINIT(AUDITVNODE1) call.

Sponsored by: The FreeBSD Foundation (auditdistd)

r243720:

IFp4 @208381:

For VOP_GETATTR() we just need vnode to be shared-locked.

Sponsored by: The FreeBSD Foundation (auditdistd)

r243722:

IFp4 @208382:

Currently on each record write we call VFS_STATFS() to get available space
on the file system as well as VOP_GETATTR() to get trail file size.

We can assume that trail file is only updated by the audit worker, so instead
of asking for file size on every write, get file size on trail switch only
(it should be zero, but it's not expensive) and use global variable audit_size
protected by the audit worker lock to keep track of trail file's size.

This eliminates VOP_GETATTR() call for every write. VFS_STATFS() is satisfied
from in-memory data (mount->mnt_stat), so shouldn't be expensive.

Sponsored by: The FreeBSD Foundation (auditdistd)

r243723:

IFp4 @208383:

Currently when we discover that trail file is greater than configured
limit we send AUDIT_TRIGGER_ROTATE_KERNEL trigger to the auditd daemon
once. If for some reason auditd didn't rotate trail file it will never
be rotated.

Change it by sending the trigger when trail file size grows by the
configured limit. For example if the limit is 1MB, we will send trigger
on 1MB, 2MB, 3MB, etc.

This is also needed for the auditd change that will be committed soon
where auditd may ignore the trigger - it might be ignored if kernel
requests the trail file to be rotated too quickly (often than once a second)
which would result in overwriting previous trail file.

Sponsored by: The FreeBSD Foundation (auditdistd)

r243726:

IFp4 @208451:

Fix path handling for *at() syscalls.

Before the change directory descriptor was totally ignored,
so the relative path argument was appended to current working
directory path and not to the path provided by descriptor, thus
wrong paths were stored in audit logs.

Now that we use directory descriptor in vfs_lookup, move
AUDIT_ARG_UPATH1() and AUDIT_ARG_UPATH2() calls to the place where
we hold file descriptors table lock, so we are sure paths will
be resolved according to the same directory in audit record and
in actual operation.

Sponsored by: The FreeBSD Foundation (auditdistd)
Reviewed by: rwatson

r243727:

IFp4 @208452:

Audit handling for missing events:
- AUE_READLINKAT
- AUE_FACCESSAT
- AUE_MKDIRAT
- AUE_MKFIFOAT
- AUE_MKNODAT
- AUE_SYMLINKAT

Sponsored by: FreeBSD Foundation (auditdistd)

r243746:

Fix one more compilation issue.

Sponsored by: FreeBSD Foundation (auditdistd)

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Add some FEATURE macros for various features (AUDIT/CAM/IPC/KTR/MAC/NFS/NTP/
PMC/SYSV/...).

No FreeBSD version bump, the userland application to query the features will
be committed last and can serve as an indication of the availablility if
needed.

Sponsored by: Google Summer of Code 2010
Submitted by: kibab
Reviewed by: arch@ (parts by rwatson, trasz, jhb)
X-MFC after: to be determined in last commit with code from this project

Audit file descriptors passed to fooat(2) system calls, which are used
instead of the root/current working directory as the starting point for
lookups. Up to two such descriptors can be audited. Add audit record
BSM encoding for fooat(2).

Note: due to an error in the OpenBSM 1.1p1 configuration file, a
further change is required to that file in order to fix openat(2)
auditing.

Approved by: re (kib)
Reviewed by: rdivacky (fooat(2) portions)
Obtained from: TrustedBSD Project
MFC after: 1 month

Dynamically allocate the gidset field in audit record.

This fixes a problem created by the recent change that allows a large
number of groups per user. The gidset field in struct kaudit_record
is now dynamically allocated to the size needed rather than statically
(using NGROUPS).

Approved by: re@ (kensmith, rwatson), gnn (mentor)

Merge OpenBSM 1.1 changes to the FreeBSD 8.x kernel:

- Add and use mapping of fcntl(2) commands to new BSM constant space.
- Adopt (int) rather than (long) arguments to a number of auditon(2)
commands, as has happened in Solaris, and add compatibility code to
handle the old comments.

Note that BSM_PF_IEEE80211 is partially but not fully removed, as the
userspace OpenBSM 1.1alpha5 code still depends on it. Once userspace
is updated, I'll GCC the kernel constant.

MFC after: 2 weeks
Sponsored by: Apple, Inc.
Obtained from: TrustedBSD Project
Portions submitted by: sson

Add a new thread-private flag, TDP_AUDITREC, to indicate whether or
not there is an audit record hung off of td_ar on the current thread.
Test this flag instead of td_ar when auditing syscall arguments or
checking for an audit record to commit on syscall return. Under
these circumstances, td_pflags is much more likely to be in the cache
(especially if there is no auditing of the current system call), so
this should help reduce cache misses in the system call return path.

MFC after: 1 week
Reported by: kris
Obtained from: TrustedBSD Project

When repeatedly accessing a thread credential, cache the credential
pointer in a local thread. While this is unlikely to significantly
improve performance given modern compiler behavior, it makes the code
more readable and reduces diffs to the Mac OS X version of the same
code (which stores things in creds in the same way, but where the
cred for a thread is reached quite differently).

Discussed with: sson
MFC after: 1 month
Sponsored by: Apple Inc.
Obtained from: TrustedBSD Project

The audit queue limit variables are size_t, so use size_t for the audit
queue length variables as well, avoiding storing the limit in a larger
type than the length.

Submitted by: sson
Sponsored by: Apple Inc.
MFC after: 1 week

Minor style tweaks and change lock name string to use _'s and not spaces
to improve parseability.

Add support for extended header BSM tokens. Currently we use the
regular header tokens. The extended header tokens contain an IP
or IPv6 address which makes it possible to identify which host an
audit record came from when audit records are centralized.

If the host information has not been specified, the system will
default to the old style headers. Otherwise, audit records that
are created as a result of system calls will contain host information.

This implemented has been designed to be consistent with the Solaris
implementation. Host information is set/retrieved using the A_GETKAUDIT
and A_SETKAUDIT auditon(2) commands. These commands require that a
pointer to a auditinfo_addr_t object is passed. Currently only IP and
IPv6 address families are supported.

The users pace bits associated with this change will follow in an
openbsm import.

Reviewed by: rwatson, (sson, wsalamon (older version))
MFC after: 1 month

Make sure we check the preselection masks present for all audit pipes.
It is possible that the audit pipe(s) have different preselection configs
then the global preselection mask.

Spotted by: Vincenzo Iozzo
MFC after: 2 weeks

Further synchronization of copyrights, licenses, white space, etc from
Apple and from the OpenBSM vendor tree.

Obtained from: Apple Inc., TrustedBSD Project
MFC after: 3 days

In preparation to sync Apple and FreeBSD versions of security audit,
pick up the Apple Computer -> Apple change in their copyright and
license templates.

Obtained from: Apple Inc.
MFC after: 3 days

Add an XXX comment regarding a bug I introduced when modifying the behavior
of audit log vnode rotation: on shutdown, we may not properly drain all
pending records, which could lead to lost records during system shutdown.

Use __FBSDID() for $FreeBSD$ IDs in the audit code.

MFC after: 3 days

In keeping with style(9)'s recommendations on macros, use a ';'
after each SYSINIT() macro invocation. This makes a number of
lightweight C parsers much happier with the FreeBSD kernel
source, including cflow's prcc and lxr.

MFC after: 1 month
Discussed with: imp, rink

Add audit_prefixes to two more globally visible functions in the Audit
implementation.

MFC after: 1 month

Rename several audit functions in the global kernel symbol namespace to
have audit_ on the front:

- canon_path -> audit_canon_path
- msgctl_to_event -> audit_msgctl_to_event
- semctl_to_event -> audit_semctl_to_event

MFC after: 1 month

Make sure that the termid type is initialized to AU_IPv4 by default.
This makes sure that process tokens credentials with un-initialized
audit contexts are handled correctly. Currently, when invariants are
enabled, this change fixes a panic by ensuring that we have a valid
termid family. Also, this fixes token generation for process tokens
making sure that userspace is always getting a valid token.

This is consistent with what Solaris does when an audit context is
un-initialized.

Obtained from: TrustedBSD Project
MFC after: 1 week

Explicitly initialize 'ret' to 0'. It lets one to build tmpfs from the
latest source tree with older compiler--gcc3.

Approved by: cognet (mentor)

Replace use of AU_NULL with 0 when no audit classes are in use; this
supports the removal of hard-coded audit class constants in OpenBSM
1.0. All audit classes are now dynamically configured via the
audit_class database.

Obtained from: TrustedBSD Project

Implement AUE_CORE, which adds process core dump support into the kernel.
This change introduces audit_proc_coredump() which is called by coredump(9)
to create an audit record for the coredump event. When a process
dumps a core, it could be security relevant. It could be an indicator that
a stack within the process has been overflowed with an incorrectly constructed
malicious payload or a number of other events.

The record that is generated looks like this:

header,111,10,process dumped core,0,Thu Oct 25 19:36:29 2007, + 179 msec
argument,0,0xb,signal
path,/usr/home/csjp/test.core
subject,csjp,csjp,staff,csjp,staff,1101,1095,50457,10.37.129.2
return,success,1
trailer,111

- We allocate a completely new record to make sure we arent clobbering
the audit data associated with the syscall that produced the core
(assuming the core is being generated in response to SIGABRT and not
an invalid memory access).
- Shuffle around expand_name() so we can use the coredump name at the very
beginning of the coredump call. Make sure we free the storage referenced
by "name" if we need to bail out early.
- Audit both successful and failed coredump creation efforts

Obtained from: TrustedBSD Project
Reviewed by: rwatson
MFC after: 1 month

Remove two boot printfs generated by Audit to announce it's presence,
and replace with software-testable sysctl node (security.audit) that
can be used to detect kernel audit support.

Obtained from: TrustedBSD Project
Approved by: re (kensmith)

Spell statistics more correctly in comments.

Clean up, and sometimes remove, a number of audit-related implementation
comments.

Obtained from: TrutstedBSD Project

Move per-process audit state from a pointer in the proc structure to
embedded storage in struct ucred. This allows audit state to be cached
with the thread, avoiding locking operations with each system call, and
makes it available in asynchronous execution contexts, such as deep in
the network stack or VFS.

Reviewed by: csjp
Approved by: re (kensmith)
Obtained from: TrustedBSD Project

Clean up audit comments--formatting, spelling, etc.

Remove AUDIT_PRINTF() debugging statements and definition; clean up or
remove associated comments.

Slip audit_file_rotate_wait assignment in audit_rotate_vnode() before
the drop of the global audit mutex.

Obtained from: TrustedBSD Project

Remove unused ar_subj_comm field from in-kernel audit record; we never
export this via BSM, so don't pay space/time cost of maintaining it.

Obtained from: TrustedBSD Project

No need to force __inline__ of currecord(), as the compiler will usefully
inline it when needed already, and the symbol is also required outside of
audit.c. This silences a new gcc warning on the topic of using __inline__
instead of __inline.

MFC after: 3 days

Fix the handling of IPv6 addresses for subject and process BSM audit
tokens. Currently, we do not support the set{get}audit_addr(2) system
calls which allows processes like sshd to set extended or ip6
information for subject tokens.

The approach that was taken was to change the process audit state
slightly to use an extended terminal ID in the kernel. This allows
us to store both IPv4 IPv6 addresses. In the case that an IPv4 address
is in use, we convert the terminal ID from an struct auditinfo_addr to
a struct auditinfo.

If getaudit(2) is called when the subject is bound to an ip6 address,
we return E2BIG.

- Change the internal audit record to store an extended terminal ID
- Introduce ARG_TERMID_ADDR
- Change the kaudit <-> BSM conversion process so that we are using
the appropriate subject token. If the address associated with the
subject is IPv4, we use the standard subject32 token. If the subject
has an IPv6 address associated with them, we use an extended subject32
token.
- Fix a couple of endian issues where we do a couple of byte swaps when
we shouldn't be. IP addresses are already in the correct byte order,
so reading the ip6 address 4 bytes at a time and swapping them results
in in-correct address data. It should be noted that the same issue was
found in the openbsm library and it has been changed there too on the
vendor branch
- Change A_GETPINFO to use the appropriate structures
- Implement A_GETPINFO_ADDR which basically does what A_GETPINFO does,
but can also handle ip6 addresses
- Adjust get{set}audit(2) syscalls to convert the data
auditinfo <-> auditinfo_addr
- Fully implement set{get}audit_addr(2)

NOTE: This adds the ability for processes to correctly set extended subject
information. The appropriate userspace utilities still need to be updated.

MFC after: 1 month
Reviewed by: rwatson
Obtained from: TrustedBSD

Remove 'MPSAFE' annotations from the comments above most system calls: all
system calls now enter without Giant held, and then in some cases, acquire
Giant explicitly.

Remove a number of other MPSAFE annotations in the credential code and
tweak one or two other adjacent comments.

Update a number of comments:

- Replace XXX with Note: in several cases where observations are made about
future functionality rather than problems or bugs.

- Remove an XXX comment about byte order and au_to_ip() -- IP headers must
be submitted in network byte order. Add a comment to this effect.

- Mention that we don't implement select/poll for /dev/audit.

Obtained from: TrustedBSD Project

Sweep kernel replacing suser(9) calls with priv(9) calls, assigning
specific privilege names to a broad range of privileges. These may
require some future tweaking.

Sponsored by: nCircle Network Security, Inc.
Obtained from: TrustedBSD Project
Discussed on: arch@
Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri,
Alex Lyashkov <umka at sevcity dot net>,
Skip Ford <skip dot ford at verizon dot net>,
Antoine Brodin <antoine dot brodin at laposte dot net>

Trim some no longer XXX comments.
Remove some commented out debugging printfs.

MFC after: 3 days
Obtained from: TrustedBSD Project

Correct a slight regression which was introduced with the implementation of
audit pipes. If the kernel record was not selected for the trail or the pipe,
any user supplied record attached to it would be tossed away, resulting in
otherwise selected events being lost.

- Introduce two new masks: AR_PRESELECT_USER_TRAIL AR_PRESELECT_USER_PIPE,
currently we have AR_PRESELECT_TRAIL and AR_PRESELECT_PIPE, which tells
the audit worker that we are interested in the kernel record, with
the additional masks we can determine if either the pipe or trail is
interested in seeing the kernel or user record.

- In audit(2), we unconditionally set the AR_PRESELECT_USER_TRAIL and
AR_PRESELECT_USER_PIPE masks under the assumption that userspace has
done the preselection [1].

Currently, there is work being done that allows the kernel to parse and
preselect user supplied records, so in the future preselection could occur
in either layer. But there is still a few details to work out here.

[1] At some point we need to teach au_preselect(3) about the interests of
all the individual audit pipes.

This is a RELENG_6 candidate.

Reviewed by: rwatson
Obtained from: TrustedBSD Project
MFC after: 1 week

Small style cleanup.

MFC after: 3 days

Audit the argv and env vectors passed in on exec:
Add the argument auditing functions for argv and env.
Add kernel-specific versions of the tokenizer functions for the
arg and env represented as a char array.
Implement the AUDIT_ARGV and AUDIT_ARGE audit policy commands to
enable/disable argv/env auditing.
Call the argument auditing from the exec system calls.

Obtained from: TrustedBSD Project
Approved by: rwatson (mentor)

Lock process when copying fields from process structure so as to
get a consistent snapshot, as well as get consistent values (i.e.,
that p_comm is properly nul-terminated).

Perforce CID: 98824
Obtained from: TrustedBSD Project

Consistently use audit_free() to free records, rather than
directly invoking uma_zfree().

Perforce change: 96652
Obtained from: TrustedBSD Project

Introduce support for per-audit pipe preselection independent from the
global audit trail configuration. This allows applications consuming
audit trails to specify parameters for which audit records are of
interest, including selecting records not required by the global trail.
Allowing application interest specification without changing the global
configuration allows intrusion detection systems to run without
interfering with global auditing or each other (if multiple are
present). To implement this:

- Kernel audit records now carry a flag to indicate whether they have
been selected by the global trail or by the audit pipe subsystem,
set during record commit, so that this information is available
after BSM conversion when delivering the BSM to the trail and audit
pipes in the audit worker thread asynchronously. Preselection by
either record target will cause the record to be kept.

- Similar changes to preselection when the audit record is created
when the system call is entering: consult both the global trail and
pipes.

- au_preselect() now accepts the class in order to avoid repeatedly
looking up the mask for each preselection test.

- Define a series of ioctls that allow applications to specify whether
they want to track the global trail, or program their own
preselection parameters: they may specify their own flags and naflags
masks, similar to the global masks of the same name, as well as a set
of per-auid masks. They also set a per-pipe mode specifying whether
they track the global trail, or user their own -- the door is left
open for future additional modes. A new ioctl is defined to allow a
user process to flush the current audit pipe queue, which can be used
after reprogramming pre-selection to make sure that only records of
interest are received in future reads.

- Audit pipe data structures are extended to hold the additional fields
necessary to support preselection. By default, audit pipes track the
global trail, so "praudit /dev/auditpipe" will track the global audit
trail even though praudit doesn't program the audit pipe selection
model.

- Comment about the complexities of potentially adding partial read
support to audit pipes.

By using a set of ioctls, applications can select which records are of
interest, and toggle the preselection mode.

Obtained from: TrustedBSD Project

Shorten audit record zone name.

Perforce change: 93598
Obtained from: TrustedBSD Project

Rename audit_cv to audit_worker_cv, as it wakes up the audit
worker.

Rename audit_commit_cv to audit_watermark_cv, since it is there to
wake up threads waiting on hitting the low watermark. Describe
properly in comment.

Obtained from: TrustedBSD Project

Merge Perforce change 93581 from TrustedBSD audit3 branch:

Mega-style patch.

Obtained from: TrustedBSD Project

Merge Perforce changes 93512, 93514, 93515 from TrustedBSD audit3
branch:

Integrate audit.c to audit_worker.c, so as to migrate the worker
thread implementation to its own .c file.

Populate audit_worker.c using parts now removed from audit.c:

- Move audit rotation global variables.
- Move audit_record_write(), audit_worker_rotate(),
audit_worker_drain(), audit_worker(), audit_rotate_vnode().
- Create audit_worker_init() from relevant parts of audit_init(),
which now calls this routine.
- Recreate audit_free(), which wraps uma_zfree() so that
audit_record_zone can be static to audit.c.
- Unstaticize various types and variables relating to the audit
record queue so that audit_worker can get to them. We may want
to wrap these in accessor methods at some point.
- Move AUDIT_PRINTF() to audit_private.h.

Addition of audit_worker.c to kernel configuration, missed in
earlier submit.

Obtained from: TrustedBSD Project

Merge perforce 93507:

Correct comment: this print is now from audit_record_write(), not
audit_worker().

Obtained from: TrustedBSD Project

Merge perforce change 93199:

Change send_trigger() prototype to return an int, so that user
space callers can tell if the message was successfully placed
in the trigger queue. This isn't quite the same as it being
successfully received, but is close enough that we can generate
a more useful warning message in audit(8).

Obtained from: TrustedBSD Project

Update src/sys/security/audit for OpenBSM 1.0 alpha 5:

- Include audit_internal.h to get definition of internal audit record
structures, as it's no longer in audit.h. Forward declare au_record
in audit_private.h as not all audit_private.h consumers care about
it.

- Remove __APPLE__ compatibility bits that are subsumed by configure
for user space.

- Don't expose in6_addr internals (non-portable, but also cleaner
looking).

- Avoid nested include of audit.h in audit_private.h.

Obtained from: TrustedBSD Project

Initialize user process audit ID to AU_DEFAUDITID so that init and
its pre-authentication children are covered by naflags.

Obtained from: TrustedBSD Project

Acquire vnode lock around call to VOP_GETATTR() in audit_record_write().
In the future, we may want to acquire the lock early in the function and
hold it across calls to vn_rdwr(), etc, to avoid multiple acquires.

Spotted by: kris (bugmagnet)
Obtained from: TrustedBSD Project

Add support for audit pipe special devices, which allow user space
applications to insert a "tee" in the live audit event stream. Records
are inserted into a per-clone queue so that user processes can pull
discreet records out of the queue. Unlike delivery to disk, audit pipes
are "lossy", dropping records in low memory conditions or when the
process falls behind real-time events. This mechanism is appropriate
for use by live monitoring systems, host-based intrusion detection, etc,
and avoids applications having to dig through active on-disk trails that
are owned by the audit daemon.

Obtained from: TrustedBSD Project

Manage audit record memory with the slab allocator, turning
initialization routines into a ctor, tear-down to a dtor, cleaning
up, etc. This will allow audit records to be allocated from
per-cpu caches.

On recent FreeBSD, dropping the audit_mtx around freeing to UMA is
no longer required (at one point it was possible to acquire Giant
on that path), so a mutex-free thread-local drain is no longer
required.

Obtained from: TrustedBSD Project

When GC'ing a thread, assert that it has no active audit record.
This should not happen, but with this assert, brueffer and I would
not have spent 45 minutes trying to figure out why he wasn't
seeing audit records with the audit version in CVS.

Obtained from: TrustedBSD Project

Add new fields to process-related data structures:

- td_ar to struct thread, which holds the in-progress audit record during
a system call.

- p_au to struct proc, which holds per-process audit state, such as the
audit identifier, audit terminal, and process audit masks.

In the earlier implementation, td_ar was added to the zero'd section of
struct thread. In order to facilitate merging to RELENG_6, it has been
moved to the end of the data structure, requiring explicit
initalization in the thread constructor.

Much help from: wsalamon
Obtained from: TrustedBSD Project

Import kernel audit framework:

- Management of audit state on processes.
- Audit system calls to configure process and system audit state.
- Reliable audit record queue implementation, audit_worker kernel
thread to asynchronously store records on disk.
- Audit event argument.
- Internal audit data structure -> BSM audit trail conversion library.
- Audit event pre-selection.
- Audit pseudo-device permitting kernel->user upcalls to notify auditd
of kernel audit events.

Much work by: wsalamon
Obtained from: TrustedBSD Project, Apple Computer, Inc.